Merge tag 'mac80211-next-for-davem-2019-09-11' of git://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211-next
Johannes Berg says:
====================
We have a number of changes, but things are settling down:
* a fix in the new 6 GHz channel support
* a fix for recent minstrel (rate control) updates
for an infinite loop
* handle interface type changes better wrt. management frame
registrations (for management frames sent to userspace)
* add in-BSS RX time to survey information
* handle HW rfkill properly if !CONFIG_RFKILL
* send deauth on IBSS station expiry, to avoid state mismatches
* handle deferred crypto tailroom updates in mac80211 better
when device restart happens
* fix a spectre-v1 - really a continuation of a previous patch
* advertise NL80211_CMD_UPDATE_FT_IES as supported if so
* add some missing parsing in VHT extended NSS support
* support HE in mac80211_hwsim
* let mac80211 drivers determine the max MTU themselves
along with the usual cleanups etc.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
diff --git a/.mailmap b/.mailmap
index acba1a61..dfd7fed 100644
--- a/.mailmap
+++ b/.mailmap
@@ -63,7 +63,11 @@
Dengcheng Zhu <dzhu@wavecomp.com> <dengcheng.zhu@imgtec.com>
Dengcheng Zhu <dzhu@wavecomp.com> <dczhu@mips.com>
Dengcheng Zhu <dzhu@wavecomp.com> <dengcheng.zhu@gmail.com>
+<dev.kurt@vandijck-laurijssen.be> <kurt.van.dijck@eia.be>
Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>
+Dmitry Safonov <0x7f454c46@gmail.com> <dsafonov@virtuozzo.com>
+Dmitry Safonov <0x7f454c46@gmail.com> <d.safonov@partner.samsung.com>
+Dmitry Safonov <0x7f454c46@gmail.com> <dima@arista.com>
Domen Puncer <domen@coderock.org>
Douglas Gilbert <dougg@torque.net>
Ed L. Cashin <ecashin@coraid.com>
@@ -160,6 +164,8 @@
Matt Ranostay <mranostay@gmail.com> <matt.ranostay@intel.com>
Matt Ranostay <matt.ranostay@konsulko.com> <matt@ranostay.consulting>
Matt Redfearn <matt.redfearn@mips.com> <matt.redfearn@imgtec.com>
+Maxime Ripard <mripard@kernel.org> <maxime.ripard@bootlin.com>
+Maxime Ripard <mripard@kernel.org> <maxime.ripard@free-electrons.com>
Mayuresh Janorkar <mayur@ti.com>
Michael Buesch <m@bues.ch>
Michel Dänzer <michel@tungstengraphics.com>
diff --git a/Documentation/PCI/index.rst b/Documentation/PCI/index.rst
index f4c6121..6768305 100644
--- a/Documentation/PCI/index.rst
+++ b/Documentation/PCI/index.rst
@@ -9,7 +9,7 @@
:numbered:
pci
- picebus-howto
+ pciebus-howto
pci-iov-howto
msi-howto
acpi-info
diff --git a/Documentation/PCI/picebus-howto.rst b/Documentation/PCI/pciebus-howto.rst
similarity index 100%
rename from Documentation/PCI/picebus-howto.rst
rename to Documentation/PCI/pciebus-howto.rst
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 47d981a..4c19719 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4090,6 +4090,13 @@
Run specified binary instead of /init from the ramdisk,
used for early userspace startup. See initrd.
+ rdrand= [X86]
+ force - Override the decision by the kernel to hide the
+ advertisement of RDRAND support (this affects
+ certain AMD processors because of buggy BIOS
+ support, specifically around the suspend/resume
+ path).
+
rdt= [HW,X86,RDT]
Turn on/off individual RDT features. List is:
cmt, mbmtotal, mbmlocal, l3cat, l3cdp, l2cat, l2cdp,
diff --git a/Documentation/devicetree/bindings/arm/mediatek/mediatek,sgmiisys.txt b/Documentation/devicetree/bindings/arm/mediatek/mediatek,sgmiisys.txt
index f5518f2..30cb645 100644
--- a/Documentation/devicetree/bindings/arm/mediatek/mediatek,sgmiisys.txt
+++ b/Documentation/devicetree/bindings/arm/mediatek/mediatek,sgmiisys.txt
@@ -9,8 +9,6 @@
- "mediatek,mt7622-sgmiisys", "syscon"
- "mediatek,mt7629-sgmiisys", "syscon"
- #clock-cells: Must be 1
-- mediatek,physpeed: Should be one of "auto", "1000" or "2500" to match up
- the capability of the target PHY.
The SGMIISYS controller uses the common clk binding from
Documentation/devicetree/bindings/clock/clock-bindings.txt
diff --git a/Documentation/devicetree/bindings/interrupt-controller/snps,archs-idu-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/snps,archs-idu-intc.txt
index 09fc02b..a5c1db9 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/snps,archs-idu-intc.txt
+++ b/Documentation/devicetree/bindings/interrupt-controller/snps,archs-idu-intc.txt
@@ -1,20 +1,30 @@
* ARC-HS Interrupt Distribution Unit
- This optional 2nd level interrupt controller can be used in SMP configurations for
- dynamic IRQ routing, load balancing of common/external IRQs towards core intc.
+ This optional 2nd level interrupt controller can be used in SMP configurations
+ for dynamic IRQ routing, load balancing of common/external IRQs towards core
+ intc.
Properties:
- compatible: "snps,archs-idu-intc"
- interrupt-controller: This is an interrupt controller.
-- #interrupt-cells: Must be <1>.
+- #interrupt-cells: Must be <1> or <2>.
- Value of the cell specifies the "common" IRQ from peripheral to IDU. Number N
- of the particular interrupt line of IDU corresponds to the line N+24 of the
- core interrupt controller.
+ Value of the first cell specifies the "common" IRQ from peripheral to IDU.
+ Number N of the particular interrupt line of IDU corresponds to the line N+24
+ of the core interrupt controller.
- intc accessed via the special ARC AUX register interface, hence "reg" property
- is not specified.
+ The (optional) second cell specifies any of the following flags:
+ - bits[3:0] trigger type and level flags
+ 1 = low-to-high edge triggered
+ 2 = NOT SUPPORTED (high-to-low edge triggered)
+ 4 = active high level-sensitive <<< DEFAULT
+ 8 = NOT SUPPORTED (active low level-sensitive)
+ When no second cell is specified, the interrupt is assumed to be level
+ sensitive.
+
+ The interrupt controller is accessed via the special ARC AUX register
+ interface, hence "reg" property is not specified.
Example:
core_intc: core-interrupt-controller {
diff --git a/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.yaml b/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.yaml
index 06b1cc8..ef446ae 100644
--- a/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.yaml
+++ b/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.yaml
@@ -17,6 +17,9 @@
compatible:
const: allwinner,sun7i-a20-gmac
+ reg:
+ maxItems: 1
+
interrupts:
maxItems: 1
diff --git a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
new file mode 100644
index 0000000..ae91aa9
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
@@ -0,0 +1,113 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+# Copyright 2019 BayLibre, SAS
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/net/amlogic,meson-dwmac.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Amlogic Meson DWMAC Ethernet controller
+
+maintainers:
+ - Neil Armstrong <narmstrong@baylibre.com>
+ - Martin Blumenstingl <martin.blumenstingl@googlemail.com>
+
+# We need a select here so we don't match all nodes with 'snps,dwmac'
+select:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - amlogic,meson6-dwmac
+ - amlogic,meson8b-dwmac
+ - amlogic,meson8m2-dwmac
+ - amlogic,meson-gxbb-dwmac
+ - amlogic,meson-axg-dwmac
+ required:
+ - compatible
+
+allOf:
+ - $ref: "snps,dwmac.yaml#"
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - amlogic,meson8b-dwmac
+ - amlogic,meson8m2-dwmac
+ - amlogic,meson-gxbb-dwmac
+ - amlogic,meson-axg-dwmac
+
+ then:
+ properties:
+ clocks:
+ items:
+ - description: GMAC main clock
+ - description: First parent clock of the internal mux
+ - description: Second parent clock of the internal mux
+
+ clock-names:
+ minItems: 3
+ maxItems: 3
+ items:
+ - const: stmmaceth
+ - const: clkin0
+ - const: clkin1
+
+ amlogic,tx-delay-ns:
+ $ref: /schemas/types.yaml#definitions/uint32
+ description:
+ The internal RGMII TX clock delay (provided by this driver) in
+ nanoseconds. Allowed values are 0ns, 2ns, 4ns, 6ns.
+ When phy-mode is set to "rgmii" then the TX delay should be
+ explicitly configured. When not configured a fallback of 2ns is
+ used. When the phy-mode is set to either "rgmii-id" or "rgmii-txid"
+ the TX clock delay is already provided by the PHY. In that case
+ this property should be set to 0ns (which disables the TX clock
+ delay in the MAC to prevent the clock from going off because both
+ PHY and MAC are adding a delay).
+ Any configuration is ignored when the phy-mode is set to "rmii".
+
+properties:
+ compatible:
+ additionalItems: true
+ maxItems: 3
+ items:
+ - enum:
+ - amlogic,meson6-dwmac
+ - amlogic,meson8b-dwmac
+ - amlogic,meson8m2-dwmac
+ - amlogic,meson-gxbb-dwmac
+ - amlogic,meson-axg-dwmac
+ contains:
+ enum:
+ - snps,dwmac-3.70a
+ - snps,dwmac
+
+ reg:
+ items:
+ - description:
+ The first register range should be the one of the DWMAC controller
+ - description:
+ The second range is is for the Amlogic specific configuration
+ (for example the PRG_ETHERNET register range on Meson8b and newer)
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - interrupt-names
+ - clocks
+ - clock-names
+ - phy-mode
+
+examples:
+ - |
+ ethmac: ethernet@c9410000 {
+ compatible = "amlogic,meson-gxbb-dwmac", "snps,dwmac";
+ reg = <0xc9410000 0x10000>, <0xc8834540 0x8>;
+ interrupts = <8>;
+ interrupt-names = "macirq";
+ clocks = <&clk_eth>, <&clkc_fclk_div2>, <&clk_mpll2>;
+ clock-names = "stmmaceth", "clkin0", "clkin1";
+ phy-mode = "rgmii";
+ };
diff --git a/Documentation/devicetree/bindings/net/broadcom-bluetooth.txt b/Documentation/devicetree/bindings/net/broadcom-bluetooth.txt
index c26f4e1..4fa00e2 100644
--- a/Documentation/devicetree/bindings/net/broadcom-bluetooth.txt
+++ b/Documentation/devicetree/bindings/net/broadcom-bluetooth.txt
@@ -13,6 +13,7 @@
* "brcm,bcm20702a1"
* "brcm,bcm4330-bt"
* "brcm,bcm43438-bt"
+ * "brcm,bcm4345c5"
Optional properties:
diff --git a/Documentation/devicetree/bindings/net/can/tcan4x5x.txt b/Documentation/devicetree/bindings/net/can/tcan4x5x.txt
index c388f7d..27e1b4c 100644
--- a/Documentation/devicetree/bindings/net/can/tcan4x5x.txt
+++ b/Documentation/devicetree/bindings/net/can/tcan4x5x.txt
@@ -10,8 +10,10 @@
- #size-cells: 0
- spi-max-frequency: Maximum frequency of the SPI bus the chip can
operate at should be less than or equal to 18 MHz.
- - data-ready-gpios: Interrupt GPIO for data and error reporting.
- device-wake-gpios: Wake up GPIO to wake up the TCAN device.
+ - interrupt-parent: the phandle to the interrupt controller which provides
+ the interrupt.
+ - interrupts: interrupt specification for data-ready.
See Documentation/devicetree/bindings/net/can/m_can.txt for additional
required property details.
@@ -30,7 +32,8 @@
#size-cells = <1>;
spi-max-frequency = <10000000>;
bosch,mram-cfg = <0x0 0 0 32 0 0 1 1>;
- data-ready-gpios = <&gpio1 14 GPIO_ACTIVE_LOW>;
+ interrupt-parent = <&gpio1>;
+ interrupts = <14 GPIO_ACTIVE_LOW>;
device-state-gpios = <&gpio3 21 GPIO_ACTIVE_HIGH>;
device-wake-gpios = <&gpio1 15 GPIO_ACTIVE_HIGH>;
reset-gpios = <&gpio1 27 GPIO_ACTIVE_LOW>;
diff --git a/Documentation/devicetree/bindings/net/dsa/ksz.txt b/Documentation/devicetree/bindings/net/dsa/ksz.txt
index 5e8429b..95e91e8 100644
--- a/Documentation/devicetree/bindings/net/dsa/ksz.txt
+++ b/Documentation/devicetree/bindings/net/dsa/ksz.txt
@@ -15,6 +15,7 @@
- "microchip,ksz8565"
- "microchip,ksz9893"
- "microchip,ksz9563"
+ - "microchip,ksz8563"
Optional properties:
diff --git a/Documentation/devicetree/bindings/net/dsa/mt7530.txt b/Documentation/devicetree/bindings/net/dsa/mt7530.txt
index 47aa205..c5ed5d2 100644
--- a/Documentation/devicetree/bindings/net/dsa/mt7530.txt
+++ b/Documentation/devicetree/bindings/net/dsa/mt7530.txt
@@ -35,6 +35,42 @@
- phy-mode: String, must be either "trgmii" or "rgmii" for port labeled
"cpu".
+Port 5 of the switch is muxed between:
+1. GMAC5: GMAC5 can interface with another external MAC or PHY.
+2. PHY of port 0 or port 4: PHY interfaces with an external MAC like 2nd GMAC
+ of the SOC. Used in many setups where port 0/4 becomes the WAN port.
+ Note: On a MT7621 SOC with integrated switch: 2nd GMAC can only connected to
+ GMAC5 when the gpios for RGMII2 (GPIO 22-33) are not used and not
+ connected to external component!
+
+Port 5 modes/configurations:
+1. Port 5 is disabled and isolated: An external phy can interface to the 2nd
+ GMAC of the SOC.
+ In the case of a build-in MT7530 switch, port 5 shares the RGMII bus with 2nd
+ GMAC and an optional external phy. Mind the GPIO/pinctl settings of the SOC!
+2. Port 5 is muxed to PHY of port 0/4: Port 0/4 interfaces with 2nd GMAC.
+ It is a simple MAC to PHY interface, port 5 needs to be setup for xMII mode
+ and RGMII delay.
+3. Port 5 is muxed to GMAC5 and can interface to an external phy.
+ Port 5 becomes an extra switch port.
+ Only works on platform where external phy TX<->RX lines are swapped.
+ Like in the Ubiquiti ER-X-SFP.
+4. Port 5 is muxed to GMAC5 and interfaces with the 2nd GAMC as 2nd CPU port.
+ Currently a 2nd CPU port is not supported by DSA code.
+
+Depending on how the external PHY is wired:
+1. normal: The PHY can only connect to 2nd GMAC but not to the switch
+2. swapped: RGMII TX, RX are swapped; external phy interface with the switch as
+ a ethernet port. But can't interface to the 2nd GMAC.
+
+Based on the DT the port 5 mode is configured.
+
+Driver tries to lookup the phy-handle of the 2nd GMAC of the master device.
+When phy-handle matches PHY of port 0 or 4 then port 5 set-up as mode 2.
+phy-mode must be set, see also example 2 below!
+ * mt7621: phy-mode = "rgmii-txid";
+ * mt7623: phy-mode = "rgmii";
+
See Documentation/devicetree/bindings/net/dsa/dsa.txt for a list of additional
required, optional properties and how the integrated switch subnodes must
be specified.
@@ -94,3 +130,181 @@
};
};
};
+
+Example 2: MT7621: Port 4 is WAN port: 2nd GMAC -> Port 5 -> PHY port 4.
+
+ð {
+ gmac0: mac@0 {
+ compatible = "mediatek,eth-mac";
+ reg = <0>;
+ phy-mode = "rgmii";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ pause;
+ };
+ };
+
+ gmac1: mac@1 {
+ compatible = "mediatek,eth-mac";
+ reg = <1>;
+ phy-mode = "rgmii-txid";
+ phy-handle = <&phy4>;
+ };
+
+ mdio: mdio-bus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* Internal phy */
+ phy4: ethernet-phy@4 {
+ reg = <4>;
+ };
+
+ mt7530: switch@1f {
+ compatible = "mediatek,mt7621";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1f>;
+ pinctrl-names = "default";
+ mediatek,mcm;
+
+ resets = <&rstctrl 2>;
+ reset-names = "mcm";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ label = "lan0";
+ };
+
+ port@1 {
+ reg = <1>;
+ label = "lan1";
+ };
+
+ port@2 {
+ reg = <2>;
+ label = "lan2";
+ };
+
+ port@3 {
+ reg = <3>;
+ label = "lan3";
+ };
+
+/* Commented out. Port 4 is handled by 2nd GMAC.
+ port@4 {
+ reg = <4>;
+ label = "lan4";
+ };
+*/
+
+ cpu_port0: port@6 {
+ reg = <6>;
+ label = "cpu";
+ ethernet = <&gmac0>;
+ phy-mode = "rgmii";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ pause;
+ };
+ };
+ };
+ };
+ };
+};
+
+Example 3: MT7621: Port 5 is connected to external PHY: Port 5 -> external PHY.
+
+ð {
+ gmac0: mac@0 {
+ compatible = "mediatek,eth-mac";
+ reg = <0>;
+ phy-mode = "rgmii";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ pause;
+ };
+ };
+
+ mdio: mdio-bus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* External phy */
+ ephy5: ethernet-phy@7 {
+ reg = <7>;
+ };
+
+ mt7530: switch@1f {
+ compatible = "mediatek,mt7621";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x1f>;
+ pinctrl-names = "default";
+ mediatek,mcm;
+
+ resets = <&rstctrl 2>;
+ reset-names = "mcm";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+ label = "lan0";
+ };
+
+ port@1 {
+ reg = <1>;
+ label = "lan1";
+ };
+
+ port@2 {
+ reg = <2>;
+ label = "lan2";
+ };
+
+ port@3 {
+ reg = <3>;
+ label = "lan3";
+ };
+
+ port@4 {
+ reg = <4>;
+ label = "lan4";
+ };
+
+ port@5 {
+ reg = <5>;
+ label = "lan5";
+ phy-mode = "rgmii";
+ phy-handle = <&ephy5>;
+ };
+
+ cpu_port0: port@6 {
+ reg = <6>;
+ label = "cpu";
+ ethernet = <&gmac0>;
+ phy-mode = "rgmii";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ pause;
+ };
+ };
+ };
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/net/macb.txt b/Documentation/devicetree/bindings/net/macb.txt
index 63c73fa..0b61a90 100644
--- a/Documentation/devicetree/bindings/net/macb.txt
+++ b/Documentation/devicetree/bindings/net/macb.txt
@@ -15,10 +15,10 @@
Use "atmel,sama5d4-gem" for the GEM IP (10/100) available on Atmel sama5d4 SoCs.
Use "cdns,zynq-gem" Xilinx Zynq-7xxx SoC.
Use "cdns,zynqmp-gem" for Zynq Ultrascale+ MPSoC.
- Use "sifive,fu540-macb" for SiFive FU540-C000 SoC.
+ Use "sifive,fu540-c000-gem" for SiFive FU540-C000 SoC.
Or the generic form: "cdns,emac".
- reg: Address and length of the register set for the device
- For "sifive,fu540-macb", second range is required to specify the
+ For "sifive,fu540-c000-gem", second range is required to specify the
address and length of the registers for GEMGXL Management block.
- interrupts: Should contain macb interrupt
- phy-mode: See ethernet.txt file in the same directory.
diff --git a/Documentation/devicetree/bindings/net/meson-dwmac.txt b/Documentation/devicetree/bindings/net/meson-dwmac.txt
deleted file mode 100644
index 1321bb1..0000000
--- a/Documentation/devicetree/bindings/net/meson-dwmac.txt
+++ /dev/null
@@ -1,71 +0,0 @@
-* Amlogic Meson DWMAC Ethernet controller
-
-The device inherits all the properties of the dwmac/stmmac devices
-described in the file stmmac.txt in the current directory with the
-following changes.
-
-Required properties on all platforms:
-
-- compatible: Depending on the platform this should be one of:
- - "amlogic,meson6-dwmac"
- - "amlogic,meson8b-dwmac"
- - "amlogic,meson8m2-dwmac"
- - "amlogic,meson-gxbb-dwmac"
- - "amlogic,meson-axg-dwmac"
- Additionally "snps,dwmac" and any applicable more
- detailed version number described in net/stmmac.txt
- should be used.
-
-- reg: The first register range should be the one of the DWMAC
- controller. The second range is is for the Amlogic specific
- configuration (for example the PRG_ETHERNET register range
- on Meson8b and newer)
-
-Required properties on Meson8b, Meson8m2, GXBB and newer:
-- clock-names: Should contain the following:
- - "stmmaceth" - see stmmac.txt
- - "clkin0" - first parent clock of the internal mux
- - "clkin1" - second parent clock of the internal mux
-
-Optional properties on Meson8b, Meson8m2, GXBB and newer:
-- amlogic,tx-delay-ns: The internal RGMII TX clock delay (provided
- by this driver) in nanoseconds. Allowed values
- are: 0ns, 2ns, 4ns, 6ns.
- When phy-mode is set to "rgmii" then the TX
- delay should be explicitly configured. When
- not configured a fallback of 2ns is used.
- When the phy-mode is set to either "rgmii-id"
- or "rgmii-txid" the TX clock delay is already
- provided by the PHY. In that case this
- property should be set to 0ns (which disables
- the TX clock delay in the MAC to prevent the
- clock from going off because both PHY and MAC
- are adding a delay).
- Any configuration is ignored when the phy-mode
- is set to "rmii".
-
-Example for Meson6:
-
- ethmac: ethernet@c9410000 {
- compatible = "amlogic,meson6-dwmac", "snps,dwmac";
- reg = <0xc9410000 0x10000
- 0xc1108108 0x4>;
- interrupts = <0 8 1>;
- interrupt-names = "macirq";
- clocks = <&clk81>;
- clock-names = "stmmaceth";
- }
-
-Example for GXBB:
- ethmac: ethernet@c9410000 {
- compatible = "amlogic,meson-gxbb-dwmac", "snps,dwmac";
- reg = <0x0 0xc9410000 0x0 0x10000>,
- <0x0 0xc8834540 0x0 0x8>;
- interrupts = <0 8 1>;
- interrupt-names = "macirq";
- clocks = <&clkc CLKID_ETH>,
- <&clkc CLKID_FCLK_DIV2>,
- <&clkc CLKID_MPLL2>;
- clock-names = "stmmaceth", "clkin0", "clkin1";
- phy-mode = "rgmii";
- };
diff --git a/Documentation/devicetree/bindings/net/snps,dwmac.yaml b/Documentation/devicetree/bindings/net/snps,dwmac.yaml
index 76fea2b..c78be15 100644
--- a/Documentation/devicetree/bindings/net/snps,dwmac.yaml
+++ b/Documentation/devicetree/bindings/net/snps,dwmac.yaml
@@ -50,6 +50,11 @@
- allwinner,sun8i-r40-emac
- allwinner,sun8i-v3s-emac
- allwinner,sun50i-a64-emac
+ - amlogic,meson6-dwmac
+ - amlogic,meson8b-dwmac
+ - amlogic,meson8m2-dwmac
+ - amlogic,meson-gxbb-dwmac
+ - amlogic,meson-axg-dwmac
- snps,dwmac
- snps,dwmac-3.50a
- snps,dwmac-3.610
@@ -61,7 +66,8 @@
- snps,dwxgmac-2.10
reg:
- maxItems: 1
+ minItems: 1
+ maxItems: 2
interrupts:
minItems: 1
diff --git a/Documentation/networking/af_xdp.rst b/Documentation/networking/af_xdp.rst
index eeedc2e..83f7ae5 100644
--- a/Documentation/networking/af_xdp.rst
+++ b/Documentation/networking/af_xdp.rst
@@ -153,10 +153,12 @@
Frames passed to the kernel are used for the ingress path (RX rings).
-The user application produces UMEM addrs to this ring. Note that the
-kernel will mask the incoming addr. E.g. for a chunk size of 2k, the
-log2(2048) LSB of the addr will be masked off, meaning that 2048, 2050
-and 3000 refers to the same chunk.
+The user application produces UMEM addrs to this ring. Note that, if
+running the application with aligned chunk mode, the kernel will mask
+the incoming addr. E.g. for a chunk size of 2k, the log2(2048) LSB of
+the addr will be masked off, meaning that 2048, 2050 and 3000 refers
+to the same chunk. If the user application is run in the unaligned
+chunks mode, then the incoming addr will be left untouched.
UMEM Completion Ring
diff --git a/Documentation/networking/device_drivers/index.rst b/Documentation/networking/device_drivers/index.rst
index 2b7fefe..57fec66 100644
--- a/Documentation/networking/device_drivers/index.rst
+++ b/Documentation/networking/device_drivers/index.rst
@@ -23,6 +23,7 @@
intel/ice
google/gve
mellanox/mlx5
+ pensando/ionic
.. only:: subproject
diff --git a/Documentation/networking/device_drivers/intel/iavf.rst b/Documentation/networking/device_drivers/intel/iavf.rst
index 2d0c3ba..cfc0884 100644
--- a/Documentation/networking/device_drivers/intel/iavf.rst
+++ b/Documentation/networking/device_drivers/intel/iavf.rst
@@ -10,11 +10,15 @@
Contents
========
+- Overview
- Identifying Your Adapter
- Additional Configurations
- Known Issues/Troubleshooting
- Support
+Overview
+========
+
This file describes the iavf Linux* Base Driver. This driver was formerly
called i40evf.
@@ -27,6 +31,7 @@
Identifying Your Adapter
========================
+
The driver in this kernel is compatible with devices based on the following:
* Intel(R) XL710 X710 Virtual Function
* Intel(R) X722 Virtual Function
@@ -50,9 +55,10 @@
restricting system messages. In order to see network driver link messages on
your console, set dmesg to eight by entering the following::
- dmesg -n 8
+ # dmesg -n 8
-NOTE: This setting is not saved across reboots.
+NOTE:
+ This setting is not saved across reboots.
ethtool
-------
@@ -72,11 +78,11 @@
To enable/disable VLAN tag stripping for a VF, issue the following command
from inside the VM in which you are running the VF::
- ethtool -K <if_name> rxvlan on/off
+ # ethtool -K <if_name> rxvlan on/off
or alternatively::
- ethtool --offload <if_name> rxvlan on/off
+ # ethtool --offload <if_name> rxvlan on/off
Adaptive Virtual Function
-------------------------
@@ -91,21 +97,21 @@
which the AVF is associated. The following are base mode features:
- 4 Queue Pairs (QP) and associated Configuration Status Registers (CSRs)
- for Tx/Rx.
-- i40e descriptors and ring format.
-- Descriptor write-back completion.
-- 1 control queue, with i40e descriptors, CSRs and ring format.
-- 5 MSI-X interrupt vectors and corresponding i40e CSRs.
-- 1 Interrupt Throttle Rate (ITR) index.
-- 1 Virtual Station Interface (VSI) per VF.
+ for Tx/Rx
+- i40e descriptors and ring format
+- Descriptor write-back completion
+- 1 control queue, with i40e descriptors, CSRs and ring format
+- 5 MSI-X interrupt vectors and corresponding i40e CSRs
+- 1 Interrupt Throttle Rate (ITR) index
+- 1 Virtual Station Interface (VSI) per VF
- 1 Traffic Class (TC), TC0
- Receive Side Scaling (RSS) with 64 entry indirection table and key,
- configured through the PF.
-- 1 unicast MAC address reserved per VF.
-- 16 MAC address filters for each VF.
-- Stateless offloads - non-tunneled checksums.
-- AVF device ID.
-- HW mailbox is used for VF to PF communications (including on Windows).
+ configured through the PF
+- 1 unicast MAC address reserved per VF
+- 16 MAC address filters for each VF
+- Stateless offloads - non-tunneled checksums
+- AVF device ID
+- HW mailbox is used for VF to PF communications (including on Windows)
IEEE 802.1ad (QinQ) Support
---------------------------
@@ -117,8 +123,8 @@
The following are examples of how to configure 802.1ad (QinQ)::
- ip link add link eth0 eth0.24 type vlan proto 802.1ad id 24
- ip link add link eth0.24 eth0.24.371 type vlan proto 802.1Q id 371
+ # ip link add link eth0 eth0.24 type vlan proto 802.1ad id 24
+ # ip link add link eth0.24 eth0.24.371 type vlan proto 802.1Q id 371
Where "24" and "371" are example VLAN IDs.
@@ -133,6 +139,19 @@
and allow Tx traffic to be rate limited per application. Follow the steps below
to set ADq.
+Requirements:
+
+- The sch_mqprio, act_mirred and cls_flower modules must be loaded
+- The latest version of iproute2
+- If another driver (for example, DPDK) has set cloud filters, you cannot
+ enable ADQ
+- Depending on the underlying PF device, ADQ cannot be enabled when the
+ following features are enabled:
+
+ + Data Center Bridging (DCB)
+ + Multiple Functions per Port (MFP)
+ + Sideband Filters
+
1. Create traffic classes (TCs). Maximum of 8 TCs can be created per interface.
The shaper bw_rlimit parameter is optional.
@@ -141,9 +160,9 @@
::
- # tc qdisc add dev <interface> root mqprio num_tc 2 map 0 0 0 0 1 1 1 1
- queues 16@0 16@16 hw 1 mode channel shaper bw_rlimit min_rate 1Gbit 2Gbit
- max_rate 1Gbit 3Gbit
+ tc qdisc add dev <interface> root mqprio num_tc 2 map 0 0 0 0 1 1 1 1
+ queues 16@0 16@16 hw 1 mode channel shaper bw_rlimit min_rate 1Gbit 2Gbit
+ max_rate 1Gbit 3Gbit
map: priority mapping for up to 16 priorities to tcs (e.g. map 0 0 0 0 1 1 1 1
sets priorities 0-3 to use tc0 and 4-7 to use tc1)
@@ -162,6 +181,10 @@
For example: min_rate 1Gbit 3Gbit: Verify bandwidth limit using network
monitoring tools such as ifstat or sar –n DEV [interval] [number of samples]
+NOTE:
+ Setting up channels via ethtool (ethtool -L) is not supported when the
+ TCs are configured using mqprio.
+
2. Enable HW TC offload on interface::
# ethtool -K <interface> hw-tc-offload on
@@ -171,16 +194,16 @@
# tc qdisc add dev <interface> ingress
NOTES:
- - Run all tc commands from the iproute2 <pathtoiproute2>/tc/ directory.
- - ADq is not compatible with cloud filters.
+ - Run all tc commands from the iproute2 <pathtoiproute2>/tc/ directory
+ - ADq is not compatible with cloud filters
- Setting up channels via ethtool (ethtool -L) is not supported when the TCs
- are configured using mqprio.
+ are configured using mqprio
- You must have iproute2 latest version
- - NVM version 6.01 or later is required.
+ - NVM version 6.01 or later is required
- ADq cannot be enabled when any the following features are enabled: Data
- Center Bridging (DCB), Multiple Functions per Port (MFP), or Sideband Filters.
+ Center Bridging (DCB), Multiple Functions per Port (MFP), or Sideband Filters
- If another driver (for example, DPDK) has set cloud filters, you cannot
- enable ADq.
+ enable ADq
- Tunnel filters are not supported in ADq. If encapsulated packets do arrive
in non-tunnel mode, filtering will be done on the inner headers. For example,
for VXLAN traffic in non-tunnel mode, PCTYPE is identified as a VXLAN
@@ -198,6 +221,16 @@
Known Issues/Troubleshooting
============================
+Bonding fails with VFs bound to an Intel(R) Ethernet Controller 700 series device
+---------------------------------------------------------------------------------
+If you bind Virtual Functions (VFs) to an Intel(R) Ethernet Controller 700
+series based device, the VF slaves may fail when they become the active slave.
+If the MAC address of the VF is set by the PF (Physical Function) of the
+device, when you add a slave, or change the active-backup slave, Linux bonding
+tries to sync the backup slave's MAC address to the same MAC address as the
+active slave. Linux bonding will fail at this point. This issue will not occur
+if the VF's MAC address is not set by the PF.
+
Traffic Is Not Being Passed Between VM and Client
-------------------------------------------------
You may not be able to pass traffic between a client system and a
@@ -215,13 +248,28 @@
Machine (VM) is bound to it. Doing so will cause the port to appear to hang.
Once the VM shuts down, or otherwise releases the VF, the command will complete.
+Using four traffic classes fails
+--------------------------------
+Do not try to reserve more than three traffic classes in the iavf driver. Doing
+so will fail to set any traffic classes and will cause the driver to write
+errors to stdout. Use a maximum of three queues to avoid this issue.
+
+Multiple log error messages on iavf driver removal
+--------------------------------------------------
+If you have several VFs and you remove the iavf driver, several instances of
+the following log errors are written to the log::
+
+ Unable to send opcode 2 to PF, err I40E_ERR_QUEUE_EMPTY, aq_err ok
+ Unable to send the message to VF 2 aq_err 12
+ ARQ Overflow Error detected
+
Virtual machine does not get link
---------------------------------
If the virtual machine has more than one virtual port assigned to it, and those
virtual ports are bound to different physical ports, you may not get link on
all of the virtual ports. The following command may work around the issue::
- ethtool -r <PF>
+ # ethtool -r <PF>
Where <PF> is the PF interface in the host, for example: p5p1. You may need to
run the command more than once to get link on all virtual ports.
@@ -251,12 +299,13 @@
If you have multiple interfaces in a server, either turn on ARP filtering by
entering::
- echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
+ # echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
-NOTE: This setting is not saved across reboots. The configuration change can be
-made permanent by adding the following line to the file /etc/sysctl.conf::
+NOTE:
+ This setting is not saved across reboots. The configuration change can be
+ made permanent by adding the following line to the file /etc/sysctl.conf::
- net.ipv4.conf.all.arp_filter = 1
+ net.ipv4.conf.all.arp_filter = 1
Another alternative is to install the interfaces in separate broadcast domains
(either in different switches or in a switch partitioned to VLANs).
diff --git a/Documentation/networking/device_drivers/mellanox/mlx5.rst b/Documentation/networking/device_drivers/mellanox/mlx5.rst
index b30a63d..d071c6b 100644
--- a/Documentation/networking/device_drivers/mellanox/mlx5.rst
+++ b/Documentation/networking/device_drivers/mellanox/mlx5.rst
@@ -11,6 +11,7 @@
- `Enabling the driver and kconfig options`_
- `Devlink info`_
+- `Devlink parameters`_
- `Devlink health reporters`_
- `mlx5 tracepoints`_
@@ -122,6 +123,38 @@
stored:
fw.version 16.26.0100
+Devlink parameters
+==================
+
+flow_steering_mode: Device flow steering mode
+---------------------------------------------
+The flow steering mode parameter controls the flow steering mode of the driver.
+Two modes are supported:
+1. 'dmfs' - Device managed flow steering.
+2. 'smfs - Software/Driver managed flow steering.
+
+In DMFS mode, the HW steering entities are created and managed through the
+Firmware.
+In SMFS mode, the HW steering entities are created and managed though by
+the driver directly into Hardware without firmware intervention.
+
+SMFS mode is faster and provides better rule inserstion rate compared to default DMFS mode.
+
+User command examples:
+
+- Set SMFS flow steering mode::
+
+ $ devlink dev param set pci/0000:06:00.0 name flow_steering_mode value "smfs" cmode runtime
+
+- Read device flow steering mode::
+
+ $ devlink dev param show pci/0000:06:00.0 name flow_steering_mode
+ pci/0000:06:00.0:
+ name flow_steering_mode type driver-specific
+ values:
+ cmode runtime value smfs
+
+
Devlink health reporters
========================
diff --git a/Documentation/networking/device_drivers/netronome/nfp.rst b/Documentation/networking/device_drivers/netronome/nfp.rst
new file mode 100644
index 0000000..6c08ac8
--- /dev/null
+++ b/Documentation/networking/device_drivers/netronome/nfp.rst
@@ -0,0 +1,133 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+=============================================
+Netronome Flow Processor (NFP) Kernel Drivers
+=============================================
+
+Copyright (c) 2019, Netronome Systems, Inc.
+
+Contents
+========
+
+- `Overview`_
+- `Acquiring Firmware`_
+
+Overview
+========
+
+This driver supports Netronome's line of Flow Processor devices,
+including the NFP4000, NFP5000, and NFP6000 models, which are also
+incorporated in the company's family of Agilio SmartNICs. The SR-IOV
+physical and virtual functions for these devices are supported by
+the driver.
+
+Acquiring Firmware
+==================
+
+The NFP4000 and NFP6000 devices require application specific firmware
+to function. Application firmware can be located either on the host file system
+or in the device flash (if supported by management firmware).
+
+Firmware files on the host filesystem contain card type (`AMDA-*` string), media
+config etc. They should be placed in `/lib/firmware/netronome` directory to
+load firmware from the host file system.
+
+Firmware for basic NIC operation is available in the upstream
+`linux-firmware.git` repository.
+
+Firmware in NVRAM
+-----------------
+
+Recent versions of management firmware supports loading application
+firmware from flash when the host driver gets probed. The firmware loading
+policy configuration may be used to configure this feature appropriately.
+
+Devlink or ethtool can be used to update the application firmware on the device
+flash by providing the appropriate `nic_AMDA*.nffw` file to the respective
+command. Users need to take care to write the correct firmware image for the
+card and media configuration to flash.
+
+Available storage space in flash depends on the card being used.
+
+Dealing with multiple projects
+------------------------------
+
+NFP hardware is fully programmable therefore there can be different
+firmware images targeting different applications.
+
+When using application firmware from host, we recommend placing
+actual firmware files in application-named subdirectories in
+`/lib/firmware/netronome` and linking the desired files, e.g.::
+
+ $ tree /lib/firmware/netronome/
+ /lib/firmware/netronome/
+ ├── bpf
+ │ ├── nic_AMDA0081-0001_1x40.nffw
+ │ └── nic_AMDA0081-0001_4x10.nffw
+ ├── flower
+ │ ├── nic_AMDA0081-0001_1x40.nffw
+ │ └── nic_AMDA0081-0001_4x10.nffw
+ ├── nic
+ │ ├── nic_AMDA0081-0001_1x40.nffw
+ │ └── nic_AMDA0081-0001_4x10.nffw
+ ├── nic_AMDA0081-0001_1x40.nffw -> bpf/nic_AMDA0081-0001_1x40.nffw
+ └── nic_AMDA0081-0001_4x10.nffw -> bpf/nic_AMDA0081-0001_4x10.nffw
+
+ 3 directories, 8 files
+
+You may need to use hard instead of symbolic links on distributions
+which use old `mkinitrd` command instead of `dracut` (e.g. Ubuntu).
+
+After changing firmware files you may need to regenerate the initramfs
+image. Initramfs contains drivers and firmware files your system may
+need to boot. Refer to the documentation of your distribution to find
+out how to update initramfs. Good indication of stale initramfs
+is system loading wrong driver or firmware on boot, but when driver is
+later reloaded manually everything works correctly.
+
+Selecting firmware per device
+-----------------------------
+
+Most commonly all cards on the system use the same type of firmware.
+If you want to load specific firmware image for a specific card, you
+can use either the PCI bus address or serial number. Driver will print
+which files it's looking for when it recognizes a NFP device::
+
+ nfp: Looking for firmware file in order of priority:
+ nfp: netronome/serial-00-12-34-aa-bb-cc-10-ff.nffw: not found
+ nfp: netronome/pci-0000:02:00.0.nffw: not found
+ nfp: netronome/nic_AMDA0081-0001_1x40.nffw: found, loading...
+
+In this case if file (or link) called *serial-00-12-34-aa-bb-5d-10-ff.nffw*
+or *pci-0000:02:00.0.nffw* is present in `/lib/firmware/netronome` this
+firmware file will take precedence over `nic_AMDA*` files.
+
+Note that `serial-*` and `pci-*` files are **not** automatically included
+in initramfs, you will have to refer to documentation of appropriate tools
+to find out how to include them.
+
+Firmware loading policy
+-----------------------
+
+Firmware loading policy is controlled via three HWinfo parameters
+stored as key value pairs in the device flash:
+
+app_fw_from_flash
+ Defines which firmware should take precedence, 'Disk' (0), 'Flash' (1) or
+ the 'Preferred' (2) firmware. When 'Preferred' is selected, the management
+ firmware makes the decision over which firmware will be loaded by comparing
+ versions of the flash firmware and the host supplied firmware.
+ This variable is configurable using the 'fw_load_policy'
+ devlink parameter.
+
+abi_drv_reset
+ Defines if the driver should reset the firmware when
+ the driver is probed, either 'Disk' (0) if firmware was found on disk,
+ 'Always' (1) reset or 'Never' (2) reset. Note that the device is always
+ reset on driver unload if firmware was loaded when the driver was probed.
+ This variable is configurable using the 'reset_dev_on_drv_probe'
+ devlink parameter.
+
+abi_drv_load_ifc
+ Defines a list of PF devices allowed to load FW on the device.
+ This variable is not currently user configurable.
diff --git a/Documentation/networking/device_drivers/pensando/ionic.rst b/Documentation/networking/device_drivers/pensando/ionic.rst
new file mode 100644
index 0000000..67b6839
--- /dev/null
+++ b/Documentation/networking/device_drivers/pensando/ionic.rst
@@ -0,0 +1,43 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+==========================================================
+Linux* Driver for the Pensando(R) Ethernet adapter family
+==========================================================
+
+Pensando Linux Ethernet driver.
+Copyright(c) 2019 Pensando Systems, Inc
+
+Contents
+========
+
+- Identifying the Adapter
+- Support
+
+Identifying the Adapter
+=======================
+
+To find if one or more Pensando PCI Ethernet devices are installed on the
+host, check for the PCI devices::
+
+ $ lspci -d 1dd8:
+ b5:00.0 Ethernet controller: Device 1dd8:1002
+ b6:00.0 Ethernet controller: Device 1dd8:1002
+
+If such devices are listed as above, then the ionic.ko driver should find
+and configure them for use. There should be log entries in the kernel
+messages such as these::
+
+ $ dmesg | grep ionic
+ ionic Pensando Ethernet NIC Driver, ver 0.15.0-k
+ ionic 0000:b5:00.0 enp181s0: renamed from eth0
+ ionic 0000:b6:00.0 enp182s0: renamed from eth0
+
+Support
+=======
+For general Linux networking support, please use the netdev mailing
+list, which is monitored by Pensando personnel::
+ netdev@vger.kernel.org
+
+For more specific support needs, please use the Pensando driver support
+email::
+ drivers@pensando.io
diff --git a/Documentation/networking/devlink-info-versions.rst b/Documentation/networking/devlink-info-versions.rst
index 4316342..4914f58 100644
--- a/Documentation/networking/devlink-info-versions.rst
+++ b/Documentation/networking/devlink-info-versions.rst
@@ -14,11 +14,27 @@
Board design revision.
+asic.id
+=======
+
+ASIC design identifier.
+
+asic.rev
+========
+
+ASIC design revision.
+
board.manufacture
=================
An identifier of the company or the facility which produced the part.
+fw
+==
+
+Overall firmware version, often representing the collection of
+fw.mgmt, fw.app, etc.
+
fw.mgmt
=======
diff --git a/Documentation/networking/devlink-params-nfp.txt b/Documentation/networking/devlink-params-nfp.txt
new file mode 100644
index 0000000..43e4d40
--- /dev/null
+++ b/Documentation/networking/devlink-params-nfp.txt
@@ -0,0 +1,5 @@
+fw_load_policy [DEVICE, GENERIC]
+ Configuration mode: permanent
+
+reset_dev_on_drv_probe [DEVICE, GENERIC]
+ Configuration mode: permanent
diff --git a/Documentation/networking/devlink-params.txt b/Documentation/networking/devlink-params.txt
index 2d26434..ddba3e9 100644
--- a/Documentation/networking/devlink-params.txt
+++ b/Documentation/networking/devlink-params.txt
@@ -48,4 +48,20 @@
Load firmware version preferred by the driver.
* DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH (1)
Load firmware currently stored in flash.
+ * DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK (2)
+ Load firmware currently available on host's disk.
+ Type: u8
+
+reset_dev_on_drv_probe [DEVICE, GENERIC]
+ Controls the device's reset policy on driver probe.
+ Valid values:
+ * DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN (0)
+ Unknown or invalid value.
+ * DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS (1)
+ Always reset device on driver probe.
+ * DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER (2)
+ Never reset device on driver probe.
+ * DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK (3)
+ Reset only if device firmware can be found in the
+ filesystem.
Type: u8
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index 37eabc1..0481d0f 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -17,6 +17,7 @@
devlink-trap
devlink-trap-netdevsim
ieee802154
+ j1939
kapi
z8530book
msg_zerocopy
diff --git a/Documentation/networking/j1939.rst b/Documentation/networking/j1939.rst
new file mode 100644
index 0000000..ce7e7a0
--- /dev/null
+++ b/Documentation/networking/j1939.rst
@@ -0,0 +1,422 @@
+.. SPDX-License-Identifier: (GPL-2.0 OR MIT)
+
+===================
+J1939 Documentation
+===================
+
+Overview / What Is J1939
+========================
+
+SAE J1939 defines a higher layer protocol on CAN. It implements a more
+sophisticated addressing scheme and extends the maximum packet size above 8
+bytes. Several derived specifications exist, which differ from the original
+J1939 on the application level, like MilCAN A, NMEA2000 and especially
+ISO-11783 (ISOBUS). This last one specifies the so-called ETP (Extended
+Transport Protocol) which is has been included in this implementation. This
+results in a maximum packet size of ((2 ^ 24) - 1) * 7 bytes == 111 MiB.
+
+Specifications used
+-------------------
+
+* SAE J1939-21 : data link layer
+* SAE J1939-81 : network management
+* ISO 11783-6 : Virtual Terminal (Extended Transport Protocol)
+
+.. _j1939-motivation:
+
+Motivation
+==========
+
+Given the fact there's something like SocketCAN with an API similar to BSD
+sockets, we found some reasons to justify a kernel implementation for the
+addressing and transport methods used by J1939.
+
+* **Addressing:** when a process on an ECU communicates via J1939, it should
+ not necessarily know its source address. Although at least one process per
+ ECU should know the source address. Other processes should be able to reuse
+ that address. This way, address parameters for different processes
+ cooperating for the same ECU, are not duplicated. This way of working is
+ closely related to the UNIX concept where programs do just one thing, and do
+ it well.
+
+* **Dynamic addressing:** Address Claiming in J1939 is time critical.
+ Furthermore data transport should be handled properly during the address
+ negotiation. Putting this functionality in the kernel eliminates it as a
+ requirement for _every_ user space process that communicates via J1939. This
+ results in a consistent J1939 bus with proper addressing.
+
+* **Transport:** both TP & ETP reuse some PGNs to relay big packets over them.
+ Different processes may thus use the same TP & ETP PGNs without actually
+ knowing it. The individual TP & ETP sessions _must_ be serialized
+ (synchronized) between different processes. The kernel solves this problem
+ properly and eliminates the serialization (synchronization) as a requirement
+ for _every_ user space process that communicates via J1939.
+
+J1939 defines some other features (relaying, gateway, fast packet transport,
+...). In-kernel code for these would not contribute to protocol stability.
+Therefore, these parts are left to user space.
+
+The J1939 sockets operate on CAN network devices (see SocketCAN). Any J1939
+user space library operating on CAN raw sockets will still operate properly.
+Since such library does not communicate with the in-kernel implementation, care
+must be taken that these two do not interfere. In practice, this means they
+cannot share ECU addresses. A single ECU (or virtual ECU) address is used by
+the library exclusively, or by the in-kernel system exclusively.
+
+J1939 concepts
+==============
+
+PGN
+---
+
+The PGN (Parameter Group Number) is a number to identify a packet. The PGN
+is composed as follows:
+1 bit : Reserved Bit
+1 bit : Data Page
+8 bits : PF (PDU Format)
+8 bits : PS (PDU Specific)
+
+In J1939-21 distinction is made between PDU1 format (where PF < 240) and PDU2
+format (where PF >= 240). Furthermore, when using PDU2 format, the PS-field
+contains a so-called Group Extension, which is part of the PGN. When using PDU2
+format, the Group Extension is set in the PS-field.
+
+On the other hand, when using PDU1 format, the PS-field contains a so-called
+Destination Address, which is _not_ part of the PGN. When communicating a PGN
+from user space to kernel (or visa versa) and PDU2 format is used, the PS-field
+of the PGN shall be set to zero. The Destination Address shall be set
+elsewhere.
+
+Regarding PGN mapping to 29-bit CAN identifier, the Destination Address shall
+be get/set from/to the appropriate bits of the identifier by the kernel.
+
+
+Addressing
+----------
+
+Both static and dynamic addressing methods can be used.
+
+For static addresses, no extra checks are made by the kernel, and provided
+addresses are considered right. This responsibility is for the OEM or system
+integrator.
+
+For dynamic addressing, so-called Address Claiming, extra support is foreseen
+in the kernel. In J1939 any ECU is known by it's 64-bit NAME. At the moment of
+a successful address claim, the kernel keeps track of both NAME and source
+address being claimed. This serves as a base for filter schemes. By default,
+packets with a destination that is not locally, will be rejected.
+
+Mixed mode packets (from a static to a dynamic address or vice versa) are
+allowed. The BSD sockets define separate API calls for getting/setting the
+local & remote address and are applicable for J1939 sockets.
+
+Filtering
+---------
+
+J1939 defines white list filters per socket that a user can set in order to
+receive a subset of the J1939 traffic. Filtering can be based on:
+
+* SA
+* SOURCE_NAME
+* PGN
+
+When multiple filters are in place for a single socket, and a packet comes in
+that matches several of those filters, the packet is only received once for
+that socket.
+
+How to Use J1939
+================
+
+API Calls
+---------
+
+On CAN, you first need to open a socket for communicating over a CAN network.
+To use J1939, #include <linux/can/j1939.h>. From there, <linux/can.h> will be
+included too. To open a socket, use:
+
+.. code-block:: C
+
+ s = socket(PF_CAN, SOCK_DGRAM, CAN_J1939);
+
+J1939 does use SOCK_DGRAM sockets. In the J1939 specification, connections are
+mentioned in the context of transport protocol sessions. These still deliver
+packets to the other end (using several CAN packets). SOCK_STREAM is not
+supported.
+
+After the successful creation of the socket, you would normally use the bind(2)
+and/or connect(2) system call to bind the socket to a CAN interface. After
+binding and/or connecting the socket, you can read(2) and write(2) from/to the
+socket or use send(2), sendto(2), sendmsg(2) and the recv*() counterpart
+operations on the socket as usual. There are also J1939 specific socket options
+described below.
+
+In order to send data, a bind(2) must have been successful. bind(2) assigns a
+local address to a socket.
+
+Different from CAN is that the payload data is just the data that get send,
+without it's header info. The header info is derived from the sockaddr supplied
+to bind(2), connect(2), sendto(2) and recvfrom(2). A write(2) with size 4 will
+result in a packet with 4 bytes.
+
+The sockaddr structure has extensions for use with J1939 as specified below:
+
+.. code-block:: C
+
+ struct sockaddr_can {
+ sa_family_t can_family;
+ int can_ifindex;
+ union {
+ struct {
+ __u64 name;
+ /* pgn:
+ * 8 bit: PS in PDU2 case, else 0
+ * 8 bit: PF
+ * 1 bit: DP
+ * 1 bit: reserved
+ */
+ __u32 pgn;
+ __u8 addr;
+ } j1939;
+ } can_addr;
+ }
+
+can_family & can_ifindex serve the same purpose as for other SocketCAN sockets.
+
+can_addr.j1939.pgn specifies the PGN (max 0x3ffff). Individual bits are
+specified above.
+
+can_addr.j1939.name contains the 64-bit J1939 NAME.
+
+can_addr.j1939.addr contains the address.
+
+The bind(2) system call assigns the local address, i.e. the source address when
+sending packages. If a PGN during bind(2) is set, it's used as a RX filter.
+I.e. only packets with a matching PGN are received. If an ADDR or NAME is set
+it is used as a receive filter, too. It will match the destination NAME or ADDR
+of the incoming packet. The NAME filter will work only if appropriate Address
+Claiming for this name was done on the CAN bus and registered/cached by the
+kernel.
+
+On the other hand connect(2) assigns the remote address, i.e. the destination
+address. The PGN from connect(2) is used as the default PGN when sending
+packets. If ADDR or NAME is set it will be used as the default destination ADDR
+or NAME. Further a set ADDR or NAME during connect(2) is used as a receive
+filter. It will match the source NAME or ADDR of the incoming packet.
+
+Both write(2) and send(2) will send a packet with local address from bind(2) and
+the remote address from connect(2). Use sendto(2) to overwrite the destination
+address.
+
+If can_addr.j1939.name is set (!= 0) the NAME is looked up by the kernel and
+the corresponding ADDR is used. If can_addr.j1939.name is not set (== 0),
+can_addr.j1939.addr is used.
+
+When creating a socket, reasonable defaults are set. Some options can be
+modified with setsockopt(2) & getsockopt(2).
+
+RX path related options:
+
+- SO_J1939_FILTER - configure array of filters
+- SO_J1939_PROMISC - disable filters set by bind(2) and connect(2)
+
+By default no broadcast packets can be send or received. To enable sending or
+receiving broadcast packets use the socket option SO_BROADCAST:
+
+.. code-block:: C
+
+ int value = 1;
+ setsockopt(sock, SOL_SOCKET, SO_BROADCAST, &value, sizeof(value));
+
+The following diagram illustrates the RX path:
+
+.. code::
+
+ +--------------------+
+ | incoming packet |
+ +--------------------+
+ |
+ V
+ +--------------------+
+ | SO_J1939_PROMISC? |
+ +--------------------+
+ | |
+ no | | yes
+ | |
+ .---------' `---------.
+ | |
+ +---------------------------+ |
+ | bind() + connect() + | |
+ | SOCK_BROADCAST filter | |
+ +---------------------------+ |
+ | |
+ |<---------------------'
+ V
+ +---------------------------+
+ | SO_J1939_FILTER |
+ +---------------------------+
+ |
+ V
+ +---------------------------+
+ | socket recv() |
+ +---------------------------+
+
+TX path related options:
+SO_J1939_SEND_PRIO - change default send priority for the socket
+
+Message Flags during send() and Related System Calls
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+send(2), sendto(2) and sendmsg(2) take a 'flags' argument. Currently
+supported flags are:
+
+* MSG_DONTWAIT, i.e. non-blocking operation.
+
+recvmsg(2)
+^^^^^^^^^
+
+In most cases recvmsg(2) is needed if you want to extract more information than
+recvfrom(2) can provide. For example package priority and timestamp. The
+Destination Address, name and packet priority (if applicable) are attached to
+the msghdr in the recvmsg(2) call. They can be extracted using cmsg(3) macros,
+with cmsg_level == SOL_J1939 && cmsg_type == SCM_J1939_DEST_ADDR,
+SCM_J1939_DEST_NAME or SCM_J1939_PRIO. The returned data is a uint8_t for
+priority and dst_addr, and uint64_t for dst_name.
+
+.. code-block:: C
+
+ uint8_t priority, dst_addr;
+ uint64_t dst_name;
+
+ for (cmsg = CMSG_FIRSTHDR(&msg); cmsg; cmsg = CMSG_NXTHDR(&msg, cmsg)) {
+ switch (cmsg->cmsg_level) {
+ case SOL_CAN_J1939:
+ if (cmsg->cmsg_type == SCM_J1939_DEST_ADDR)
+ dst_addr = *CMSG_DATA(cmsg);
+ else if (cmsg->cmsg_type == SCM_J1939_DEST_NAME)
+ memcpy(&dst_name, CMSG_DATA(cmsg), cmsg->cmsg_len - CMSG_LEN(0));
+ else if (cmsg->cmsg_type == SCM_J1939_PRIO)
+ priority = *CMSG_DATA(cmsg);
+ break;
+ }
+ }
+
+Dynamic Addressing
+------------------
+
+Distinction has to be made between using the claimed address and doing an
+address claim. To use an already claimed address, one has to fill in the
+j1939.name member and provide it to bind(2). If the name had claimed an address
+earlier, all further messages being sent will use that address. And the
+j1939.addr member will be ignored.
+
+An exception on this is PGN 0x0ee00. This is the "Address Claim/Cannot Claim
+Address" message and the kernel will use the j1939.addr member for that PGN if
+necessary.
+
+To claim an address following code example can be used:
+
+.. code-block:: C
+
+ struct sockaddr_can baddr = {
+ .can_family = AF_CAN,
+ .can_addr.j1939 = {
+ .name = name,
+ .addr = J1939_IDLE_ADDR,
+ .pgn = J1939_NO_PGN, /* to disable bind() rx filter for PGN */
+ },
+ .can_ifindex = if_nametoindex("can0"),
+ };
+
+ bind(sock, (struct sockaddr *)&baddr, sizeof(baddr));
+
+ /* for Address Claiming broadcast must be allowed */
+ int value = 1;
+ setsockopt(sock, SOL_SOCKET, SO_BROADCAST, &value, sizeof(value));
+
+ /* configured advanced RX filter with PGN needed for Address Claiming */
+ const struct j1939_filter filt[] = {
+ {
+ .pgn = J1939_PGN_ADDRESS_CLAIMED,
+ .pgn_mask = J1939_PGN_PDU1_MAX,
+ }, {
+ .pgn = J1939_PGN_ADDRESS_REQUEST,
+ .pgn_mask = J1939_PGN_PDU1_MAX,
+ }, {
+ .pgn = J1939_PGN_ADDRESS_COMMANDED,
+ .pgn_mask = J1939_PGN_MAX,
+ },
+ };
+
+ setsockopt(sock, SOL_CAN_J1939, SO_J1939_FILTER, &filt, sizeof(filt));
+
+ uint64_t dat = htole64(name);
+ const struct sockaddr_can saddr = {
+ .can_family = AF_CAN,
+ .can_addr.j1939 = {
+ .pgn = J1939_PGN_ADDRESS_CLAIMED,
+ .addr = J1939_NO_ADDR,
+ },
+ };
+
+ /* Afterwards do a sendto(2) with data set to the NAME (Little Endian). If the
+ * NAME provided, does not match the j1939.name provided to bind(2), EPROTO
+ * will be returned.
+ */
+ sendto(sock, dat, sizeof(dat), 0, (const struct sockaddr *)&saddr, sizeof(saddr));
+
+If no-one else contests the address claim within 250ms after transmission, the
+kernel marks the NAME-SA assignment as valid. The valid assignment will be kept
+among other valid NAME-SA assignments. From that point, any socket bound to the
+NAME can send packets.
+
+If another ECU claims the address, the kernel will mark the NAME-SA expired.
+No socket bound to the NAME can send packets (other than address claims). To
+claim another address, some socket bound to NAME, must bind(2) again, but with
+only j1939.addr changed to the new SA, and must then send a valid address claim
+packet. This restarts the state machine in the kernel (and any other
+participant on the bus) for this NAME.
+
+can-utils also include the jacd tool, so it can be used as code example or as
+default Address Claiming daemon.
+
+Send Examples
+-------------
+
+Static Addressing
+^^^^^^^^^^^^^^^^^
+
+This example will send a PGN (0x12300) from SA 0x20 to DA 0x30.
+
+Bind:
+
+.. code-block:: C
+
+ struct sockaddr_can baddr = {
+ .can_family = AF_CAN,
+ .can_addr.j1939 = {
+ .name = J1939_NO_NAME,
+ .addr = 0x20,
+ .pgn = J1939_NO_PGN,
+ },
+ .can_ifindex = if_nametoindex("can0"),
+ };
+
+ bind(sock, (struct sockaddr *)&baddr, sizeof(baddr));
+
+Now, the socket 'sock' is bound to the SA 0x20. Since no connect(2) was called,
+at this point we can use only sendto(2) or sendmsg(2).
+
+Send:
+
+.. code-block:: C
+
+ const struct sockaddr_can saddr = {
+ .can_family = AF_CAN,
+ .can_addr.j1939 = {
+ .name = J1939_NO_NAME;
+ .pgn = 0x30,
+ .addr = 0x12300,
+ },
+ };
+
+ sendto(sock, dat, sizeof(dat), 0, (const struct sockaddr *)&saddr, sizeof(saddr));
diff --git a/Documentation/process/embargoed-hardware-issues.rst b/Documentation/process/embargoed-hardware-issues.rst
new file mode 100644
index 0000000..d37cbc5
--- /dev/null
+++ b/Documentation/process/embargoed-hardware-issues.rst
@@ -0,0 +1,279 @@
+Embargoed hardware issues
+=========================
+
+Scope
+-----
+
+Hardware issues which result in security problems are a different category
+of security bugs than pure software bugs which only affect the Linux
+kernel.
+
+Hardware issues like Meltdown, Spectre, L1TF etc. must be treated
+differently because they usually affect all Operating Systems ("OS") and
+therefore need coordination across different OS vendors, distributions,
+hardware vendors and other parties. For some of the issues, software
+mitigations can depend on microcode or firmware updates, which need further
+coordination.
+
+.. _Contact:
+
+Contact
+-------
+
+The Linux kernel hardware security team is separate from the regular Linux
+kernel security team.
+
+The team only handles the coordination of embargoed hardware security
+issues. Reports of pure software security bugs in the Linux kernel are not
+handled by this team and the reporter will be guided to contact the regular
+Linux kernel security team (:ref:`Documentation/admin-guide/
+<securitybugs>`) instead.
+
+The team can be contacted by email at <hardware-security@kernel.org>. This
+is a private list of security officers who will help you to coordinate an
+issue according to our documented process.
+
+The list is encrypted and email to the list can be sent by either PGP or
+S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME
+certificate. The list's PGP key and S/MIME certificate are available from
+https://www.kernel.org/....
+
+While hardware security issues are often handled by the affected hardware
+vendor, we welcome contact from researchers or individuals who have
+identified a potential hardware flaw.
+
+Hardware security officers
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The current team of hardware security officers:
+
+ - Linus Torvalds (Linux Foundation Fellow)
+ - Greg Kroah-Hartman (Linux Foundation Fellow)
+ - Thomas Gleixner (Linux Foundation Fellow)
+
+Operation of mailing-lists
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The encrypted mailing-lists which are used in our process are hosted on
+Linux Foundation's IT infrastructure. By providing this service Linux
+Foundation's director of IT Infrastructure security technically has the
+ability to access the embargoed information, but is obliged to
+confidentiality by his employment contract. Linux Foundation's director of
+IT Infrastructure security is also responsible for the kernel.org
+infrastructure.
+
+The Linux Foundation's current director of IT Infrastructure security is
+Konstantin Ryabitsev.
+
+
+Non-disclosure agreements
+-------------------------
+
+The Linux kernel hardware security team is not a formal body and therefore
+unable to enter into any non-disclosure agreements. The kernel community
+is aware of the sensitive nature of such issues and offers a Memorandum of
+Understanding instead.
+
+
+Memorandum of Understanding
+---------------------------
+
+The Linux kernel community has a deep understanding of the requirement to
+keep hardware security issues under embargo for coordination between
+different OS vendors, distributors, hardware vendors and other parties.
+
+The Linux kernel community has successfully handled hardware security
+issues in the past and has the necessary mechanisms in place to allow
+community compliant development under embargo restrictions.
+
+The Linux kernel community has a dedicated hardware security team for
+initial contact, which oversees the process of handling such issues under
+embargo rules.
+
+The hardware security team identifies the developers (domain experts) who
+will form the initial response team for a particular issue. The initial
+response team can bring in further developers (domain experts) to address
+the issue in the best technical way.
+
+All involved developers pledge to adhere to the embargo rules and to keep
+the received information confidential. Violation of the pledge will lead to
+immediate exclusion from the current issue and removal from all related
+mailing-lists. In addition, the hardware security team will also exclude
+the offender from future issues. The impact of this consequence is a highly
+effective deterrent in our community. In case a violation happens the
+hardware security team will inform the involved parties immediately. If you
+or anyone becomes aware of a potential violation, please report it
+immediately to the Hardware security officers.
+
+
+Process
+^^^^^^^
+
+Due to the globally distributed nature of Linux kernel development,
+face-to-face meetings are almost impossible to address hardware security
+issues. Phone conferences are hard to coordinate due to time zones and
+other factors and should be only used when absolutely necessary. Encrypted
+email has been proven to be the most effective and secure communication
+method for these types of issues.
+
+Start of Disclosure
+"""""""""""""""""""
+
+Disclosure starts by contacting the Linux kernel hardware security team by
+email. This initial contact should contain a description of the problem and
+a list of any known affected hardware. If your organization builds or
+distributes the affected hardware, we encourage you to also consider what
+other hardware could be affected.
+
+The hardware security team will provide an incident-specific encrypted
+mailing-list which will be used for initial discussion with the reporter,
+further disclosure and coordination.
+
+The hardware security team will provide the disclosing party a list of
+developers (domain experts) who should be informed initially about the
+issue after confirming with the developers that they will adhere to this
+Memorandum of Understanding and the documented process. These developers
+form the initial response team and will be responsible for handling the
+issue after initial contact. The hardware security team is supporting the
+response team, but is not necessarily involved in the mitigation
+development process.
+
+While individual developers might be covered by a non-disclosure agreement
+via their employer, they cannot enter individual non-disclosure agreements
+in their role as Linux kernel developers. They will, however, agree to
+adhere to this documented process and the Memorandum of Understanding.
+
+
+Disclosure
+""""""""""
+
+The disclosing party provides detailed information to the initial response
+team via the specific encrypted mailing-list.
+
+From our experience the technical documentation of these issues is usually
+a sufficient starting point and further technical clarification is best
+done via email.
+
+Mitigation development
+""""""""""""""""""""""
+
+The initial response team sets up an encrypted mailing-list or repurposes
+an existing one if appropriate. The disclosing party should provide a list
+of contacts for all other parties who have already been, or should be
+informed about the issue. The response team contacts these parties so they
+can name experts who should be subscribed to the mailing-list.
+
+Using a mailing-list is close to the normal Linux development process and
+has been successfully used in developing mitigations for various hardware
+security issues in the past.
+
+The mailing-list operates in the same way as normal Linux development.
+Patches are posted, discussed and reviewed and if agreed on applied to a
+non-public git repository which is only accessible to the participating
+developers via a secure connection. The repository contains the main
+development branch against the mainline kernel and backport branches for
+stable kernel versions as necessary.
+
+The initial response team will identify further experts from the Linux
+kernel developer community as needed and inform the disclosing party about
+their participation. Bringing in experts can happen at any time of the
+development process and often needs to be handled in a timely manner.
+
+Coordinated release
+"""""""""""""""""""
+
+The involved parties will negotiate the date and time where the embargo
+ends. At that point the prepared mitigations are integrated into the
+relevant kernel trees and published.
+
+While we understand that hardware security issues need coordinated embargo
+time, the embargo time should be constrained to the minimum time which is
+required for all involved parties to develop, test and prepare the
+mitigations. Extending embargo time artificially to meet conference talk
+dates or other non-technical reasons is creating more work and burden for
+the involved developers and response teams as the patches need to be kept
+up to date in order to follow the ongoing upstream kernel development,
+which might create conflicting changes.
+
+CVE assignment
+""""""""""""""
+
+Neither the hardware security team nor the initial response team assign
+CVEs, nor are CVEs required for the development process. If CVEs are
+provided by the disclosing party they can be used for documentation
+purposes.
+
+Process ambassadors
+-------------------
+
+For assistance with this process we have established ambassadors in various
+organizations, who can answer questions about or provide guidance on the
+reporting process and further handling. Ambassadors are not involved in the
+disclosure of a particular issue, unless requested by a response team or by
+an involved disclosed party. The current ambassadors list:
+
+ ============= ========================================================
+ ARM
+ AMD
+ IBM
+ Intel
+ Qualcomm
+
+ Microsoft
+ VMware
+ XEN
+
+ Canonical Tyler Hicks <tyhicks@canonical.com>
+ Debian Ben Hutchings <ben@decadent.org.uk>
+ Oracle Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
+ Red Hat Josh Poimboeuf <jpoimboe@redhat.com>
+ SUSE Jiri Kosina <jkosina@suse.cz>
+
+ Amazon
+ Google
+ ============== ========================================================
+
+If you want your organization to be added to the ambassadors list, please
+contact the hardware security team. The nominated ambassador has to
+understand and support our process fully and is ideally well connected in
+the Linux kernel community.
+
+Encrypted mailing-lists
+-----------------------
+
+We use encrypted mailing-lists for communication. The operating principle
+of these lists is that email sent to the list is encrypted either with the
+list's PGP key or with the list's S/MIME certificate. The mailing-list
+software decrypts the email and re-encrypts it individually for each
+subscriber with the subscriber's PGP key or S/MIME certificate. Details
+about the mailing-list software and the setup which is used to ensure the
+security of the lists and protection of the data can be found here:
+https://www.kernel.org/....
+
+List keys
+^^^^^^^^^
+
+For initial contact see :ref:`Contact`. For incident specific mailing-lists
+the key and S/MIME certificate are conveyed to the subscribers by email
+sent from the specific list.
+
+Subscription to incident specific lists
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Subscription is handled by the response teams. Disclosed parties who want
+to participate in the communication send a list of potential subscribers to
+the response team so the response team can validate subscription requests.
+
+Each subscriber needs to send a subscription request to the response team
+by email. The email must be signed with the subscriber's PGP key or S/MIME
+certificate. If a PGP key is used, it must be available from a public key
+server and is ideally connected to the Linux kernel's PGP web of trust. See
+also: https://www.kernel.org/signature.html.
+
+The response team verifies that the subscriber request is valid and adds
+the subscriber to the list. After subscription the subscriber will receive
+email from the mailing-list which is signed either with the list's PGP key
+or the list's S/MIME certificate. The subscriber's email client can extract
+the PGP key or the S/MIME certificate from the signature so the subscriber
+can send encrypted email to the list.
+
diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst
index 878ebfd..e2c9ffc 100644
--- a/Documentation/process/index.rst
+++ b/Documentation/process/index.rst
@@ -45,6 +45,7 @@
submit-checklist
kernel-docs
deprecated
+ embargoed-hardware-issues
These are some overall technical guides that have been put here for now for
lack of a better place.
diff --git a/MAINTAINERS b/MAINTAINERS
index 9860853..84bb347 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -683,7 +683,7 @@
F: drivers/crypto/sunxi-ss/
ALLWINNER VPU DRIVER
-M: Maxime Ripard <maxime.ripard@bootlin.com>
+M: Maxime Ripard <mripard@kernel.org>
M: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
L: linux-media@vger.kernel.org
S: Maintained
@@ -1416,7 +1416,7 @@
F: drivers/clk/sunxi/
ARM/Allwinner sunXi SoC support
-M: Maxime Ripard <maxime.ripard@bootlin.com>
+M: Maxime Ripard <mripard@kernel.org>
M: Chen-Yu Tsai <wens@csie.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
@@ -2923,6 +2923,7 @@
M: Marek Lindner <mareklindner@neomailbox.ch>
M: Simon Wunderlich <sw@simonwunderlich.de>
M: Antonio Quartulli <a@unstable.cc>
+M: Sven Eckelmann <sven@narfation.org>
L: b.a.t.m.a.n@lists.open-mesh.org (moderated for non-subscribers)
W: https://www.open-mesh.org/
B: https://www.open-mesh.org/projects/batman-adv/issues
@@ -3585,7 +3586,7 @@
F: fs/cachefiles/
CADENCE MIPI-CSI2 BRIDGES
-M: Maxime Ripard <maxime.ripard@bootlin.com>
+M: Maxime Ripard <mripard@kernel.org>
L: linux-media@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/media/cdns,*.txt
@@ -3668,6 +3669,16 @@
F: include/uapi/linux/can/raw.h
F: include/uapi/linux/can/gw.h
+CAN-J1939 NETWORK LAYER
+M: Robin van der Gracht <robin@protonic.nl>
+M: Oleksij Rempel <o.rempel@pengutronix.de>
+R: Pengutronix Kernel Team <kernel@pengutronix.de>
+L: linux-can@vger.kernel.org
+S: Maintained
+F: Documentation/networking/j1939.txt
+F: net/can/j1939/
+F: include/uapi/linux/can/j1939.h
+
CAPABILITIES
M: Serge Hallyn <serge@hallyn.com>
L: linux-security-module@vger.kernel.org
@@ -5308,7 +5319,7 @@
DRM DRIVERS AND MISC GPU PATCHES
M: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
-M: Maxime Ripard <maxime.ripard@bootlin.com>
+M: Maxime Ripard <mripard@kernel.org>
M: Sean Paul <sean@poorly.run>
W: https://01.org/linuxgraphics/gfx-docs/maintainer-tools/drm-misc.html
S: Maintained
@@ -5321,7 +5332,7 @@
F: include/linux/vga*
DRM DRIVERS FOR ALLWINNER A10
-M: Maxime Ripard <maxime.ripard@bootlin.com>
+M: Maxime Ripard <mripard@kernel.org>
L: dri-devel@lists.freedesktop.org
S: Supported
F: drivers/gpu/drm/sun4i/
@@ -7527,7 +7538,7 @@
M: Gregory CLEMENT <gregory.clement@bootlin.com>
L: linux-i2c@vger.kernel.org
S: Maintained
-F: Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt
+F: Documentation/devicetree/bindings/i2c/marvell,mv64xxx-i2c.yaml
F: drivers/i2c/busses/i2c-mv64xxx.c
I2C OVER PARALLEL PORT
@@ -8468,11 +8479,6 @@
F: fs/io_uring.c
F: include/uapi/linux/io_uring.h
-IP MASQUERADING
-M: Juanjo Ciarlante <jjciarla@raiz.uncu.edu.ar>
-S: Maintained
-F: net/ipv4/netfilter/ipt_MASQUERADE.c
-
IPMI SUBSYSTEM
M: Corey Minyard <minyard@acm.org>
L: openipmi-developer@lists.sourceforge.net (moderated for non-subscribers)
@@ -8846,14 +8852,6 @@
F: tools/kvm/
F: tools/testing/selftests/kvm/
-KERNEL VIRTUAL MACHINE FOR AMD-V (KVM/amd)
-M: Joerg Roedel <joro@8bytes.org>
-L: kvm@vger.kernel.org
-W: http://www.linux-kvm.org/
-S: Maintained
-F: arch/x86/include/asm/svm.h
-F: arch/x86/kvm/svm.c
-
KERNEL VIRTUAL MACHINE FOR ARM/ARM64 (KVM/arm, KVM/arm64)
M: Marc Zyngier <maz@kernel.org>
R: James Morse <james.morse@arm.com>
@@ -8896,7 +8894,7 @@
M: Janosch Frank <frankja@linux.ibm.com>
R: David Hildenbrand <david@redhat.com>
R: Cornelia Huck <cohuck@redhat.com>
-L: linux-s390@vger.kernel.org
+L: kvm@vger.kernel.org
W: http://www.ibm.com/developerworks/linux/linux390/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git
S: Supported
@@ -8911,6 +8909,11 @@
KERNEL VIRTUAL MACHINE FOR X86 (KVM/x86)
M: Paolo Bonzini <pbonzini@redhat.com>
M: Radim Krčmář <rkrcmar@redhat.com>
+R: Sean Christopherson <sean.j.christopherson@intel.com>
+R: Vitaly Kuznetsov <vkuznets@redhat.com>
+R: Wanpeng Li <wanpengli@tencent.com>
+R: Jim Mattson <jmattson@google.com>
+R: Joerg Roedel <joro@8bytes.org>
L: kvm@vger.kernel.org
W: http://www.linux-kvm.org
T: git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
@@ -8918,8 +8921,12 @@
F: arch/x86/kvm/
F: arch/x86/kvm/*/
F: arch/x86/include/uapi/asm/kvm*
+F: arch/x86/include/uapi/asm/vmx.h
+F: arch/x86/include/uapi/asm/svm.h
F: arch/x86/include/asm/kvm*
F: arch/x86/include/asm/pvclock-abi.h
+F: arch/x86/include/asm/svm.h
+F: arch/x86/include/asm/vmx.h
F: arch/x86/kernel/kvm.c
F: arch/x86/kernel/kvmclock.c
@@ -9247,6 +9254,18 @@
F: include/linux/libnvdimm.h
F: include/uapi/linux/ndctl.h
+LICENSES and SPDX stuff
+M: Thomas Gleixner <tglx@linutronix.de>
+M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
+L: linux-spdx@vger.kernel.org
+S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx.git
+F: COPYING
+F: Documentation/process/license-rules.rst
+F: LICENSES/
+F: scripts/spdxcheck-test.sh
+F: scripts/spdxcheck.py
+
LIGHTNVM PLATFORM SUPPORT
M: Matias Bjorling <mb@lightnvm.io>
W: http://github/OpenChannelSSD
@@ -11099,7 +11118,7 @@
M: Sridhar Samudrala <sridhar.samudrala@intel.com>
L: netdev@vger.kernel.org
S: Supported
-F: driver/net/net_failover.c
+F: drivers/net/net_failover.c
F: include/net/net_failover.h
F: Documentation/networking/net_failover.rst
@@ -12599,6 +12618,14 @@
S: Maintained
F: drivers/platform/x86/peaq-wmi.c
+PENSANDO ETHERNET DRIVERS
+M: Shannon Nelson <snelson@pensando.io>
+M: Pensando Drivers <drivers@pensando.io>
+L: netdev@vger.kernel.org
+S: Supported
+F: Documentation/networking/device_drivers/pensando/ionic.rst
+F: drivers/net/ethernet/pensando/
+
PER-CPU MEMORY ALLOCATOR
M: Dennis Zhou <dennis@kernel.org>
M: Tejun Heo <tj@kernel.org>
@@ -14491,6 +14518,7 @@
F: drivers/net/phy/sfp*
F: include/linux/phylink.h
F: include/linux/sfp.h
+K: phylink
SGI GRU DRIVER
M: Dimitri Sivanich <sivanich@sgi.com>
@@ -14896,9 +14924,9 @@
F: include/uapi/linux/arm_sdei.h
SOFTWARE RAID (Multiple Disks) SUPPORT
-M: Shaohua Li <shli@kernel.org>
+M: Song Liu <song@kernel.org>
L: linux-raid@vger.kernel.org
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/shli/md.git
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/song/md.git
S: Supported
F: drivers/md/Makefile
F: drivers/md/Kconfig
diff --git a/Makefile b/Makefile
index 9fa1861..0cbe871 100644
--- a/Makefile
+++ b/Makefile
@@ -2,7 +2,7 @@
VERSION = 5
PATCHLEVEL = 3
SUBLEVEL = 0
-EXTRAVERSION = -rc5
+EXTRAVERSION = -rc7
NAME = Bobtail Squid
# *DOCUMENTATION*
diff --git a/arch/arc/boot/dts/Makefile b/arch/arc/boot/dts/Makefile
index a83c4f5..8483a86 100644
--- a/arch/arc/boot/dts/Makefile
+++ b/arch/arc/boot/dts/Makefile
@@ -12,3 +12,6 @@
# for CONFIG_OF_ALL_DTBS test
dtstree := $(srctree)/$(src)
dtb- := $(patsubst $(dtstree)/%.dts,%.dtb, $(wildcard $(dtstree)/*.dts))
+
+# board-specific dtc flags
+DTC_FLAGS_hsdk += --pad 20
diff --git a/arch/arc/include/asm/entry-arcv2.h b/arch/arc/include/asm/entry-arcv2.h
index f5ae394..41b16f2 100644
--- a/arch/arc/include/asm/entry-arcv2.h
+++ b/arch/arc/include/asm/entry-arcv2.h
@@ -256,7 +256,7 @@
.macro FAKE_RET_FROM_EXCPN
lr r9, [status32]
- bic r9, r9, (STATUS_U_MASK|STATUS_DE_MASK|STATUS_AE_MASK)
+ bic r9, r9, STATUS_AE_MASK
or r9, r9, STATUS_IE_MASK
kflag r9
.endm
diff --git a/arch/arc/include/asm/linkage.h b/arch/arc/include/asm/linkage.h
index a0eeb9f..d9ee43c6 100644
--- a/arch/arc/include/asm/linkage.h
+++ b/arch/arc/include/asm/linkage.h
@@ -62,15 +62,15 @@
#else /* !__ASSEMBLY__ */
#ifdef CONFIG_ARC_HAS_ICCM
-#define __arcfp_code __attribute__((__section__(".text.arcfp")))
+#define __arcfp_code __section(.text.arcfp)
#else
-#define __arcfp_code __attribute__((__section__(".text")))
+#define __arcfp_code __section(.text)
#endif
#ifdef CONFIG_ARC_HAS_DCCM
-#define __arcfp_data __attribute__((__section__(".data.arcfp")))
+#define __arcfp_data __section(.data.arcfp)
#else
-#define __arcfp_data __attribute__((__section__(".data")))
+#define __arcfp_data __section(.data)
#endif
#endif /* __ASSEMBLY__ */
diff --git a/arch/arc/include/asm/mach_desc.h b/arch/arc/include/asm/mach_desc.h
index 8ac0e2a..73746ed 100644
--- a/arch/arc/include/asm/mach_desc.h
+++ b/arch/arc/include/asm/mach_desc.h
@@ -53,8 +53,7 @@ extern const struct machine_desc __arch_info_begin[], __arch_info_end[];
*/
#define MACHINE_START(_type, _name) \
static const struct machine_desc __mach_desc_##_type \
-__used \
-__attribute__((__section__(".arch.info.init"))) = { \
+__used __section(.arch.info.init) = { \
.name = _name,
#define MACHINE_END \
diff --git a/arch/arc/kernel/mcip.c b/arch/arc/kernel/mcip.c
index 18b493d..abf9398 100644
--- a/arch/arc/kernel/mcip.c
+++ b/arch/arc/kernel/mcip.c
@@ -202,8 +202,8 @@ static void idu_set_dest(unsigned int cmn_irq, unsigned int cpu_mask)
__mcip_cmd_data(CMD_IDU_SET_DEST, cmn_irq, cpu_mask);
}
-static void idu_set_mode(unsigned int cmn_irq, unsigned int lvl,
- unsigned int distr)
+static void idu_set_mode(unsigned int cmn_irq, bool set_lvl, unsigned int lvl,
+ bool set_distr, unsigned int distr)
{
union {
unsigned int word;
@@ -212,8 +212,11 @@ static void idu_set_mode(unsigned int cmn_irq, unsigned int lvl,
};
} data;
- data.distr = distr;
- data.lvl = lvl;
+ data.word = __mcip_cmd_read(CMD_IDU_READ_MODE, cmn_irq);
+ if (set_distr)
+ data.distr = distr;
+ if (set_lvl)
+ data.lvl = lvl;
__mcip_cmd_data(CMD_IDU_SET_MODE, cmn_irq, data.word);
}
@@ -240,6 +243,25 @@ static void idu_irq_unmask(struct irq_data *data)
raw_spin_unlock_irqrestore(&mcip_lock, flags);
}
+static void idu_irq_ack(struct irq_data *data)
+{
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&mcip_lock, flags);
+ __mcip_cmd(CMD_IDU_ACK_CIRQ, data->hwirq);
+ raw_spin_unlock_irqrestore(&mcip_lock, flags);
+}
+
+static void idu_irq_mask_ack(struct irq_data *data)
+{
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&mcip_lock, flags);
+ __mcip_cmd_data(CMD_IDU_SET_MASK, data->hwirq, 1);
+ __mcip_cmd(CMD_IDU_ACK_CIRQ, data->hwirq);
+ raw_spin_unlock_irqrestore(&mcip_lock, flags);
+}
+
static int
idu_irq_set_affinity(struct irq_data *data, const struct cpumask *cpumask,
bool force)
@@ -263,13 +285,36 @@ idu_irq_set_affinity(struct irq_data *data, const struct cpumask *cpumask,
else
distribution_mode = IDU_M_DISTRI_RR;
- idu_set_mode(data->hwirq, IDU_M_TRIG_LEVEL, distribution_mode);
+ idu_set_mode(data->hwirq, false, 0, true, distribution_mode);
raw_spin_unlock_irqrestore(&mcip_lock, flags);
return IRQ_SET_MASK_OK;
}
+static int idu_irq_set_type(struct irq_data *data, u32 type)
+{
+ unsigned long flags;
+
+ /*
+ * ARCv2 IDU HW does not support inverse polarity, so these are the
+ * only interrupt types supported.
+ */
+ if (type & ~(IRQ_TYPE_EDGE_RISING | IRQ_TYPE_LEVEL_HIGH))
+ return -EINVAL;
+
+ raw_spin_lock_irqsave(&mcip_lock, flags);
+
+ idu_set_mode(data->hwirq, true,
+ type & IRQ_TYPE_EDGE_RISING ? IDU_M_TRIG_EDGE :
+ IDU_M_TRIG_LEVEL,
+ false, 0);
+
+ raw_spin_unlock_irqrestore(&mcip_lock, flags);
+
+ return 0;
+}
+
static void idu_irq_enable(struct irq_data *data)
{
/*
@@ -289,7 +334,10 @@ static struct irq_chip idu_irq_chip = {
.name = "MCIP IDU Intc",
.irq_mask = idu_irq_mask,
.irq_unmask = idu_irq_unmask,
+ .irq_ack = idu_irq_ack,
+ .irq_mask_ack = idu_irq_mask_ack,
.irq_enable = idu_irq_enable,
+ .irq_set_type = idu_irq_set_type,
#ifdef CONFIG_SMP
.irq_set_affinity = idu_irq_set_affinity,
#endif
@@ -317,7 +365,7 @@ static int idu_irq_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t
}
static const struct irq_domain_ops idu_irq_ops = {
- .xlate = irq_domain_xlate_onecell,
+ .xlate = irq_domain_xlate_onetwocell,
.map = idu_irq_map,
};
diff --git a/arch/arc/kernel/unwind.c b/arch/arc/kernel/unwind.c
index c2663fc..dc05a63 100644
--- a/arch/arc/kernel/unwind.c
+++ b/arch/arc/kernel/unwind.c
@@ -572,6 +572,7 @@ static unsigned long read_pointer(const u8 **pLoc, const void *end,
#else
BUILD_BUG_ON(sizeof(u32) != sizeof(value));
#endif
+ /* Fall through */
case DW_EH_PE_native:
if (end < (const void *)(ptr.pul + 1))
return 0;
@@ -826,7 +827,7 @@ static int processCFI(const u8 *start, const u8 *end, unsigned long targetLoc,
case DW_CFA_def_cfa:
state->cfa.reg = get_uleb128(&ptr.p8, end);
unw_debug("cfa_def_cfa: r%lu ", state->cfa.reg);
- /*nobreak*/
+ /* fall through */
case DW_CFA_def_cfa_offset:
state->cfa.offs = get_uleb128(&ptr.p8, end);
unw_debug("cfa_def_cfa_offset: 0x%lx ",
@@ -834,7 +835,7 @@ static int processCFI(const u8 *start, const u8 *end, unsigned long targetLoc,
break;
case DW_CFA_def_cfa_sf:
state->cfa.reg = get_uleb128(&ptr.p8, end);
- /*nobreak */
+ /* fall through */
case DW_CFA_def_cfa_offset_sf:
state->cfa.offs = get_sleb128(&ptr.p8, end)
* state->dataAlign;
diff --git a/arch/arc/mm/dma.c b/arch/arc/mm/dma.c
index 62c210e..70a3fbe7 100644
--- a/arch/arc/mm/dma.c
+++ b/arch/arc/mm/dma.c
@@ -101,7 +101,7 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
if (is_isa_arcv2() && ioc_enable && coherent)
dev->dma_coherent = true;
- dev_info(dev, "use %sncoherent DMA ops\n",
+ dev_info(dev, "use %scoherent DMA ops\n",
dev->dma_coherent ? "" : "non");
}
diff --git a/arch/arc/plat-hsdk/platform.c b/arch/arc/plat-hsdk/platform.c
index 7dd2dd3..0b961a2 100644
--- a/arch/arc/plat-hsdk/platform.c
+++ b/arch/arc/plat-hsdk/platform.c
@@ -6,11 +6,15 @@
*/
#include <linux/init.h>
+#include <linux/of_fdt.h>
+#include <linux/libfdt.h>
#include <linux/smp.h>
#include <asm/arcregs.h>
#include <asm/io.h>
#include <asm/mach_desc.h>
+int arc_hsdk_axi_dmac_coherent __section(.data) = 0;
+
#define ARC_CCM_UNUSED_ADDR 0x60000000
static void __init hsdk_init_per_cpu(unsigned int cpu)
@@ -97,6 +101,42 @@ static void __init hsdk_enable_gpio_intc_wire(void)
iowrite32(GPIO_INT_CONNECTED_MASK, (void __iomem *) GPIO_INTEN);
}
+static int __init hsdk_tweak_node_coherency(const char *path, bool coherent)
+{
+ void *fdt = initial_boot_params;
+ const void *prop;
+ int node, ret;
+ bool dt_coh_set;
+
+ node = fdt_path_offset(fdt, path);
+ if (node < 0)
+ goto tweak_fail;
+
+ prop = fdt_getprop(fdt, node, "dma-coherent", &ret);
+ if (!prop && ret != -FDT_ERR_NOTFOUND)
+ goto tweak_fail;
+
+ dt_coh_set = ret != -FDT_ERR_NOTFOUND;
+ ret = 0;
+
+ /* need to remove "dma-coherent" property */
+ if (dt_coh_set && !coherent)
+ ret = fdt_delprop(fdt, node, "dma-coherent");
+
+ /* need to set "dma-coherent" property */
+ if (!dt_coh_set && coherent)
+ ret = fdt_setprop(fdt, node, "dma-coherent", NULL, 0);
+
+ if (ret < 0)
+ goto tweak_fail;
+
+ return 0;
+
+tweak_fail:
+ pr_err("failed to tweak %s to %scoherent\n", path, coherent ? "" : "non");
+ return -EFAULT;
+}
+
enum hsdk_axi_masters {
M_HS_CORE = 0,
M_HS_RTT,
@@ -162,6 +202,39 @@ enum hsdk_axi_masters {
#define CREG_PAE ((void __iomem *)(CREG_BASE + 0x180))
#define CREG_PAE_UPDT ((void __iomem *)(CREG_BASE + 0x194))
+static void __init hsdk_init_memory_bridge_axi_dmac(void)
+{
+ bool coherent = !!arc_hsdk_axi_dmac_coherent;
+ u32 axi_m_slv1, axi_m_oft1;
+
+ /*
+ * Don't tweak memory bridge configuration if we failed to tweak DTB
+ * as we will end up in a inconsistent state.
+ */
+ if (hsdk_tweak_node_coherency("/soc/dmac@80000", coherent))
+ return;
+
+ if (coherent) {
+ axi_m_slv1 = 0x77999999;
+ axi_m_oft1 = 0x76DCBA98;
+ } else {
+ axi_m_slv1 = 0x77777777;
+ axi_m_oft1 = 0x76543210;
+ }
+
+ writel(0x77777777, CREG_AXI_M_SLV0(M_DMAC_0));
+ writel(0xFEDCBA98, CREG_AXI_M_OFT0(M_DMAC_0));
+ writel(axi_m_slv1, CREG_AXI_M_SLV1(M_DMAC_0));
+ writel(axi_m_oft1, CREG_AXI_M_OFT1(M_DMAC_0));
+ writel(UPDATE_VAL, CREG_AXI_M_UPDT(M_DMAC_0));
+
+ writel(0x77777777, CREG_AXI_M_SLV0(M_DMAC_1));
+ writel(0xFEDCBA98, CREG_AXI_M_OFT0(M_DMAC_1));
+ writel(axi_m_slv1, CREG_AXI_M_SLV1(M_DMAC_1));
+ writel(axi_m_oft1, CREG_AXI_M_OFT1(M_DMAC_1));
+ writel(UPDATE_VAL, CREG_AXI_M_UPDT(M_DMAC_1));
+}
+
static void __init hsdk_init_memory_bridge(void)
{
u32 reg;
@@ -227,24 +300,14 @@ static void __init hsdk_init_memory_bridge(void)
writel(0x76543210, CREG_AXI_M_OFT1(M_GPU));
writel(UPDATE_VAL, CREG_AXI_M_UPDT(M_GPU));
- writel(0x77777777, CREG_AXI_M_SLV0(M_DMAC_0));
- writel(0x77777777, CREG_AXI_M_SLV1(M_DMAC_0));
- writel(0xFEDCBA98, CREG_AXI_M_OFT0(M_DMAC_0));
- writel(0x76543210, CREG_AXI_M_OFT1(M_DMAC_0));
- writel(UPDATE_VAL, CREG_AXI_M_UPDT(M_DMAC_0));
-
- writel(0x77777777, CREG_AXI_M_SLV0(M_DMAC_1));
- writel(0x77777777, CREG_AXI_M_SLV1(M_DMAC_1));
- writel(0xFEDCBA98, CREG_AXI_M_OFT0(M_DMAC_1));
- writel(0x76543210, CREG_AXI_M_OFT1(M_DMAC_1));
- writel(UPDATE_VAL, CREG_AXI_M_UPDT(M_DMAC_1));
-
writel(0x00000000, CREG_AXI_M_SLV0(M_DVFS));
writel(0x60000000, CREG_AXI_M_SLV1(M_DVFS));
writel(0x00000000, CREG_AXI_M_OFT0(M_DVFS));
writel(0x00000000, CREG_AXI_M_OFT1(M_DVFS));
writel(UPDATE_VAL, CREG_AXI_M_UPDT(M_DVFS));
+ hsdk_init_memory_bridge_axi_dmac();
+
/*
* PAE remapping for DMA clients does not work due to an RTL bug, so
* CREG_PAE register must be programmed to all zeroes, otherwise it
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 33b0057..2436021 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -7,6 +7,8 @@
select ARCH_HAS_BINFMT_FLAT
select ARCH_HAS_DEBUG_VIRTUAL if MMU
select ARCH_HAS_DEVMEM_IS_ALLOWED
+ select ARCH_HAS_DMA_COHERENT_TO_PFN if SWIOTLB
+ select ARCH_HAS_DMA_MMAP_PGPROT if SWIOTLB
select ARCH_HAS_ELF_RANDOMIZE
select ARCH_HAS_FORTIFY_SOURCE
select ARCH_HAS_KEEPINITRD
@@ -18,6 +20,8 @@
select ARCH_HAS_SET_MEMORY
select ARCH_HAS_STRICT_KERNEL_RWX if MMU && !XIP_KERNEL
select ARCH_HAS_STRICT_MODULE_RWX if MMU
+ select ARCH_HAS_SYNC_DMA_FOR_DEVICE if SWIOTLB
+ select ARCH_HAS_SYNC_DMA_FOR_CPU if SWIOTLB
select ARCH_HAS_TEARDOWN_DMA_OPS if MMU
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
select ARCH_HAVE_CUSTOM_GPIO_H
diff --git a/arch/arm/boot/dts/am33xx-l4.dtsi b/arch/arm/boot/dts/am33xx-l4.dtsi
index ced1a19..46849d6 100644
--- a/arch/arm/boot/dts/am33xx-l4.dtsi
+++ b/arch/arm/boot/dts/am33xx-l4.dtsi
@@ -185,7 +185,7 @@
uart0: serial@0 {
compatible = "ti,am3352-uart", "ti,omap3-uart";
clock-frequency = <48000000>;
- reg = <0x0 0x2000>;
+ reg = <0x0 0x1000>;
interrupts = <72>;
status = "disabled";
dmas = <&edma 26 0>, <&edma 27 0>;
@@ -934,7 +934,7 @@
uart1: serial@0 {
compatible = "ti,am3352-uart", "ti,omap3-uart";
clock-frequency = <48000000>;
- reg = <0x0 0x2000>;
+ reg = <0x0 0x1000>;
interrupts = <73>;
status = "disabled";
dmas = <&edma 28 0>, <&edma 29 0>;
@@ -966,7 +966,7 @@
uart2: serial@0 {
compatible = "ti,am3352-uart", "ti,omap3-uart";
clock-frequency = <48000000>;
- reg = <0x0 0x2000>;
+ reg = <0x0 0x1000>;
interrupts = <74>;
status = "disabled";
dmas = <&edma 30 0>, <&edma 31 0>;
@@ -1614,7 +1614,7 @@
uart3: serial@0 {
compatible = "ti,am3352-uart", "ti,omap3-uart";
clock-frequency = <48000000>;
- reg = <0x0 0x2000>;
+ reg = <0x0 0x1000>;
interrupts = <44>;
status = "disabled";
};
@@ -1644,7 +1644,7 @@
uart4: serial@0 {
compatible = "ti,am3352-uart", "ti,omap3-uart";
clock-frequency = <48000000>;
- reg = <0x0 0x2000>;
+ reg = <0x0 0x1000>;
interrupts = <45>;
status = "disabled";
};
@@ -1674,7 +1674,7 @@
uart5: serial@0 {
compatible = "ti,am3352-uart", "ti,omap3-uart";
clock-frequency = <48000000>;
- reg = <0x0 0x2000>;
+ reg = <0x0 0x1000>;
interrupts = <46>;
status = "disabled";
};
@@ -1758,6 +1758,8 @@
target-module@cc000 { /* 0x481cc000, ap 60 46.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
+ reg = <0xcc020 0x4>;
+ reg-names = "rev";
ti,hwmods = "d_can0";
/* Domains (P, C): per_pwrdm, l4ls_clkdm */
clocks = <&l4ls_clkctrl AM3_L4LS_D_CAN0_CLKCTRL 0>,
@@ -1780,6 +1782,8 @@
target-module@d0000 { /* 0x481d0000, ap 62 42.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
+ reg = <0xd0020 0x4>;
+ reg-names = "rev";
ti,hwmods = "d_can1";
/* Domains (P, C): per_pwrdm, l4ls_clkdm */
clocks = <&l4ls_clkctrl AM3_L4LS_D_CAN1_CLKCTRL 0>,
diff --git a/arch/arm/boot/dts/am33xx.dtsi b/arch/arm/boot/dts/am33xx.dtsi
index e5c2f71..fb6b8aa 100644
--- a/arch/arm/boot/dts/am33xx.dtsi
+++ b/arch/arm/boot/dts/am33xx.dtsi
@@ -234,13 +234,33 @@
interrupt-names = "edma3_tcerrint";
};
- mmc3: mmc@47810000 {
- compatible = "ti,omap4-hsmmc";
+ target-module@47810000 {
+ compatible = "ti,sysc-omap2", "ti,sysc";
ti,hwmods = "mmc3";
- ti,needs-special-reset;
- interrupts = <29>;
- reg = <0x47810000 0x1000>;
- status = "disabled";
+ reg = <0x478102fc 0x4>,
+ <0x47810110 0x4>,
+ <0x47810114 0x4>;
+ reg-names = "rev", "sysc", "syss";
+ ti,sysc-mask = <(SYSC_OMAP2_CLOCKACTIVITY |
+ SYSC_OMAP2_ENAWAKEUP |
+ SYSC_OMAP2_SOFTRESET |
+ SYSC_OMAP2_AUTOIDLE)>;
+ ti,sysc-sidle = <SYSC_IDLE_FORCE>,
+ <SYSC_IDLE_NO>,
+ <SYSC_IDLE_SMART>;
+ ti,syss-mask = <1>;
+ clocks = <&l3s_clkctrl AM3_L3S_MMC3_CLKCTRL 0>;
+ clock-names = "fck";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x47810000 0x1000>;
+
+ mmc3: mmc@0 {
+ compatible = "ti,omap4-hsmmc";
+ ti,needs-special-reset;
+ interrupts = <29>;
+ reg = <0x0 0x1000>;
+ };
};
usb: usb@47400000 {
diff --git a/arch/arm/boot/dts/am4372.dtsi b/arch/arm/boot/dts/am4372.dtsi
index 55aff4d..848e2a8 100644
--- a/arch/arm/boot/dts/am4372.dtsi
+++ b/arch/arm/boot/dts/am4372.dtsi
@@ -228,13 +228,33 @@
interrupt-names = "edma3_tcerrint";
};
- mmc3: mmc@47810000 {
- compatible = "ti,omap4-hsmmc";
- reg = <0x47810000 0x1000>;
+ target-module@47810000 {
+ compatible = "ti,sysc-omap2", "ti,sysc";
ti,hwmods = "mmc3";
- ti,needs-special-reset;
- interrupts = <GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>;
- status = "disabled";
+ reg = <0x478102fc 0x4>,
+ <0x47810110 0x4>,
+ <0x47810114 0x4>;
+ reg-names = "rev", "sysc", "syss";
+ ti,sysc-mask = <(SYSC_OMAP2_CLOCKACTIVITY |
+ SYSC_OMAP2_ENAWAKEUP |
+ SYSC_OMAP2_SOFTRESET |
+ SYSC_OMAP2_AUTOIDLE)>;
+ ti,sysc-sidle = <SYSC_IDLE_FORCE>,
+ <SYSC_IDLE_NO>,
+ <SYSC_IDLE_SMART>;
+ ti,syss-mask = <1>;
+ clocks = <&l3s_clkctrl AM4_L3S_MMC3_CLKCTRL 0>;
+ clock-names = "fck";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0x0 0x47810000 0x1000>;
+
+ mmc3: mmc@0 {
+ compatible = "ti,omap4-hsmmc";
+ ti,needs-special-reset;
+ interrupts = <GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>;
+ reg = <0x0 0x1000>;
+ };
};
sham: sham@53100000 {
diff --git a/arch/arm/boot/dts/am437x-l4.dtsi b/arch/arm/boot/dts/am437x-l4.dtsi
index 989cb60..04bee4f 100644
--- a/arch/arm/boot/dts/am437x-l4.dtsi
+++ b/arch/arm/boot/dts/am437x-l4.dtsi
@@ -1574,6 +1574,8 @@
target-module@cc000 { /* 0x481cc000, ap 50 46.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
+ reg = <0xcc020 0x4>;
+ reg-names = "rev";
ti,hwmods = "d_can0";
/* Domains (P, C): per_pwrdm, l4ls_clkdm */
clocks = <&l4ls_clkctrl AM4_L4LS_D_CAN0_CLKCTRL 0>;
@@ -1593,6 +1595,8 @@
target-module@d0000 { /* 0x481d0000, ap 52 3a.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
+ reg = <0xd0020 0x4>;
+ reg-names = "rev";
ti,hwmods = "d_can1";
/* Domains (P, C): per_pwrdm, l4ls_clkdm */
clocks = <&l4ls_clkctrl AM4_L4LS_D_CAN1_CLKCTRL 0>;
diff --git a/arch/arm/boot/dts/am571x-idk.dts b/arch/arm/boot/dts/am571x-idk.dts
index 1d5e999..0aaacea 100644
--- a/arch/arm/boot/dts/am571x-idk.dts
+++ b/arch/arm/boot/dts/am571x-idk.dts
@@ -175,14 +175,9 @@
};
&mmc1 {
- pinctrl-names = "default", "hs", "sdr12", "sdr25", "sdr50", "ddr50", "sdr104";
+ pinctrl-names = "default", "hs";
pinctrl-0 = <&mmc1_pins_default_no_clk_pu>;
pinctrl-1 = <&mmc1_pins_hs>;
- pinctrl-2 = <&mmc1_pins_sdr12>;
- pinctrl-3 = <&mmc1_pins_sdr25>;
- pinctrl-4 = <&mmc1_pins_sdr50>;
- pinctrl-5 = <&mmc1_pins_ddr50_rev20 &mmc1_iodelay_ddr50_conf>;
- pinctrl-6 = <&mmc1_pins_sdr104 &mmc1_iodelay_sdr104_rev20_conf>;
};
&mmc2 {
diff --git a/arch/arm/boot/dts/am572x-idk.dts b/arch/arm/boot/dts/am572x-idk.dts
index c65d7f6..ea1c119f 100644
--- a/arch/arm/boot/dts/am572x-idk.dts
+++ b/arch/arm/boot/dts/am572x-idk.dts
@@ -16,14 +16,9 @@
};
&mmc1 {
- pinctrl-names = "default", "hs", "sdr12", "sdr25", "sdr50", "ddr50", "sdr104";
+ pinctrl-names = "default", "hs";
pinctrl-0 = <&mmc1_pins_default_no_clk_pu>;
pinctrl-1 = <&mmc1_pins_hs>;
- pinctrl-2 = <&mmc1_pins_sdr12>;
- pinctrl-3 = <&mmc1_pins_sdr25>;
- pinctrl-4 = <&mmc1_pins_sdr50>;
- pinctrl-5 = <&mmc1_pins_ddr50 &mmc1_iodelay_ddr_rev20_conf>;
- pinctrl-6 = <&mmc1_pins_sdr104 &mmc1_iodelay_sdr104_rev20_conf>;
};
&mmc2 {
diff --git a/arch/arm/boot/dts/am574x-idk.dts b/arch/arm/boot/dts/am574x-idk.dts
index dc5141c..7935d70 100644
--- a/arch/arm/boot/dts/am574x-idk.dts
+++ b/arch/arm/boot/dts/am574x-idk.dts
@@ -24,14 +24,9 @@
};
&mmc1 {
- pinctrl-names = "default", "hs", "sdr12", "sdr25", "sdr50", "ddr50", "sdr104";
+ pinctrl-names = "default", "hs";
pinctrl-0 = <&mmc1_pins_default_no_clk_pu>;
pinctrl-1 = <&mmc1_pins_hs>;
- pinctrl-2 = <&mmc1_pins_default>;
- pinctrl-3 = <&mmc1_pins_hs>;
- pinctrl-4 = <&mmc1_pins_sdr50>;
- pinctrl-5 = <&mmc1_pins_ddr50 &mmc1_iodelay_ddr_conf>;
- pinctrl-6 = <&mmc1_pins_ddr50 &mmc1_iodelay_sdr104_conf>;
};
&mmc2 {
diff --git a/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi b/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
index d02f5fa..bc76f17 100644
--- a/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
+++ b/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
@@ -379,7 +379,7 @@
};
};
-&gpio7 {
+&gpio7_target {
ti,no-reset-on-init;
ti,no-idle-on-init;
};
@@ -430,6 +430,7 @@
bus-width = <4>;
cd-gpios = <&gpio6 27 GPIO_ACTIVE_LOW>; /* gpio 219 */
+ no-1-8-v;
};
&mmc2 {
diff --git a/arch/arm/boot/dts/am57xx-beagle-x15-revb1.dts b/arch/arm/boot/dts/am57xx-beagle-x15-revb1.dts
index a374b5c..7b113b5 100644
--- a/arch/arm/boot/dts/am57xx-beagle-x15-revb1.dts
+++ b/arch/arm/boot/dts/am57xx-beagle-x15-revb1.dts
@@ -16,14 +16,9 @@
};
&mmc1 {
- pinctrl-names = "default", "hs", "sdr12", "sdr25", "sdr50", "ddr50", "sdr104";
+ pinctrl-names = "default", "hs";
pinctrl-0 = <&mmc1_pins_default>;
pinctrl-1 = <&mmc1_pins_hs>;
- pinctrl-2 = <&mmc1_pins_sdr12>;
- pinctrl-3 = <&mmc1_pins_sdr25>;
- pinctrl-4 = <&mmc1_pins_sdr50>;
- pinctrl-5 = <&mmc1_pins_ddr50 &mmc1_iodelay_ddr_rev11_conf>;
- pinctrl-6 = <&mmc1_pins_sdr104 &mmc1_iodelay_sdr104_rev11_conf>;
vmmc-supply = <&vdd_3v3>;
vqmmc-supply = <&ldo1_reg>;
};
diff --git a/arch/arm/boot/dts/am57xx-beagle-x15-revc.dts b/arch/arm/boot/dts/am57xx-beagle-x15-revc.dts
index 4badd21..30c500b 100644
--- a/arch/arm/boot/dts/am57xx-beagle-x15-revc.dts
+++ b/arch/arm/boot/dts/am57xx-beagle-x15-revc.dts
@@ -16,14 +16,9 @@
};
&mmc1 {
- pinctrl-names = "default", "hs", "sdr12", "sdr25", "sdr50", "ddr50", "sdr104";
+ pinctrl-names = "default", "hs";
pinctrl-0 = <&mmc1_pins_default>;
pinctrl-1 = <&mmc1_pins_hs>;
- pinctrl-2 = <&mmc1_pins_sdr12>;
- pinctrl-3 = <&mmc1_pins_sdr25>;
- pinctrl-4 = <&mmc1_pins_sdr50>;
- pinctrl-5 = <&mmc1_pins_ddr50 &mmc1_iodelay_ddr_rev20_conf>;
- pinctrl-6 = <&mmc1_pins_sdr104 &mmc1_iodelay_sdr104_rev20_conf>;
vmmc-supply = <&vdd_3v3>;
vqmmc-supply = <&ldo1_reg>;
};
diff --git a/arch/arm/boot/dts/dra7-evm.dts b/arch/arm/boot/dts/dra7-evm.dts
index 714e971..de7f85e 100644
--- a/arch/arm/boot/dts/dra7-evm.dts
+++ b/arch/arm/boot/dts/dra7-evm.dts
@@ -498,7 +498,7 @@
phy-supply = <&ldousb_reg>;
};
-&gpio7 {
+&gpio7_target {
ti,no-reset-on-init;
ti,no-idle-on-init;
};
diff --git a/arch/arm/boot/dts/dra7-l4.dtsi b/arch/arm/boot/dts/dra7-l4.dtsi
index 23faede..21e5914 100644
--- a/arch/arm/boot/dts/dra7-l4.dtsi
+++ b/arch/arm/boot/dts/dra7-l4.dtsi
@@ -1261,7 +1261,7 @@
};
};
- target-module@51000 { /* 0x48051000, ap 45 2e.0 */
+ gpio7_target: target-module@51000 { /* 0x48051000, ap 45 2e.0 */
compatible = "ti,sysc-omap2", "ti,sysc";
ti,hwmods = "gpio7";
reg = <0x51000 0x4>,
@@ -3025,7 +3025,7 @@
target-module@80000 { /* 0x48480000, ap 31 16.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
- reg = <0x80000 0x4>;
+ reg = <0x80020 0x4>;
reg-names = "rev";
clocks = <&l4per2_clkctrl DRA7_L4PER2_DCAN2_CLKCTRL 0>;
clock-names = "fck";
@@ -4577,7 +4577,7 @@
target-module@c000 { /* 0x4ae3c000, ap 30 04.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
- reg = <0xc000 0x4>;
+ reg = <0xc020 0x4>;
reg-names = "rev";
clocks = <&wkupaon_clkctrl DRA7_WKUPAON_DCAN1_CLKCTRL 0>;
clock-names = "fck";
diff --git a/arch/arm/boot/dts/dra74x-mmc-iodelay.dtsi b/arch/arm/boot/dts/dra74x-mmc-iodelay.dtsi
index 28ebb4e..214b9e6 100644
--- a/arch/arm/boot/dts/dra74x-mmc-iodelay.dtsi
+++ b/arch/arm/boot/dts/dra74x-mmc-iodelay.dtsi
@@ -32,7 +32,7 @@
*
* Datamanual Revisions:
*
- * AM572x Silicon Revision 2.0: SPRS953B, Revised November 2016
+ * AM572x Silicon Revision 2.0: SPRS953F, Revised May 2019
* AM572x Silicon Revision 1.1: SPRS915R, Revised November 2016
*
*/
@@ -229,45 +229,45 @@
mmc3_pins_default: mmc3_pins_default {
pinctrl-single,pins = <
- DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
- DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
- DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
- DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
- DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
- DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
+ DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
+ DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
+ DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
+ DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
+ DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
+ DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
>;
};
mmc3_pins_hs: mmc3_pins_hs {
pinctrl-single,pins = <
- DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
- DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
- DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
- DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
- DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
- DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
+ DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
+ DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
+ DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
+ DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
+ DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
+ DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
>;
};
mmc3_pins_sdr12: mmc3_pins_sdr12 {
pinctrl-single,pins = <
- DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
- DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
- DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
- DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
- DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
- DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
+ DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
+ DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
+ DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
+ DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
+ DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
+ DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
>;
};
mmc3_pins_sdr25: mmc3_pins_sdr25 {
pinctrl-single,pins = <
- DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
- DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
- DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
- DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
- DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
- DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
+ DRA7XX_CORE_IOPAD(0x377c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_clk.mmc3_clk */
+ DRA7XX_CORE_IOPAD(0x3780, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_cmd.mmc3_cmd */
+ DRA7XX_CORE_IOPAD(0x3784, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat0.mmc3_dat0 */
+ DRA7XX_CORE_IOPAD(0x3788, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat1.mmc3_dat1 */
+ DRA7XX_CORE_IOPAD(0x378c, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat2.mmc3_dat2 */
+ DRA7XX_CORE_IOPAD(0x3790, (PIN_INPUT_PULLUP | MODE_SELECT | MUX_MODE0)) /* mmc3_dat3.mmc3_dat3 */
>;
};
diff --git a/arch/arm/boot/dts/vf610-bk4.dts b/arch/arm/boot/dts/vf610-bk4.dts
index 3fa0cbe..0f3870d 100644
--- a/arch/arm/boot/dts/vf610-bk4.dts
+++ b/arch/arm/boot/dts/vf610-bk4.dts
@@ -246,13 +246,13 @@
reg = <0>;
};
- n25q128a13_2: flash@1 {
+ n25q128a13_2: flash@2 {
compatible = "n25q128a13", "jedec,spi-nor";
#address-cells = <1>;
#size-cells = <1>;
spi-max-frequency = <66000000>;
spi-rx-bus-width = <2>;
- reg = <1>;
+ reg = <2>;
};
};
diff --git a/arch/arm/lib/backtrace.S b/arch/arm/lib/backtrace.S
index 1d5210eb..58292523 100644
--- a/arch/arm/lib/backtrace.S
+++ b/arch/arm/lib/backtrace.S
@@ -66,7 +66,7 @@
1003: ldr r2, [sv_pc, #-4] @ if stmfd sp!, {args} exists,
ldr r3, .Ldsi+4 @ adjust saved 'pc' back one
- teq r3, r2, lsr #10 @ instruction
+ teq r3, r2, lsr #11 @ instruction
subne r0, sv_pc, #4 @ allow for mov
subeq r0, sv_pc, #8 @ allow for mov + stmia
diff --git a/arch/arm/mach-omap1/ams-delta-fiq-handler.S b/arch/arm/mach-omap1/ams-delta-fiq-handler.S
index 81159af..14a6c3e 100644
--- a/arch/arm/mach-omap1/ams-delta-fiq-handler.S
+++ b/arch/arm/mach-omap1/ams-delta-fiq-handler.S
@@ -126,6 +126,8 @@
orr r11, r11, r13 @ mask all requested interrupts
str r11, [r12, #OMAP1510_GPIO_INT_MASK]
+ str r13, [r12, #OMAP1510_GPIO_INT_STATUS] @ ack all requested interrupts
+
ands r10, r13, #KEYBRD_CLK_MASK @ extract keyboard status - set?
beq hksw @ no - try next source
@@ -133,7 +135,6 @@
@@@@@@@@@@@@@@@@@@@@@@
@ Keyboard clock FIQ mode interrupt handler
@ r10 now contains KEYBRD_CLK_MASK, use it
- str r10, [r12, #OMAP1510_GPIO_INT_STATUS] @ ack the interrupt
bic r11, r11, r10 @ unmask it
str r11, [r12, #OMAP1510_GPIO_INT_MASK]
diff --git a/arch/arm/mach-omap1/ams-delta-fiq.c b/arch/arm/mach-omap1/ams-delta-fiq.c
index 43899fa..0254eb9 100644
--- a/arch/arm/mach-omap1/ams-delta-fiq.c
+++ b/arch/arm/mach-omap1/ams-delta-fiq.c
@@ -70,9 +70,7 @@ static irqreturn_t deferred_fiq(int irq, void *dev_id)
* interrupts default to since commit 80ac93c27441
* requires interrupt already acked and unmasked.
*/
- if (irq_chip->irq_ack)
- irq_chip->irq_ack(d);
- if (irq_chip->irq_unmask)
+ if (!WARN_ON_ONCE(!irq_chip->irq_unmask))
irq_chip->irq_unmask(d);
}
for (; irq_counter[gpio] < fiq_count; irq_counter[gpio]++)
diff --git a/arch/arm/mach-omap2/omap4-common.c b/arch/arm/mach-omap2/omap4-common.c
index f9c02f9..5c38457 100644
--- a/arch/arm/mach-omap2/omap4-common.c
+++ b/arch/arm/mach-omap2/omap4-common.c
@@ -127,6 +127,9 @@ static int __init omap4_sram_init(void)
struct device_node *np;
struct gen_pool *sram_pool;
+ if (!soc_is_omap44xx() && !soc_is_omap54xx())
+ return 0;
+
np = of_find_compatible_node(NULL, NULL, "ti,omap4-mpu");
if (!np)
pr_warn("%s:Unable to allocate sram needed to handle errata I688\n",
diff --git a/arch/arm/mach-omap2/omap_hwmod_7xx_data.c b/arch/arm/mach-omap2/omap_hwmod_7xx_data.c
index 4a5b4ae..1ec21e9 100644
--- a/arch/arm/mach-omap2/omap_hwmod_7xx_data.c
+++ b/arch/arm/mach-omap2/omap_hwmod_7xx_data.c
@@ -379,7 +379,8 @@ static struct omap_hwmod dra7xx_dcan2_hwmod = {
static struct omap_hwmod_class_sysconfig dra7xx_epwmss_sysc = {
.rev_offs = 0x0,
.sysc_offs = 0x4,
- .sysc_flags = SYSC_HAS_SIDLEMODE | SYSC_HAS_SOFTRESET,
+ .sysc_flags = SYSC_HAS_SIDLEMODE | SYSC_HAS_SOFTRESET |
+ SYSC_HAS_RESET_STATUS,
.idlemodes = (SIDLE_FORCE | SIDLE_NO | SIDLE_SMART),
.sysc_fields = &omap_hwmod_sysc_type2,
};
diff --git a/arch/arm/mach-rpc/riscpc.c b/arch/arm/mach-rpc/riscpc.c
index 0ce56ad..ea2c842 100644
--- a/arch/arm/mach-rpc/riscpc.c
+++ b/arch/arm/mach-rpc/riscpc.c
@@ -46,6 +46,7 @@ static int __init parse_tag_acorn(const struct tag *tag)
switch (tag->u.acorn.vram_pages) {
case 512:
vram_size += PAGE_SIZE * 256;
+ /* Fall through - ??? */
case 256:
vram_size += PAGE_SIZE * 256;
default:
diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
index c54cd7e..c1222c0 100644
--- a/arch/arm/mm/Kconfig
+++ b/arch/arm/mm/Kconfig
@@ -664,10 +664,6 @@
!CPU_32v4 && !CPU_32v3
select PHYS_ADDR_T_64BIT
select SWIOTLB
- select ARCH_HAS_DMA_COHERENT_TO_PFN
- select ARCH_HAS_DMA_MMAP_PGPROT
- select ARCH_HAS_SYNC_DMA_FOR_DEVICE
- select ARCH_HAS_SYNC_DMA_FOR_CPU
help
Say Y if you have an ARMv7 processor supporting the LPAE page
table format and you would like to access memory beyond the
diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c
index 16d373d..b4be3ba 100644
--- a/arch/arm/mm/init.c
+++ b/arch/arm/mm/init.c
@@ -175,6 +175,11 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max_low,
#ifdef CONFIG_HAVE_ARCH_PFN_VALID
int pfn_valid(unsigned long pfn)
{
+ phys_addr_t addr = __pfn_to_phys(pfn);
+
+ if (__phys_to_pfn(addr) != pfn)
+ return 0;
+
return memblock_is_map_memory(__pfn_to_phys(pfn));
}
EXPORT_SYMBOL(pfn_valid);
@@ -628,7 +633,8 @@ static void update_sections_early(struct section_perm perms[], int n)
if (t->flags & PF_KTHREAD)
continue;
for_each_thread(t, s)
- set_section_perms(perms, n, true, s->mm);
+ if (s->mm)
+ set_section_perms(perms, n, true, s->mm);
}
set_section_perms(perms, n, true, current->active_mm);
set_section_perms(perms, n, true, &init_mm);
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12a-sei510.dts b/arch/arm64/boot/dts/amlogic/meson-g12a-sei510.dts
index c7a8736..12aa7ea 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12a-sei510.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-g12a-sei510.dts
@@ -339,6 +339,12 @@
pinctrl-names = "default";
};
+&ir {
+ status = "okay";
+ pinctrl-0 = <&remote_input_ao_pins>;
+ pinctrl-names = "default";
+};
+
&pwm_ef {
status = "okay";
pinctrl-0 = <&pwm_e_pins>;
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12a.dtsi b/arch/arm64/boot/dts/amlogic/meson-g12a.dtsi
index f8d43e3..1785552 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12a.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-g12a.dtsi
@@ -2386,6 +2386,7 @@
clocks = <&clkc CLKID_USB1_DDR_BRIDGE>;
clock-names = "ddr";
phys = <&usb2_phy1>;
+ phy-names = "usb2-phy";
dr_mode = "peripheral";
g-rx-fifo-size = <192>;
g-np-tx-fifo-size = <128>;
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dts b/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dts
index 81780ff..4e916e1 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dts
@@ -53,6 +53,7 @@
gpio = <&gpio_ao GPIOAO_8 GPIO_ACTIVE_HIGH>;
enable-active-high;
+ regulator-always-on;
};
tf_io: gpio-regulator-tf_io {
diff --git a/arch/arm64/boot/dts/mediatek/mt7622-bananapi-bpi-r64.dts b/arch/arm64/boot/dts/mediatek/mt7622-bananapi-bpi-r64.dts
index 710c5c3..83e1059 100644
--- a/arch/arm64/boot/dts/mediatek/mt7622-bananapi-bpi-r64.dts
+++ b/arch/arm64/boot/dts/mediatek/mt7622-bananapi-bpi-r64.dts
@@ -115,24 +115,34 @@
};
ð {
- pinctrl-names = "default";
- pinctrl-0 = <ð_pins>;
status = "okay";
+ gmac0: mac@0 {
+ compatible = "mediatek,eth-mac";
+ reg = <0>;
+ phy-mode = "2500base-x";
+
+ fixed-link {
+ speed = <2500>;
+ full-duplex;
+ pause;
+ };
+ };
gmac1: mac@1 {
compatible = "mediatek,eth-mac";
reg = <1>;
- phy-handle = <&phy5>;
+ phy-mode = "rgmii";
+
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ pause;
+ };
};
- mdio-bus {
+ mdio: mdio-bus {
#address-cells = <1>;
#size-cells = <0>;
-
- phy5: ethernet-phy@5 {
- reg = <5>;
- phy-mode = "sgmii";
- };
};
};
diff --git a/arch/arm64/boot/dts/mediatek/mt7622.dtsi b/arch/arm64/boot/dts/mediatek/mt7622.dtsi
index d1e13d3..dac51e9 100644
--- a/arch/arm64/boot/dts/mediatek/mt7622.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt7622.dtsi
@@ -931,6 +931,5 @@
"syscon";
reg = <0 0x1b128000 0 0x3000>;
#clock-cells = <1>;
- mediatek,physpeed = "2500";
};
};
diff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h
index cb7ab50..eb73f9f 100644
--- a/arch/arm64/net/bpf_jit.h
+++ b/arch/arm64/net/bpf_jit.h
@@ -171,6 +171,9 @@
/* Rd = Ra + Rn * Rm */
#define A64_MADD(sf, Rd, Ra, Rn, Rm) aarch64_insn_gen_data3(Rd, Ra, Rn, Rm, \
A64_VARIANT(sf), AARCH64_INSN_DATA3_MADD)
+/* Rd = Ra - Rn * Rm */
+#define A64_MSUB(sf, Rd, Ra, Rn, Rm) aarch64_insn_gen_data3(Rd, Ra, Rn, Rm, \
+ A64_VARIANT(sf), AARCH64_INSN_DATA3_MSUB)
/* Rd = Rn * Rm */
#define A64_MUL(sf, Rd, Rn, Rm) A64_MADD(sf, Rd, A64_ZR, Rn, Rm)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index f5b437f..cdc79de 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -409,8 +409,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
break;
case BPF_MOD:
emit(A64_UDIV(is64, tmp, dst, src), ctx);
- emit(A64_MUL(is64, tmp, tmp, src), ctx);
- emit(A64_SUB(is64, dst, dst, tmp), ctx);
+ emit(A64_MSUB(is64, dst, dst, tmp, src), ctx);
break;
}
break;
@@ -516,8 +515,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_ALU64 | BPF_MOD | BPF_K:
emit_a64_mov_i(is64, tmp2, imm, ctx);
emit(A64_UDIV(is64, tmp, dst, tmp2), ctx);
- emit(A64_MUL(is64, tmp, tmp, tmp2), ctx);
- emit(A64_SUB(is64, dst, dst, tmp), ctx);
+ emit(A64_MSUB(is64, dst, dst, tmp, tmp2), ctx);
break;
case BPF_ALU | BPF_LSH | BPF_K:
case BPF_ALU64 | BPF_LSH | BPF_K:
diff --git a/arch/mips/include/asm/octeon/cvmx-sli-defs.h b/arch/mips/include/asm/octeon/cvmx-sli-defs.h
index 52cf96e..cbc7cda 100644
--- a/arch/mips/include/asm/octeon/cvmx-sli-defs.h
+++ b/arch/mips/include/asm/octeon/cvmx-sli-defs.h
@@ -46,6 +46,7 @@ static inline uint64_t CVMX_SLI_PCIE_MSI_RCV_FUNC(void)
case OCTEON_CN78XX & OCTEON_FAMILY_MASK:
if (OCTEON_IS_MODEL(OCTEON_CN78XX_PASS1_X))
return 0x0000000000003CB0ull;
+ /* Else, fall through */
default:
return 0x0000000000023CB0ull;
}
diff --git a/arch/mips/include/asm/sn/ioc3.h b/arch/mips/include/asm/sn/ioc3.h
index 25c8dcc..a947eed 100644
--- a/arch/mips/include/asm/sn/ioc3.h
+++ b/arch/mips/include/asm/sn/ioc3.h
@@ -3,169 +3,161 @@
* Copyright (C) 1999, 2000 Ralf Baechle
* Copyright (C) 1999, 2000 Silicon Graphics, Inc.
*/
-#ifndef _IOC3_H
-#define _IOC3_H
+#ifndef MIPS_SN_IOC3_H
+#define MIPS_SN_IOC3_H
#include <linux/types.h>
-/* SUPERIO uart register map */
-typedef volatile struct ioc3_uartregs {
- union {
- volatile u8 rbr; /* read only, DLAB == 0 */
- volatile u8 thr; /* write only, DLAB == 0 */
- volatile u8 dll; /* DLAB == 1 */
- } u1;
- union {
- volatile u8 ier; /* DLAB == 0 */
- volatile u8 dlm; /* DLAB == 1 */
- } u2;
- union {
- volatile u8 iir; /* read only */
- volatile u8 fcr; /* write only */
- } u3;
- volatile u8 iu_lcr;
- volatile u8 iu_mcr;
- volatile u8 iu_lsr;
- volatile u8 iu_msr;
- volatile u8 iu_scr;
-} ioc3_uregs_t;
+/* serial port register map */
+struct ioc3_serialregs {
+ u32 sscr;
+ u32 stpir;
+ u32 stcir;
+ u32 srpir;
+ u32 srcir;
+ u32 srtr;
+ u32 shadow;
+};
-#define iu_rbr u1.rbr
-#define iu_thr u1.thr
-#define iu_dll u1.dll
-#define iu_ier u2.ier
-#define iu_dlm u2.dlm
-#define iu_iir u3.iir
-#define iu_fcr u3.fcr
+/* SUPERIO uart register map */
+struct ioc3_uartregs {
+ union {
+ u8 iu_rbr; /* read only, DLAB == 0 */
+ u8 iu_thr; /* write only, DLAB == 0 */
+ u8 iu_dll; /* DLAB == 1 */
+ };
+ union {
+ u8 iu_ier; /* DLAB == 0 */
+ u8 iu_dlm; /* DLAB == 1 */
+ };
+ union {
+ u8 iu_iir; /* read only */
+ u8 iu_fcr; /* write only */
+ };
+ u8 iu_lcr;
+ u8 iu_mcr;
+ u8 iu_lsr;
+ u8 iu_msr;
+ u8 iu_scr;
+};
struct ioc3_sioregs {
- volatile u8 fill[0x141]; /* starts at 0x141 */
+ u8 fill[0x141]; /* starts at 0x141 */
- volatile u8 uartc;
- volatile u8 kbdcg;
+ u8 uartc;
+ u8 kbdcg;
- volatile u8 fill0[0x150 - 0x142 - 1];
+ u8 fill0[0x150 - 0x142 - 1];
- volatile u8 pp_data;
- volatile u8 pp_dsr;
- volatile u8 pp_dcr;
+ u8 pp_data;
+ u8 pp_dsr;
+ u8 pp_dcr;
- volatile u8 fill1[0x158 - 0x152 - 1];
+ u8 fill1[0x158 - 0x152 - 1];
- volatile u8 pp_fifa;
- volatile u8 pp_cfgb;
- volatile u8 pp_ecr;
+ u8 pp_fifa;
+ u8 pp_cfgb;
+ u8 pp_ecr;
- volatile u8 fill2[0x168 - 0x15a - 1];
+ u8 fill2[0x168 - 0x15a - 1];
- volatile u8 rtcad;
- volatile u8 rtcdat;
+ u8 rtcad;
+ u8 rtcdat;
- volatile u8 fill3[0x170 - 0x169 - 1];
+ u8 fill3[0x170 - 0x169 - 1];
struct ioc3_uartregs uartb; /* 0x20170 */
struct ioc3_uartregs uarta; /* 0x20178 */
};
+struct ioc3_ethregs {
+ u32 emcr; /* 0x000f0 */
+ u32 eisr; /* 0x000f4 */
+ u32 eier; /* 0x000f8 */
+ u32 ercsr; /* 0x000fc */
+ u32 erbr_h; /* 0x00100 */
+ u32 erbr_l; /* 0x00104 */
+ u32 erbar; /* 0x00108 */
+ u32 ercir; /* 0x0010c */
+ u32 erpir; /* 0x00110 */
+ u32 ertr; /* 0x00114 */
+ u32 etcsr; /* 0x00118 */
+ u32 ersr; /* 0x0011c */
+ u32 etcdc; /* 0x00120 */
+ u32 ebir; /* 0x00124 */
+ u32 etbr_h; /* 0x00128 */
+ u32 etbr_l; /* 0x0012c */
+ u32 etcir; /* 0x00130 */
+ u32 etpir; /* 0x00134 */
+ u32 emar_h; /* 0x00138 */
+ u32 emar_l; /* 0x0013c */
+ u32 ehar_h; /* 0x00140 */
+ u32 ehar_l; /* 0x00144 */
+ u32 micr; /* 0x00148 */
+ u32 midr_r; /* 0x0014c */
+ u32 midr_w; /* 0x00150 */
+};
+
+struct ioc3_serioregs {
+ u32 km_csr; /* 0x0009c */
+ u32 k_rd; /* 0x000a0 */
+ u32 m_rd; /* 0x000a4 */
+ u32 k_wd; /* 0x000a8 */
+ u32 m_wd; /* 0x000ac */
+};
+
/* Register layout of IOC3 in configuration space. */
struct ioc3 {
- volatile u32 pad0[7]; /* 0x00000 */
- volatile u32 sio_ir; /* 0x0001c */
- volatile u32 sio_ies; /* 0x00020 */
- volatile u32 sio_iec; /* 0x00024 */
- volatile u32 sio_cr; /* 0x00028 */
- volatile u32 int_out; /* 0x0002c */
- volatile u32 mcr; /* 0x00030 */
+ /* PCI Config Space registers */
+ u32 pci_id; /* 0x00000 */
+ u32 pci_scr; /* 0x00004 */
+ u32 pci_rev; /* 0x00008 */
+ u32 pci_lat; /* 0x0000c */
+ u32 pci_addr; /* 0x00010 */
+ u32 pci_err_addr_l; /* 0x00014 */
+ u32 pci_err_addr_h; /* 0x00018 */
+
+ u32 sio_ir; /* 0x0001c */
+ u32 sio_ies; /* 0x00020 */
+ u32 sio_iec; /* 0x00024 */
+ u32 sio_cr; /* 0x00028 */
+ u32 int_out; /* 0x0002c */
+ u32 mcr; /* 0x00030 */
/* General Purpose I/O registers */
- volatile u32 gpcr_s; /* 0x00034 */
- volatile u32 gpcr_c; /* 0x00038 */
- volatile u32 gpdr; /* 0x0003c */
- volatile u32 gppr_0; /* 0x00040 */
- volatile u32 gppr_1; /* 0x00044 */
- volatile u32 gppr_2; /* 0x00048 */
- volatile u32 gppr_3; /* 0x0004c */
- volatile u32 gppr_4; /* 0x00050 */
- volatile u32 gppr_5; /* 0x00054 */
- volatile u32 gppr_6; /* 0x00058 */
- volatile u32 gppr_7; /* 0x0005c */
- volatile u32 gppr_8; /* 0x00060 */
- volatile u32 gppr_9; /* 0x00064 */
- volatile u32 gppr_10; /* 0x00068 */
- volatile u32 gppr_11; /* 0x0006c */
- volatile u32 gppr_12; /* 0x00070 */
- volatile u32 gppr_13; /* 0x00074 */
- volatile u32 gppr_14; /* 0x00078 */
- volatile u32 gppr_15; /* 0x0007c */
+ u32 gpcr_s; /* 0x00034 */
+ u32 gpcr_c; /* 0x00038 */
+ u32 gpdr; /* 0x0003c */
+ u32 gppr[16]; /* 0x00040 */
/* Parallel Port Registers */
- volatile u32 ppbr_h_a; /* 0x00080 */
- volatile u32 ppbr_l_a; /* 0x00084 */
- volatile u32 ppcr_a; /* 0x00088 */
- volatile u32 ppcr; /* 0x0008c */
- volatile u32 ppbr_h_b; /* 0x00090 */
- volatile u32 ppbr_l_b; /* 0x00094 */
- volatile u32 ppcr_b; /* 0x00098 */
+ u32 ppbr_h_a; /* 0x00080 */
+ u32 ppbr_l_a; /* 0x00084 */
+ u32 ppcr_a; /* 0x00088 */
+ u32 ppcr; /* 0x0008c */
+ u32 ppbr_h_b; /* 0x00090 */
+ u32 ppbr_l_b; /* 0x00094 */
+ u32 ppcr_b; /* 0x00098 */
/* Keyboard and Mouse Registers */
- volatile u32 km_csr; /* 0x0009c */
- volatile u32 k_rd; /* 0x000a0 */
- volatile u32 m_rd; /* 0x000a4 */
- volatile u32 k_wd; /* 0x000a8 */
- volatile u32 m_wd; /* 0x000ac */
+ struct ioc3_serioregs serio;
/* Serial Port Registers */
- volatile u32 sbbr_h; /* 0x000b0 */
- volatile u32 sbbr_l; /* 0x000b4 */
- volatile u32 sscr_a; /* 0x000b8 */
- volatile u32 stpir_a; /* 0x000bc */
- volatile u32 stcir_a; /* 0x000c0 */
- volatile u32 srpir_a; /* 0x000c4 */
- volatile u32 srcir_a; /* 0x000c8 */
- volatile u32 srtr_a; /* 0x000cc */
- volatile u32 shadow_a; /* 0x000d0 */
- volatile u32 sscr_b; /* 0x000d4 */
- volatile u32 stpir_b; /* 0x000d8 */
- volatile u32 stcir_b; /* 0x000dc */
- volatile u32 srpir_b; /* 0x000e0 */
- volatile u32 srcir_b; /* 0x000e4 */
- volatile u32 srtr_b; /* 0x000e8 */
- volatile u32 shadow_b; /* 0x000ec */
+ u32 sbbr_h; /* 0x000b0 */
+ u32 sbbr_l; /* 0x000b4 */
+ struct ioc3_serialregs port_a;
+ struct ioc3_serialregs port_b;
- /* Ethernet Registers */
- volatile u32 emcr; /* 0x000f0 */
- volatile u32 eisr; /* 0x000f4 */
- volatile u32 eier; /* 0x000f8 */
- volatile u32 ercsr; /* 0x000fc */
- volatile u32 erbr_h; /* 0x00100 */
- volatile u32 erbr_l; /* 0x00104 */
- volatile u32 erbar; /* 0x00108 */
- volatile u32 ercir; /* 0x0010c */
- volatile u32 erpir; /* 0x00110 */
- volatile u32 ertr; /* 0x00114 */
- volatile u32 etcsr; /* 0x00118 */
- volatile u32 ersr; /* 0x0011c */
- volatile u32 etcdc; /* 0x00120 */
- volatile u32 ebir; /* 0x00124 */
- volatile u32 etbr_h; /* 0x00128 */
- volatile u32 etbr_l; /* 0x0012c */
- volatile u32 etcir; /* 0x00130 */
- volatile u32 etpir; /* 0x00134 */
- volatile u32 emar_h; /* 0x00138 */
- volatile u32 emar_l; /* 0x0013c */
- volatile u32 ehar_h; /* 0x00140 */
- volatile u32 ehar_l; /* 0x00144 */
- volatile u32 micr; /* 0x00148 */
- volatile u32 midr_r; /* 0x0014c */
- volatile u32 midr_w; /* 0x00150 */
- volatile u32 pad1[(0x20000 - 0x00154) / 4];
+ /* Ethernet Registers */
+ struct ioc3_ethregs eth;
+ u32 pad1[(0x20000 - 0x00154) / 4];
/* SuperIO Registers XXX */
struct ioc3_sioregs sregs; /* 0x20000 */
- volatile u32 pad2[(0x40000 - 0x20180) / 4];
+ u32 pad2[(0x40000 - 0x20180) / 4];
/* SSRAM Diagnostic Access */
- volatile u32 ssram[(0x80000 - 0x40000) / 4];
+ u32 ssram[(0x80000 - 0x40000) / 4];
/* Bytebus device offsets
0x80000 - Access to the generic devices selected with DEV0
@@ -178,6 +170,20 @@ struct ioc3 {
0xFFFFF bytebus DEV_SEL_3 */
};
+
+#define PCI_LAT 0xc /* Latency Timer */
+#define PCI_SCR_DROP_MODE_EN 0x00008000 /* drop pios on parity err */
+#define UARTA_BASE 0x178
+#define UARTB_BASE 0x170
+
+/*
+ * Bytebus device space
+ */
+#define IOC3_BYTEBUS_DEV0 0x80000L
+#define IOC3_BYTEBUS_DEV1 0xa0000L
+#define IOC3_BYTEBUS_DEV2 0xc0000L
+#define IOC3_BYTEBUS_DEV3 0xe0000L
+
/*
* Ethernet RX Buffer
*/
@@ -233,28 +239,20 @@ struct ioc3_etxd {
#define ETXD_B2CNT_MASK 0x7ff00000
#define ETXD_B2CNT_SHIFT 20
-/*
- * Bytebus device space
- */
-#define IOC3_BYTEBUS_DEV0 0x80000L
-#define IOC3_BYTEBUS_DEV1 0xa0000L
-#define IOC3_BYTEBUS_DEV2 0xc0000L
-#define IOC3_BYTEBUS_DEV3 0xe0000L
-
/* ------------------------------------------------------------------------- */
/* Superio Registers (PIO Access) */
#define IOC3_SIO_BASE 0x20000
#define IOC3_SIO_UARTC (IOC3_SIO_BASE+0x141) /* UART Config */
#define IOC3_SIO_KBDCG (IOC3_SIO_BASE+0x142) /* KBD Config */
-#define IOC3_SIO_PP_BASE (IOC3_SIO_BASE+PP_BASE) /* Parallel Port */
+#define IOC3_SIO_PP_BASE (IOC3_SIO_BASE+PP_BASE) /* Parallel Port */
#define IOC3_SIO_RTC_BASE (IOC3_SIO_BASE+0x168) /* Real Time Clock */
#define IOC3_SIO_UB_BASE (IOC3_SIO_BASE+UARTB_BASE) /* UART B */
#define IOC3_SIO_UA_BASE (IOC3_SIO_BASE+UARTA_BASE) /* UART A */
/* SSRAM Diagnostic Access */
#define IOC3_SSRAM IOC3_RAM_OFF /* base of SSRAM diagnostic access */
-#define IOC3_SSRAM_LEN 0x40000 /* 256kb (address space size, may not be fully populated) */
+#define IOC3_SSRAM_LEN 0x40000 /* 256kb (addrspc sz, may not be populated) */
#define IOC3_SSRAM_DM 0x0000ffff /* data mask */
#define IOC3_SSRAM_PM 0x00010000 /* parity mask */
@@ -294,10 +292,10 @@ struct ioc3_etxd {
SIO_IR to assert */
#define KM_CSR_M_TO_EN 0x00080000 /* KM_CSR_M_TO + KM_CSR_M_TO_EN = cause
SIO_IR to assert */
-#define KM_CSR_K_CLAMP_ONE 0x00100000 /* Pull K_CLK low after rec. one char */
-#define KM_CSR_M_CLAMP_ONE 0x00200000 /* Pull M_CLK low after rec. one char */
-#define KM_CSR_K_CLAMP_THREE 0x00400000 /* Pull K_CLK low after rec. three chars */
-#define KM_CSR_M_CLAMP_THREE 0x00800000 /* Pull M_CLK low after rec. three char */
+#define KM_CSR_K_CLAMP_1 0x00100000 /* Pull K_CLK low aft recv 1 char */
+#define KM_CSR_M_CLAMP_1 0x00200000 /* Pull M_CLK low aft recv 1 char */
+#define KM_CSR_K_CLAMP_3 0x00400000 /* Pull K_CLK low aft recv 3 chars */
+#define KM_CSR_M_CLAMP_3 0x00800000 /* Pull M_CLK low aft recv 3 chars */
/* bitmasks for IOC3_K_RD and IOC3_M_RD */
#define KM_RD_DATA_2 0x000000ff /* 3rd char recvd since last read */
@@ -440,10 +438,6 @@ struct ioc3_etxd {
SIO_IR_PP_INTB | SIO_IR_PP_MEMERR)
#define SIO_IR_RT (SIO_IR_RT_INT | SIO_IR_GEN_INT1)
-/* macro to load pending interrupts */
-#define IOC3_PENDING_INTRS(mem) (PCI_INW(&((mem)->sio_ir)) & \
- PCI_INW(&((mem)->sio_ies_ro)))
-
/* bitmasks for SIO_CR */
#define SIO_CR_SIO_RESET 0x00000001 /* reset the SIO */
#define SIO_CR_SER_A_BASE 0x000000fe /* DMA poll addr port A */
@@ -500,10 +494,11 @@ struct ioc3_etxd {
#define GPCR_UARTB_MODESEL 0x40 /* pin is output to port B mode sel */
#define GPCR_UARTA_MODESEL 0x80 /* pin is output to port A mode sel */
-#define GPPR_PHY_RESET_PIN 5 /* GIO pin controlling phy reset */
-#define GPPR_UARTB_MODESEL_PIN 6 /* GIO pin controlling uart b mode select */
-#define GPPR_UARTA_MODESEL_PIN 7 /* GIO pin controlling uart a mode select */
+#define GPPR_PHY_RESET_PIN 5 /* GIO pin cntrlling phy reset */
+#define GPPR_UARTB_MODESEL_PIN 6 /* GIO pin cntrlling uart b mode sel */
+#define GPPR_UARTA_MODESEL_PIN 7 /* GIO pin cntrlling uart a mode sel */
+/* ethernet */
#define EMCR_DUPLEX 0x00000001
#define EMCR_PROMISC 0x00000002
#define EMCR_PADEN 0x00000004
@@ -595,70 +590,4 @@ struct ioc3_etxd {
#define MIDR_DATA_MASK 0x0000ffff
-#define ERXBUF_IPCKSUM_MASK 0x0000ffff
-#define ERXBUF_BYTECNT_MASK 0x07ff0000
-#define ERXBUF_BYTECNT_SHIFT 16
-#define ERXBUF_V 0x80000000
-
-#define ERXBUF_CRCERR 0x00000001 /* aka RSV15 */
-#define ERXBUF_FRAMERR 0x00000002 /* aka RSV14 */
-#define ERXBUF_CODERR 0x00000004 /* aka RSV13 */
-#define ERXBUF_INVPREAMB 0x00000008 /* aka RSV18 */
-#define ERXBUF_LOLEN 0x00007000 /* aka RSV2_0 */
-#define ERXBUF_HILEN 0x03ff0000 /* aka RSV12_3 */
-#define ERXBUF_MULTICAST 0x04000000 /* aka RSV16 */
-#define ERXBUF_BROADCAST 0x08000000 /* aka RSV17 */
-#define ERXBUF_LONGEVENT 0x10000000 /* aka RSV19 */
-#define ERXBUF_BADPKT 0x20000000 /* aka RSV20 */
-#define ERXBUF_GOODPKT 0x40000000 /* aka RSV21 */
-#define ERXBUF_CARRIER 0x80000000 /* aka RSV22 */
-
-#define ETXD_BYTECNT_MASK 0x000007ff /* total byte count */
-#define ETXD_INTWHENDONE 0x00001000 /* intr when done */
-#define ETXD_D0V 0x00010000 /* data 0 valid */
-#define ETXD_B1V 0x00020000 /* buf 1 valid */
-#define ETXD_B2V 0x00040000 /* buf 2 valid */
-#define ETXD_DOCHECKSUM 0x00080000 /* insert ip cksum */
-#define ETXD_CHKOFF_MASK 0x07f00000 /* cksum byte offset */
-#define ETXD_CHKOFF_SHIFT 20
-
-#define ETXD_D0CNT_MASK 0x0000007f
-#define ETXD_B1CNT_MASK 0x0007ff00
-#define ETXD_B1CNT_SHIFT 8
-#define ETXD_B2CNT_MASK 0x7ff00000
-#define ETXD_B2CNT_SHIFT 20
-
-typedef enum ioc3_subdevs_e {
- ioc3_subdev_ether,
- ioc3_subdev_generic,
- ioc3_subdev_nic,
- ioc3_subdev_kbms,
- ioc3_subdev_ttya,
- ioc3_subdev_ttyb,
- ioc3_subdev_ecpp,
- ioc3_subdev_rt,
- ioc3_nsubdevs
-} ioc3_subdev_t;
-
-/* subdevice disable bits,
- * from the standard INFO_LBL_SUBDEVS
- */
-#define IOC3_SDB_ETHER (1<<ioc3_subdev_ether)
-#define IOC3_SDB_GENERIC (1<<ioc3_subdev_generic)
-#define IOC3_SDB_NIC (1<<ioc3_subdev_nic)
-#define IOC3_SDB_KBMS (1<<ioc3_subdev_kbms)
-#define IOC3_SDB_TTYA (1<<ioc3_subdev_ttya)
-#define IOC3_SDB_TTYB (1<<ioc3_subdev_ttyb)
-#define IOC3_SDB_ECPP (1<<ioc3_subdev_ecpp)
-#define IOC3_SDB_RT (1<<ioc3_subdev_rt)
-
-#define IOC3_ALL_SUBDEVS ((1<<ioc3_nsubdevs)-1)
-
-#define IOC3_SDB_SERIAL (IOC3_SDB_TTYA|IOC3_SDB_TTYB)
-
-#define IOC3_STD_SUBDEVS IOC3_ALL_SUBDEVS
-
-#define IOC3_INTA_SUBDEVS IOC3_SDB_ETHER
-#define IOC3_INTB_SUBDEVS (IOC3_SDB_GENERIC|IOC3_SDB_KBMS|IOC3_SDB_SERIAL|IOC3_SDB_ECPP|IOC3_SDB_RT)
-
-#endif /* _IOC3_H */
+#endif /* MIPS_SN_IOC3_H */
diff --git a/arch/mips/sgi-ip27/ip27-console.c b/arch/mips/sgi-ip27/ip27-console.c
index 6bdb48d..5886bee 100644
--- a/arch/mips/sgi-ip27/ip27-console.c
+++ b/arch/mips/sgi-ip27/ip27-console.c
@@ -35,6 +35,7 @@ void prom_putchar(char c)
{
struct ioc3_uartregs *uart = console_uart();
- while ((uart->iu_lsr & 0x20) == 0);
- uart->iu_thr = c;
+ while ((readb(&uart->iu_lsr) & 0x20) == 0)
+ ;
+ writeb(c, &uart->iu_thr);
}
diff --git a/arch/mips/sgi-ip27/ip27-init.c b/arch/mips/sgi-ip27/ip27-init.c
index 066b33f5..59d5375 100644
--- a/arch/mips/sgi-ip27/ip27-init.c
+++ b/arch/mips/sgi-ip27/ip27-init.c
@@ -130,17 +130,6 @@ cnodeid_t get_compact_nodeid(void)
return NASID_TO_COMPACT_NODEID(get_nasid());
}
-static inline void ioc3_eth_init(void)
-{
- struct ioc3 *ioc3;
- nasid_t nid;
-
- nid = get_nasid();
- ioc3 = (struct ioc3 *) KL_CONFIG_CH_CONS_INFO(nid)->memory_base;
-
- ioc3->eier = 0;
-}
-
extern void ip27_reboot_setup(void);
void __init plat_mem_setup(void)
@@ -182,8 +171,6 @@ void __init plat_mem_setup(void)
panic("Kernel compiled for N mode.");
#endif
- ioc3_eth_init();
-
ioport_resource.start = 0;
ioport_resource.end = ~0UL;
set_io_port_base(IO_BASE);
diff --git a/arch/nds32/kernel/signal.c b/arch/nds32/kernel/signal.c
index fe615139..330b19f 100644
--- a/arch/nds32/kernel/signal.c
+++ b/arch/nds32/kernel/signal.c
@@ -316,6 +316,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs)
regs->uregs[0] = -EINTR;
break;
}
+ /* Else, fall through */
case -ERESTARTNOINTR:
regs->uregs[0] = regs->orig_r0;
regs->ipc -= 4;
@@ -360,6 +361,7 @@ static void do_signal(struct pt_regs *regs)
switch (regs->uregs[0]) {
case -ERESTART_RESTARTBLOCK:
regs->uregs[15] = __NR_restart_syscall;
+ /* Fall through */
case -ERESTARTNOHAND:
case -ERESTARTSYS:
case -ERESTARTNOINTR:
diff --git a/arch/parisc/include/asm/pgtable.h b/arch/parisc/include/asm/pgtable.h
index a39b079..6d58c17 100644
--- a/arch/parisc/include/asm/pgtable.h
+++ b/arch/parisc/include/asm/pgtable.h
@@ -2,6 +2,7 @@
#ifndef _PARISC_PGTABLE_H
#define _PARISC_PGTABLE_H
+#include <asm/page.h>
#include <asm-generic/4level-fixup.h>
#include <asm/fixmap.h>
@@ -98,8 +99,6 @@ static inline void purge_tlb_entries(struct mm_struct *mm, unsigned long addr)
#endif /* !__ASSEMBLY__ */
-#include <asm/page.h>
-
#define pte_ERROR(e) \
printk("%s:%d: bad pte %08lx.\n", __FILE__, __LINE__, pte_val(e))
#define pmd_ERROR(e) \
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index e99a147..c4b606f 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -660,8 +660,10 @@ long kvmppc_h_put_tce_indirect(struct kvm_vcpu *vcpu,
}
tce = be64_to_cpu(tce);
- if (kvmppc_tce_to_ua(vcpu->kvm, tce, &ua))
- return H_PARAMETER;
+ if (kvmppc_tce_to_ua(vcpu->kvm, tce, &ua)) {
+ ret = H_PARAMETER;
+ goto unlock_exit;
+ }
list_for_each_entry_lockless(stit, &stt->iommu_tables, next) {
ret = kvmppc_tce_iommu_map(vcpu->kvm, stt,
diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c
index f50bbee..b4f20f1 100644
--- a/arch/powerpc/kvm/book3s_64_vio_hv.c
+++ b/arch/powerpc/kvm/book3s_64_vio_hv.c
@@ -556,8 +556,10 @@ long kvmppc_rm_h_put_tce_indirect(struct kvm_vcpu *vcpu,
unsigned long tce = be64_to_cpu(((u64 *)tces)[i]);
ua = 0;
- if (kvmppc_rm_tce_to_ua(vcpu->kvm, tce, &ua, NULL))
- return H_PARAMETER;
+ if (kvmppc_rm_tce_to_ua(vcpu->kvm, tce, &ua, NULL)) {
+ ret = H_PARAMETER;
+ goto unlock_exit;
+ }
list_for_each_entry_lockless(stit, &stt->iommu_tables, next) {
ret = kvmppc_rm_tce_iommu_map(vcpu->kvm, stt,
diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h
index 9c66033..161f28d 100644
--- a/arch/riscv/include/asm/fixmap.h
+++ b/arch/riscv/include/asm/fixmap.h
@@ -30,10 +30,6 @@ enum fixed_addresses {
__end_of_fixed_addresses
};
-#define FIXADDR_SIZE (__end_of_fixed_addresses * PAGE_SIZE)
-#define FIXADDR_TOP (VMALLOC_START)
-#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
-
#define FIXMAP_PAGE_IO PAGE_KERNEL
#define __early_set_fixmap __set_fixmap
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index a364aba..c24a083 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -420,14 +420,22 @@ static inline void pgtable_cache_init(void)
#define VMALLOC_END (PAGE_OFFSET - 1)
#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE)
+#define FIXADDR_TOP VMALLOC_START
+#ifdef CONFIG_64BIT
+#define FIXADDR_SIZE PMD_SIZE
+#else
+#define FIXADDR_SIZE PGDIR_SIZE
+#endif
+#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE)
+
/*
- * Task size is 0x4000000000 for RV64 or 0xb800000 for RV32.
+ * Task size is 0x4000000000 for RV64 or 0x9fc00000 for RV32.
* Note that PGDIR_SIZE must evenly divide TASK_SIZE.
*/
#ifdef CONFIG_64BIT
#define TASK_SIZE (PGDIR_SIZE * PTRS_PER_PGD / 2)
#else
-#define TASK_SIZE VMALLOC_START
+#define TASK_SIZE FIXADDR_START
#endif
#include <asm-generic/pgtable.h>
diff --git a/arch/s390/include/asm/qdio.h b/arch/s390/include/asm/qdio.h
index f647d56..78e8a88 100644
--- a/arch/s390/include/asm/qdio.h
+++ b/arch/s390/include/asm/qdio.h
@@ -16,6 +16,7 @@
#define QDIO_MAX_QUEUES_PER_IRQ 4
#define QDIO_MAX_BUFFERS_PER_Q 128
#define QDIO_MAX_BUFFERS_MASK (QDIO_MAX_BUFFERS_PER_Q - 1)
+#define QDIO_BUFNR(num) ((num) & QDIO_MAX_BUFFERS_MASK)
#define QDIO_MAX_ELEMENTS_PER_BUFFER 16
#define QDIO_SBAL_SIZE 256
@@ -359,7 +360,7 @@ struct qdio_initialize {
qdio_handler_t *output_handler;
void (**queue_start_poll_array) (struct ccw_device *, int,
unsigned long);
- int scan_threshold;
+ unsigned int scan_threshold;
unsigned long int_parm;
struct qdio_buffer **input_sbal_addr_array;
struct qdio_buffer **output_sbal_addr_array;
@@ -416,6 +417,9 @@ extern int do_QDIO(struct ccw_device *, unsigned int, int, unsigned int,
extern int qdio_start_irq(struct ccw_device *, int);
extern int qdio_stop_irq(struct ccw_device *, int);
extern int qdio_get_next_buffers(struct ccw_device *, int, int *, int *);
+extern int qdio_inspect_queue(struct ccw_device *cdev, unsigned int nr,
+ bool is_input, unsigned int *bufnr,
+ unsigned int *error);
extern int qdio_shutdown(struct ccw_device *, int);
extern int qdio_free(struct ccw_device *);
extern int qdio_get_ssqd_desc(struct ccw_device *, struct qdio_ssqd_desc *);
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index e636728..ce88211 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -502,7 +502,8 @@ static void bpf_jit_epilogue(struct bpf_jit *jit, u32 stack_depth)
* NOTE: Use noinline because for gcov (-fprofile-arcs) gcc allocates a lot of
* stack space for the large switch statement.
*/
-static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i)
+static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
+ int i, bool extra_pass)
{
struct bpf_insn *insn = &fp->insnsi[i];
int jmp_off, last, insn_count = 1;
@@ -863,7 +864,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i
break;
case BPF_ALU64 | BPF_NEG: /* dst = -dst */
/* lcgr %dst,%dst */
- EMIT4(0xb9130000, dst_reg, dst_reg);
+ EMIT4(0xb9030000, dst_reg, dst_reg);
break;
/*
* BPF_FROM_BE/LE
@@ -1011,10 +1012,14 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i
*/
case BPF_JMP | BPF_CALL:
{
- /*
- * b0 = (__bpf_call_base + imm)(b1, b2, b3, b4, b5)
- */
- const u64 func = (u64)__bpf_call_base + imm;
+ u64 func;
+ bool func_addr_fixed;
+ int ret;
+
+ ret = bpf_jit_get_func_addr(fp, insn, extra_pass,
+ &func, &func_addr_fixed);
+ if (ret < 0)
+ return -1;
REG_SET_SEEN(BPF_REG_5);
jit->seen |= SEEN_FUNC;
@@ -1049,8 +1054,8 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i
/* llgf %w1,map.max_entries(%b2) */
EMIT6_DISP_LH(0xe3000000, 0x0016, REG_W1, REG_0, BPF_REG_2,
offsetof(struct bpf_array, map.max_entries));
- /* clgrj %b3,%w1,0xa,label0: if %b3 >= %w1 goto out */
- EMIT6_PCREL_LABEL(0xec000000, 0x0065, BPF_REG_3,
+ /* clrj %b3,%w1,0xa,label0: if (u32)%b3 >= (u32)%w1 goto out */
+ EMIT6_PCREL_LABEL(0xec000000, 0x0077, BPF_REG_3,
REG_W1, 0, 0xa);
/*
@@ -1076,8 +1081,10 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i
* goto out;
*/
- /* sllg %r1,%b3,3: %r1 = index * 8 */
- EMIT6_DISP_LH(0xeb000000, 0x000d, REG_1, BPF_REG_3, REG_0, 3);
+ /* llgfr %r1,%b3: %r1 = (u32) index */
+ EMIT4(0xb9160000, REG_1, BPF_REG_3);
+ /* sllg %r1,%r1,3: %r1 *= 8 */
+ EMIT6_DISP_LH(0xeb000000, 0x000d, REG_1, REG_1, REG_0, 3);
/* lg %r1,prog(%b2,%r1) */
EMIT6_DISP_LH(0xe3000000, 0x0004, REG_1, BPF_REG_2,
REG_1, offsetof(struct bpf_array, ptrs));
@@ -1281,7 +1288,8 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i
/*
* Compile eBPF program into s390x code
*/
-static int bpf_jit_prog(struct bpf_jit *jit, struct bpf_prog *fp)
+static int bpf_jit_prog(struct bpf_jit *jit, struct bpf_prog *fp,
+ bool extra_pass)
{
int i, insn_count;
@@ -1290,7 +1298,7 @@ static int bpf_jit_prog(struct bpf_jit *jit, struct bpf_prog *fp)
bpf_jit_prologue(jit, fp->aux->stack_depth);
for (i = 0; i < fp->len; i += insn_count) {
- insn_count = bpf_jit_insn(jit, fp, i);
+ insn_count = bpf_jit_insn(jit, fp, i, extra_pass);
if (insn_count < 0)
return -1;
/* Next instruction address */
@@ -1309,6 +1317,12 @@ bool bpf_jit_needs_zext(void)
return true;
}
+struct s390_jit_data {
+ struct bpf_binary_header *header;
+ struct bpf_jit ctx;
+ int pass;
+};
+
/*
* Compile eBPF program "fp"
*/
@@ -1316,7 +1330,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
struct bpf_prog *tmp, *orig_fp = fp;
struct bpf_binary_header *header;
+ struct s390_jit_data *jit_data;
bool tmp_blinded = false;
+ bool extra_pass = false;
struct bpf_jit jit;
int pass;
@@ -1335,6 +1351,23 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fp = tmp;
}
+ jit_data = fp->aux->jit_data;
+ if (!jit_data) {
+ jit_data = kzalloc(sizeof(*jit_data), GFP_KERNEL);
+ if (!jit_data) {
+ fp = orig_fp;
+ goto out;
+ }
+ fp->aux->jit_data = jit_data;
+ }
+ if (jit_data->ctx.addrs) {
+ jit = jit_data->ctx;
+ header = jit_data->header;
+ extra_pass = true;
+ pass = jit_data->pass + 1;
+ goto skip_init_ctx;
+ }
+
memset(&jit, 0, sizeof(jit));
jit.addrs = kcalloc(fp->len + 1, sizeof(*jit.addrs), GFP_KERNEL);
if (jit.addrs == NULL) {
@@ -1347,7 +1380,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
* - 3: Calculate program size and addrs arrray
*/
for (pass = 1; pass <= 3; pass++) {
- if (bpf_jit_prog(&jit, fp)) {
+ if (bpf_jit_prog(&jit, fp, extra_pass)) {
fp = orig_fp;
goto free_addrs;
}
@@ -1359,12 +1392,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fp = orig_fp;
goto free_addrs;
}
+
header = bpf_jit_binary_alloc(jit.size, &jit.prg_buf, 2, jit_fill_hole);
if (!header) {
fp = orig_fp;
goto free_addrs;
}
- if (bpf_jit_prog(&jit, fp)) {
+skip_init_ctx:
+ if (bpf_jit_prog(&jit, fp, extra_pass)) {
bpf_jit_binary_free(header);
fp = orig_fp;
goto free_addrs;
@@ -1373,12 +1408,24 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
bpf_jit_dump(fp->len, jit.size, pass, jit.prg_buf);
print_fn_code(jit.prg_buf, jit.size_prg);
}
- bpf_jit_binary_lock_ro(header);
+ if (!fp->is_func || extra_pass) {
+ bpf_jit_binary_lock_ro(header);
+ } else {
+ jit_data->header = header;
+ jit_data->ctx = jit;
+ jit_data->pass = pass;
+ }
fp->bpf_func = (void *) jit.prg_buf;
fp->jited = 1;
fp->jited_len = jit.size;
+
+ if (!fp->is_func || extra_pass) {
+ bpf_prog_fill_jited_linfo(fp, jit.addrs + 1);
free_addrs:
- kfree(jit.addrs);
+ kfree(jit.addrs);
+ kfree(jit_data);
+ fp->aux->jit_data = NULL;
+ }
out:
if (tmp_blinded)
bpf_jit_prog_release_other(fp, fp == orig_fp ?
diff --git a/arch/um/include/shared/timer-internal.h b/arch/um/include/shared/timer-internal.h
index 8574338..9991ec2 100644
--- a/arch/um/include/shared/timer-internal.h
+++ b/arch/um/include/shared/timer-internal.h
@@ -34,10 +34,13 @@ static inline void time_travel_set_time(unsigned long long ns)
time_travel_time = ns;
}
-static inline void time_travel_set_timer(enum time_travel_timer_mode mode,
- unsigned long long expiry)
+static inline void time_travel_set_timer_mode(enum time_travel_timer_mode mode)
{
time_travel_timer_mode = mode;
+}
+
+static inline void time_travel_set_timer_expiry(unsigned long long expiry)
+{
time_travel_timer_expiry = expiry;
}
#else
@@ -50,8 +53,11 @@ static inline void time_travel_set_time(unsigned long long ns)
{
}
-static inline void time_travel_set_timer(enum time_travel_timer_mode mode,
- unsigned long long expiry)
+static inline void time_travel_set_timer_mode(enum time_travel_timer_mode mode)
+{
+}
+
+static inline void time_travel_set_timer_expiry(unsigned long long expiry)
{
}
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 67c0d1a..6bede78 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -213,7 +213,7 @@ static void time_travel_sleep(unsigned long long duration)
if (time_travel_timer_mode != TT_TMR_DISABLED ||
time_travel_timer_expiry < next) {
if (time_travel_timer_mode == TT_TMR_ONESHOT)
- time_travel_set_timer(TT_TMR_DISABLED, 0);
+ time_travel_set_timer_mode(TT_TMR_DISABLED);
/*
* time_travel_time will be adjusted in the timer
* IRQ handler so it works even when the signal
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 6a051b0..2347572 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -50,7 +50,7 @@ void timer_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
static int itimer_shutdown(struct clock_event_device *evt)
{
if (time_travel_mode != TT_MODE_OFF)
- time_travel_set_timer(TT_TMR_DISABLED, 0);
+ time_travel_set_timer_mode(TT_TMR_DISABLED);
if (time_travel_mode != TT_MODE_INFCPU)
os_timer_disable();
@@ -62,9 +62,10 @@ static int itimer_set_periodic(struct clock_event_device *evt)
{
unsigned long long interval = NSEC_PER_SEC / HZ;
- if (time_travel_mode != TT_MODE_OFF)
- time_travel_set_timer(TT_TMR_PERIODIC,
- time_travel_time + interval);
+ if (time_travel_mode != TT_MODE_OFF) {
+ time_travel_set_timer_mode(TT_TMR_PERIODIC);
+ time_travel_set_timer_expiry(time_travel_time + interval);
+ }
if (time_travel_mode != TT_MODE_INFCPU)
os_timer_set_interval(interval);
@@ -77,9 +78,10 @@ static int itimer_next_event(unsigned long delta,
{
delta += 1;
- if (time_travel_mode != TT_MODE_OFF)
- time_travel_set_timer(TT_TMR_ONESHOT,
- time_travel_time + delta);
+ if (time_travel_mode != TT_MODE_OFF) {
+ time_travel_set_timer_mode(TT_TMR_ONESHOT);
+ time_travel_set_timer_expiry(time_travel_time + delta);
+ }
if (time_travel_mode != TT_MODE_INFCPU)
return os_timer_one_shot(delta);
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 56e748a..94df086 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -38,6 +38,7 @@
REALMODE_CFLAGS += $(call __cc-option, $(CC), $(REALMODE_CFLAGS), -ffreestanding)
REALMODE_CFLAGS += $(call __cc-option, $(CC), $(REALMODE_CFLAGS), -fno-stack-protector)
+REALMODE_CFLAGS += $(call __cc-option, $(CC), $(REALMODE_CFLAGS), -Wno-address-of-packed-member)
REALMODE_CFLAGS += $(call __cc-option, $(CC), $(REALMODE_CFLAGS), $(cc_stack_align4))
export REALMODE_CFLAGS
diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c
index 5f2d030..c886269 100644
--- a/arch/x86/boot/compressed/pgtable_64.c
+++ b/arch/x86/boot/compressed/pgtable_64.c
@@ -72,6 +72,8 @@ static unsigned long find_trampoline_placement(void)
/* Find the first usable memory region under bios_start. */
for (i = boot_params->e820_entries - 1; i >= 0; i--) {
+ unsigned long new = bios_start;
+
entry = &boot_params->e820_table[i];
/* Skip all entries above bios_start. */
@@ -84,15 +86,20 @@ static unsigned long find_trampoline_placement(void)
/* Adjust bios_start to the end of the entry if needed. */
if (bios_start > entry->addr + entry->size)
- bios_start = entry->addr + entry->size;
+ new = entry->addr + entry->size;
/* Keep bios_start page-aligned. */
- bios_start = round_down(bios_start, PAGE_SIZE);
+ new = round_down(new, PAGE_SIZE);
/* Skip the entry if it's too small. */
- if (bios_start - TRAMPOLINE_32BIT_SIZE < entry->addr)
+ if (new - TRAMPOLINE_32BIT_SIZE < entry->addr)
continue;
+ /* Protect against underflow. */
+ if (new - TRAMPOLINE_32BIT_SIZE > bios_start)
+ break;
+
+ bios_start = new;
break;
}
diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c
index 62f317c..5b35b7e 100644
--- a/arch/x86/events/amd/ibs.c
+++ b/arch/x86/events/amd/ibs.c
@@ -661,10 +661,17 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs)
throttle = perf_event_overflow(event, &data, ®s);
out:
- if (throttle)
+ if (throttle) {
perf_ibs_stop(event, 0);
- else
- perf_ibs_enable_event(perf_ibs, hwc, period >> 4);
+ } else {
+ period >>= 4;
+
+ if ((ibs_caps & IBS_CAPS_RDWROPCNT) &&
+ (*config & IBS_OP_CNT_CTL))
+ period |= *config & IBS_OP_CUR_CNT_RAND;
+
+ perf_ibs_enable_event(perf_ibs, hwc, period);
+ }
perf_event_update_userpage(event);
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 81b005e..325959d 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -1236,7 +1236,7 @@ void x86_pmu_enable_event(struct perf_event *event)
* Add a single event to the PMU.
*
* The event is added to the group of enabled events
- * but only if it can be scehduled with existing events.
+ * but only if it can be scheduled with existing events.
*/
static int x86_pmu_add(struct perf_event *event, int flags)
{
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 648260b5..e4c2cb6 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3572,6 +3572,11 @@ static u64 bdw_limit_period(struct perf_event *event, u64 left)
return left;
}
+static u64 nhm_limit_period(struct perf_event *event, u64 left)
+{
+ return max(left, 32ULL);
+}
+
PMU_FORMAT_ATTR(event, "config:0-7" );
PMU_FORMAT_ATTR(umask, "config:8-15" );
PMU_FORMAT_ATTR(edge, "config:18" );
@@ -4606,6 +4611,7 @@ __init int intel_pmu_init(void)
x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;
x86_pmu.enable_all = intel_pmu_nhm_enable_all;
x86_pmu.extra_regs = intel_nehalem_extra_regs;
+ x86_pmu.limit_period = nhm_limit_period;
mem_attr = nhm_mem_events_attrs;
diff --git a/arch/x86/include/asm/bootparam_utils.h b/arch/x86/include/asm/bootparam_utils.h
index f5e90a8..9e5f3c7 100644
--- a/arch/x86/include/asm/bootparam_utils.h
+++ b/arch/x86/include/asm/bootparam_utils.h
@@ -59,7 +59,6 @@ static void sanitize_boot_params(struct boot_params *boot_params)
BOOT_PARAM_PRESERVE(apm_bios_info),
BOOT_PARAM_PRESERVE(tboot_addr),
BOOT_PARAM_PRESERVE(ist_info),
- BOOT_PARAM_PRESERVE(acpi_rsdp_addr),
BOOT_PARAM_PRESERVE(hd0_info),
BOOT_PARAM_PRESERVE(hd1_info),
BOOT_PARAM_PRESERVE(sys_desc_table),
@@ -71,6 +70,7 @@ static void sanitize_boot_params(struct boot_params *boot_params)
BOOT_PARAM_PRESERVE(eddbuf_entries),
BOOT_PARAM_PRESERVE(edd_mbr_sig_buf_entries),
BOOT_PARAM_PRESERVE(edd_mbr_sig_buffer),
+ BOOT_PARAM_PRESERVE(hdr),
BOOT_PARAM_PRESERVE(e820_table),
BOOT_PARAM_PRESERVE(eddbuf),
};
diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
index 287f1f7..c38a666 100644
--- a/arch/x86/include/asm/ftrace.h
+++ b/arch/x86/include/asm/ftrace.h
@@ -16,7 +16,6 @@
#define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
#ifndef __ASSEMBLY__
-extern void mcount(void);
extern atomic_t modifying_ftrace_code;
extern void __fentry__(void);
diff --git a/arch/x86/include/asm/intel-family.h b/arch/x86/include/asm/intel-family.h
index 0278aa6..fe7c205 100644
--- a/arch/x86/include/asm/intel-family.h
+++ b/arch/x86/include/asm/intel-family.h
@@ -11,6 +11,21 @@
* While adding a new CPUID for a new microarchitecture, add a new
* group to keep logically sorted out in chronological order. Within
* that group keep the CPUID for the variants sorted by model number.
+ *
+ * The defined symbol names have the following form:
+ * INTEL_FAM6{OPTFAMILY}_{MICROARCH}{OPTDIFF}
+ * where:
+ * OPTFAMILY Describes the family of CPUs that this belongs to. Default
+ * is assumed to be "_CORE" (and should be omitted). Other values
+ * currently in use are _ATOM and _XEON_PHI
+ * MICROARCH Is the code name for the micro-architecture for this core.
+ * N.B. Not the platform name.
+ * OPTDIFF If needed, a short string to differentiate by market segment.
+ * Exact strings here will vary over time. _DESKTOP, _MOBILE, and
+ * _X (short for Xeon server) should be used when they are
+ * appropriate.
+ *
+ * The #define line may optionally include a comment including platform names.
*/
#define INTEL_FAM6_CORE_YONAH 0x0E
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 6b4fc27..271d837 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -381,6 +381,7 @@
#define MSR_AMD64_PATCH_LEVEL 0x0000008b
#define MSR_AMD64_TSC_RATIO 0xc0000104
#define MSR_AMD64_NB_CFG 0xc001001f
+#define MSR_AMD64_CPUID_FN_1 0xc0011004
#define MSR_AMD64_PATCH_LOADER 0xc0010020
#define MSR_AMD64_OSVW_ID_LENGTH 0xc0010140
#define MSR_AMD64_OSVW_STATUS 0xc0010141
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 109f974..80bc209 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -192,7 +192,7 @@
" lfence;\n" \
" jmp 902b;\n" \
" .align 16\n" \
- "903: addl $4, %%esp;\n" \
+ "903: lea 4(%%esp), %%esp;\n" \
" pushl %[thunk_target];\n" \
" ret;\n" \
" .align 16\n" \
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index 1392d5e..ee26e92 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -252,16 +252,20 @@ struct pebs_lbr {
#define IBSCTL_LVT_OFFSET_VALID (1ULL<<8)
#define IBSCTL_LVT_OFFSET_MASK 0x0F
-/* ibs fetch bits/masks */
+/* IBS fetch bits/masks */
#define IBS_FETCH_RAND_EN (1ULL<<57)
#define IBS_FETCH_VAL (1ULL<<49)
#define IBS_FETCH_ENABLE (1ULL<<48)
#define IBS_FETCH_CNT 0xFFFF0000ULL
#define IBS_FETCH_MAX_CNT 0x0000FFFFULL
-/* ibs op bits/masks */
-/* lower 4 bits of the current count are ignored: */
-#define IBS_OP_CUR_CNT (0xFFFF0ULL<<32)
+/*
+ * IBS op bits/masks
+ * The lower 7 bits of the current count are random bits
+ * preloaded by hardware and ignored in software
+ */
+#define IBS_OP_CUR_CNT (0xFFF80ULL<<32)
+#define IBS_OP_CUR_CNT_RAND (0x0007FULL<<32)
#define IBS_OP_CNT_CTL (1ULL<<19)
#define IBS_OP_VAL (1ULL<<18)
#define IBS_OP_ENABLE (1ULL<<17)
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index f529136..dba2828 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -722,7 +722,7 @@ static __initdata unsigned long lapic_cal_pm1, lapic_cal_pm2;
static __initdata unsigned long lapic_cal_j1, lapic_cal_j2;
/*
- * Temporary interrupt handler.
+ * Temporary interrupt handler and polled calibration function.
*/
static void __init lapic_cal_handler(struct clock_event_device *dev)
{
@@ -851,7 +851,8 @@ bool __init apic_needs_pit(void)
static int __init calibrate_APIC_clock(void)
{
struct clock_event_device *levt = this_cpu_ptr(&lapic_events);
- void (*real_handler)(struct clock_event_device *dev);
+ u64 tsc_perj = 0, tsc_start = 0;
+ unsigned long jif_start;
unsigned long deltaj;
long delta, deltatsc;
int pm_referenced = 0;
@@ -878,29 +879,65 @@ static int __init calibrate_APIC_clock(void)
apic_printk(APIC_VERBOSE, "Using local APIC timer interrupts.\n"
"calibrating APIC timer ...\n");
+ /*
+ * There are platforms w/o global clockevent devices. Instead of
+ * making the calibration conditional on that, use a polling based
+ * approach everywhere.
+ */
local_irq_disable();
- /* Replace the global interrupt handler */
- real_handler = global_clock_event->event_handler;
- global_clock_event->event_handler = lapic_cal_handler;
-
/*
* Setup the APIC counter to maximum. There is no way the lapic
* can underflow in the 100ms detection time frame
*/
__setup_APIC_LVTT(0xffffffff, 0, 0);
- /* Let the interrupts run */
+ /*
+ * Methods to terminate the calibration loop:
+ * 1) Global clockevent if available (jiffies)
+ * 2) TSC if available and frequency is known
+ */
+ jif_start = READ_ONCE(jiffies);
+
+ if (tsc_khz) {
+ tsc_start = rdtsc();
+ tsc_perj = div_u64((u64)tsc_khz * 1000, HZ);
+ }
+
+ /*
+ * Enable interrupts so the tick can fire, if a global
+ * clockevent device is available
+ */
local_irq_enable();
- while (lapic_cal_loops <= LAPIC_CAL_LOOPS)
- cpu_relax();
+ while (lapic_cal_loops <= LAPIC_CAL_LOOPS) {
+ /* Wait for a tick to elapse */
+ while (1) {
+ if (tsc_khz) {
+ u64 tsc_now = rdtsc();
+ if ((tsc_now - tsc_start) >= tsc_perj) {
+ tsc_start += tsc_perj;
+ break;
+ }
+ } else {
+ unsigned long jif_now = READ_ONCE(jiffies);
+
+ if (time_after(jif_now, jif_start)) {
+ jif_start = jif_now;
+ break;
+ }
+ }
+ cpu_relax();
+ }
+
+ /* Invoke the calibration routine */
+ local_irq_disable();
+ lapic_cal_handler(NULL);
+ local_irq_enable();
+ }
local_irq_disable();
- /* Restore the real event handler */
- global_clock_event->event_handler = real_handler;
-
/* Build delta t1-t2 as apic timer counts down */
delta = lapic_cal_t1 - lapic_cal_t2;
apic_printk(APIC_VERBOSE, "... lapic delta = %ld\n", delta);
@@ -943,10 +980,11 @@ static int __init calibrate_APIC_clock(void)
levt->features &= ~CLOCK_EVT_FEAT_DUMMY;
/*
- * PM timer calibration failed or not turned on
- * so lets try APIC timer based calibration
+ * PM timer calibration failed or not turned on so lets try APIC
+ * timer based calibration, if a global clockevent device is
+ * available.
*/
- if (!pm_referenced) {
+ if (!pm_referenced && global_clock_event) {
apic_printk(APIC_VERBOSE, "... verify APIC timer\n");
/*
@@ -1141,6 +1179,10 @@ void clear_local_APIC(void)
apic_write(APIC_LVT0, v | APIC_LVT_MASKED);
v = apic_read(APIC_LVT1);
apic_write(APIC_LVT1, v | APIC_LVT_MASKED);
+ if (!x2apic_enabled()) {
+ v = apic_read(APIC_LDR) & ~APIC_LDR_MASK;
+ apic_write(APIC_LDR, v);
+ }
if (maxlvt >= 4) {
v = apic_read(APIC_LVTPC);
apic_write(APIC_LVTPC, v | APIC_LVT_MASKED);
diff --git a/arch/x86/kernel/apic/bigsmp_32.c b/arch/x86/kernel/apic/bigsmp_32.c
index afee386..caedd8d 100644
--- a/arch/x86/kernel/apic/bigsmp_32.c
+++ b/arch/x86/kernel/apic/bigsmp_32.c
@@ -38,32 +38,12 @@ static int bigsmp_early_logical_apicid(int cpu)
return early_per_cpu(x86_cpu_to_apicid, cpu);
}
-static inline unsigned long calculate_ldr(int cpu)
-{
- unsigned long val, id;
-
- val = apic_read(APIC_LDR) & ~APIC_LDR_MASK;
- id = per_cpu(x86_bios_cpu_apicid, cpu);
- val |= SET_APIC_LOGICAL_ID(id);
-
- return val;
-}
-
/*
- * Set up the logical destination ID.
- *
- * Intel recommends to set DFR, LDR and TPR before enabling
- * an APIC. See e.g. "AP-388 82489DX User's Manual" (Intel
- * document number 292116). So here it goes...
+ * bigsmp enables physical destination mode
+ * and doesn't use LDR and DFR
*/
static void bigsmp_init_apic_ldr(void)
{
- unsigned long val;
- int cpu = smp_processor_id();
-
- apic_write(APIC_DFR, APIC_DFR_FLAT);
- val = calculate_ldr(cpu);
- apic_write(APIC_LDR, val);
}
static void bigsmp_setup_apic_routing(void)
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index c7bb6c6..d6af97f 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -2438,7 +2438,13 @@ unsigned int arch_dynirq_lower_bound(unsigned int from)
* dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use
* gsi_top if ioapic_dynirq_base hasn't been initialized yet.
*/
- return ioapic_initialized ? ioapic_dynirq_base : gsi_top;
+ if (!ioapic_initialized)
+ return gsi_top;
+ /*
+ * For DT enabled machines ioapic_dynirq_base is irrelevant and not
+ * updated. So simply return @from if ioapic_dynirq_base == 0.
+ */
+ return ioapic_dynirq_base ? : from;
}
#ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 8d4e5042..68c363c 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -804,6 +804,64 @@ static void init_amd_ln(struct cpuinfo_x86 *c)
msr_set_bit(MSR_AMD64_DE_CFG, 31);
}
+static bool rdrand_force;
+
+static int __init rdrand_cmdline(char *str)
+{
+ if (!str)
+ return -EINVAL;
+
+ if (!strcmp(str, "force"))
+ rdrand_force = true;
+ else
+ return -EINVAL;
+
+ return 0;
+}
+early_param("rdrand", rdrand_cmdline);
+
+static void clear_rdrand_cpuid_bit(struct cpuinfo_x86 *c)
+{
+ /*
+ * Saving of the MSR used to hide the RDRAND support during
+ * suspend/resume is done by arch/x86/power/cpu.c, which is
+ * dependent on CONFIG_PM_SLEEP.
+ */
+ if (!IS_ENABLED(CONFIG_PM_SLEEP))
+ return;
+
+ /*
+ * The nordrand option can clear X86_FEATURE_RDRAND, so check for
+ * RDRAND support using the CPUID function directly.
+ */
+ if (!(cpuid_ecx(1) & BIT(30)) || rdrand_force)
+ return;
+
+ msr_clear_bit(MSR_AMD64_CPUID_FN_1, 62);
+
+ /*
+ * Verify that the CPUID change has occurred in case the kernel is
+ * running virtualized and the hypervisor doesn't support the MSR.
+ */
+ if (cpuid_ecx(1) & BIT(30)) {
+ pr_info_once("BIOS may not properly restore RDRAND after suspend, but hypervisor does not support hiding RDRAND via CPUID.\n");
+ return;
+ }
+
+ clear_cpu_cap(c, X86_FEATURE_RDRAND);
+ pr_info_once("BIOS may not properly restore RDRAND after suspend, hiding RDRAND via CPUID. Use rdrand=force to reenable.\n");
+}
+
+static void init_amd_jg(struct cpuinfo_x86 *c)
+{
+ /*
+ * Some BIOS implementations do not restore proper RDRAND support
+ * across suspend and resume. Check on whether to hide the RDRAND
+ * instruction support via CPUID.
+ */
+ clear_rdrand_cpuid_bit(c);
+}
+
static void init_amd_bd(struct cpuinfo_x86 *c)
{
u64 value;
@@ -818,6 +876,13 @@ static void init_amd_bd(struct cpuinfo_x86 *c)
wrmsrl_safe(MSR_F15H_IC_CFG, value);
}
}
+
+ /*
+ * Some BIOS implementations do not restore proper RDRAND support
+ * across suspend and resume. Check on whether to hide the RDRAND
+ * instruction support via CPUID.
+ */
+ clear_rdrand_cpuid_bit(c);
}
static void init_amd_zn(struct cpuinfo_x86 *c)
@@ -860,6 +925,7 @@ static void init_amd(struct cpuinfo_x86 *c)
case 0x10: init_amd_gh(c); break;
case 0x12: init_amd_ln(c); break;
case 0x15: init_amd_bd(c); break;
+ case 0x16: init_amd_jg(c); break;
case 0x17: init_amd_zn(c); break;
}
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index d8359eb..8cd745e 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -508,9 +508,12 @@ struct uprobe_xol_ops {
void (*abort)(struct arch_uprobe *, struct pt_regs *);
};
-static inline int sizeof_long(void)
+static inline int sizeof_long(struct pt_regs *regs)
{
- return in_ia32_syscall() ? 4 : 8;
+ /*
+ * Check registers for mode as in_xxx_syscall() does not apply here.
+ */
+ return user_64bit_mode(regs) ? 8 : 4;
}
static int default_pre_xol_op(struct arch_uprobe *auprobe, struct pt_regs *regs)
@@ -521,9 +524,9 @@ static int default_pre_xol_op(struct arch_uprobe *auprobe, struct pt_regs *regs)
static int emulate_push_stack(struct pt_regs *regs, unsigned long val)
{
- unsigned long new_sp = regs->sp - sizeof_long();
+ unsigned long new_sp = regs->sp - sizeof_long(regs);
- if (copy_to_user((void __user *)new_sp, &val, sizeof_long()))
+ if (copy_to_user((void __user *)new_sp, &val, sizeof_long(regs)))
return -EFAULT;
regs->sp = new_sp;
@@ -556,7 +559,7 @@ static int default_post_xol_op(struct arch_uprobe *auprobe, struct pt_regs *regs
long correction = utask->vaddr - utask->xol_vaddr;
regs->ip += correction;
} else if (auprobe->defparam.fixups & UPROBE_FIX_CALL) {
- regs->sp += sizeof_long(); /* Pop incorrect return address */
+ regs->sp += sizeof_long(regs); /* Pop incorrect return address */
if (emulate_push_stack(regs, utask->vaddr + auprobe->defparam.ilen))
return -ERESTART;
}
@@ -675,7 +678,7 @@ static int branch_post_xol_op(struct arch_uprobe *auprobe, struct pt_regs *regs)
* "call" insn was executed out-of-line. Just restore ->sp and restart.
* We could also restore ->ip and try to call branch_emulate_op() again.
*/
- regs->sp += sizeof_long();
+ regs->sp += sizeof_long(regs);
return -ERESTART;
}
@@ -1056,7 +1059,7 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
unsigned long
arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs)
{
- int rasize = sizeof_long(), nleft;
+ int rasize = sizeof_long(regs), nleft;
unsigned long orig_ret_vaddr = 0; /* clear high bits for 32-bit apps */
if (copy_from_user(&orig_ret_vaddr, (void __user *)regs->sp, rasize))
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index c10a8b1..fff790a 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1781,7 +1781,7 @@ int kvm_vm_ioctl_hv_eventfd(struct kvm *kvm, struct kvm_hyperv_eventfd *args)
int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
struct kvm_cpuid_entry2 __user *entries)
{
- uint16_t evmcs_ver = kvm_x86_ops->nested_get_evmcs_version(vcpu);
+ uint16_t evmcs_ver = 0;
struct kvm_cpuid_entry2 cpuid_entries[] = {
{ .function = HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS },
{ .function = HYPERV_CPUID_INTERFACE },
@@ -1793,6 +1793,9 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
};
int i, nent = ARRAY_SIZE(cpuid_entries);
+ if (kvm_x86_ops->nested_get_evmcs_version)
+ evmcs_ver = kvm_x86_ops->nested_get_evmcs_version(vcpu);
+
/* Skip NESTED_FEATURES if eVMCS is not supported */
if (!evmcs_ver)
--nent;
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 685d17c..e904ff0 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -216,6 +216,9 @@ static void recalculate_apic_map(struct kvm *kvm)
if (!apic_x2apic_mode(apic) && !new->phys_map[xapic_id])
new->phys_map[xapic_id] = apic;
+ if (!kvm_apic_sw_enabled(apic))
+ continue;
+
ldr = kvm_lapic_get_reg(apic, APIC_LDR);
if (apic_x2apic_mode(apic)) {
@@ -258,6 +261,8 @@ static inline void apic_set_spiv(struct kvm_lapic *apic, u32 val)
static_key_slow_dec_deferred(&apic_sw_disabled);
else
static_key_slow_inc(&apic_sw_disabled.key);
+
+ recalculate_apic_map(apic->vcpu->kvm);
}
}
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 24843cf..218b277 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -5653,38 +5653,7 @@ static void kvm_mmu_invalidate_zap_pages_in_memslot(struct kvm *kvm,
struct kvm_memory_slot *slot,
struct kvm_page_track_notifier_node *node)
{
- struct kvm_mmu_page *sp;
- LIST_HEAD(invalid_list);
- unsigned long i;
- bool flush;
- gfn_t gfn;
-
- spin_lock(&kvm->mmu_lock);
-
- if (list_empty(&kvm->arch.active_mmu_pages))
- goto out_unlock;
-
- flush = slot_handle_all_level(kvm, slot, kvm_zap_rmapp, false);
-
- for (i = 0; i < slot->npages; i++) {
- gfn = slot->base_gfn + i;
-
- for_each_valid_sp(kvm, sp, gfn) {
- if (sp->gfn != gfn)
- continue;
-
- kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list);
- }
- if (need_resched() || spin_needbreak(&kvm->mmu_lock)) {
- kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, flush);
- flush = false;
- cond_resched_lock(&kvm->mmu_lock);
- }
- }
- kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, flush);
-
-out_unlock:
- spin_unlock(&kvm->mmu_lock);
+ kvm_mmu_zap_all(kvm);
}
void kvm_mmu_init_vm(struct kvm *kvm)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index d685491..e036807 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -1714,7 +1714,6 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu)
if (!entry)
return -EINVAL;
- new_entry = READ_ONCE(*entry);
new_entry = __sme_set((page_to_phys(svm->avic_backing_page) &
AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
AVIC_PHYSICAL_ID_ENTRY_VALID_MASK);
@@ -7129,12 +7128,6 @@ static int svm_unregister_enc_region(struct kvm *kvm,
return ret;
}
-static uint16_t nested_get_evmcs_version(struct kvm_vcpu *vcpu)
-{
- /* Not supported */
- return 0;
-}
-
static int nested_enable_evmcs(struct kvm_vcpu *vcpu,
uint16_t *vmcs_version)
{
@@ -7333,7 +7326,7 @@ static struct kvm_x86_ops svm_x86_ops __ro_after_init = {
.mem_enc_unreg_region = svm_unregister_enc_region,
.nested_enable_evmcs = nested_enable_evmcs,
- .nested_get_evmcs_version = nested_get_evmcs_version,
+ .nested_get_evmcs_version = NULL,
.need_emulation_on_page_fault = svm_need_emulation_on_page_fault,
};
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 42ed3fa..c030c96 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7797,6 +7797,7 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = {
.set_nested_state = NULL,
.get_vmcs12_pages = NULL,
.nested_enable_evmcs = NULL,
+ .nested_get_evmcs_version = NULL,
.need_emulation_on_page_fault = vmx_need_emulation_on_page_fault,
};
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 93b0bd4..290c3c3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6594,12 +6594,13 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,
unsigned long rflags = kvm_x86_ops->get_rflags(vcpu);
toggle_interruptibility(vcpu, ctxt->interruptibility);
vcpu->arch.emulate_regs_need_sync_to_vcpu = false;
- kvm_rip_write(vcpu, ctxt->eip);
- if (r == EMULATE_DONE && ctxt->tf)
- kvm_vcpu_do_singlestep(vcpu, &r);
if (!ctxt->have_exception ||
- exception_type(ctxt->exception.vector) == EXCPT_TRAP)
+ exception_type(ctxt->exception.vector) == EXCPT_TRAP) {
+ kvm_rip_write(vcpu, ctxt->eip);
+ if (r == EMULATE_DONE && ctxt->tf)
+ kvm_vcpu_do_singlestep(vcpu, &r);
__kvm_set_rflags(vcpu, ctxt->eflags);
+ }
/*
* For STI, interrupts are shadowed; so KVM_REQ_EVENT will
diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c
index 6a9a77a..e14e95e 100644
--- a/arch/x86/mm/pageattr.c
+++ b/arch/x86/mm/pageattr.c
@@ -516,7 +516,7 @@ static inline void check_conflict(int warnlvl, pgprot_t prot, pgprotval_t val,
*/
static inline pgprot_t static_protections(pgprot_t prot, unsigned long start,
unsigned long pfn, unsigned long npg,
- int warnlvl)
+ unsigned long lpsize, int warnlvl)
{
pgprotval_t forbidden, res;
unsigned long end;
@@ -535,9 +535,17 @@ static inline pgprot_t static_protections(pgprot_t prot, unsigned long start,
check_conflict(warnlvl, prot, res, start, end, pfn, "Text NX");
forbidden = res;
- res = protect_kernel_text_ro(start, end);
- check_conflict(warnlvl, prot, res, start, end, pfn, "Text RO");
- forbidden |= res;
+ /*
+ * Special case to preserve a large page. If the change spawns the
+ * full large page mapping then there is no point to split it
+ * up. Happens with ftrace and is going to be removed once ftrace
+ * switched to text_poke().
+ */
+ if (lpsize != (npg * PAGE_SIZE) || (start & (lpsize - 1))) {
+ res = protect_kernel_text_ro(start, end);
+ check_conflict(warnlvl, prot, res, start, end, pfn, "Text RO");
+ forbidden |= res;
+ }
/* Check the PFN directly */
res = protect_pci_bios(pfn, pfn + npg - 1);
@@ -819,7 +827,7 @@ static int __should_split_large_page(pte_t *kpte, unsigned long address,
* extra conditional required here.
*/
chk_prot = static_protections(old_prot, lpaddr, old_pfn, numpages,
- CPA_CONFLICT);
+ psize, CPA_CONFLICT);
if (WARN_ON_ONCE(pgprot_val(chk_prot) != pgprot_val(old_prot))) {
/*
@@ -855,7 +863,7 @@ static int __should_split_large_page(pte_t *kpte, unsigned long address,
* protection requirement in the large page.
*/
new_prot = static_protections(req_prot, lpaddr, old_pfn, numpages,
- CPA_DETECT);
+ psize, CPA_DETECT);
/*
* If there is a conflict, split the large page.
@@ -906,7 +914,8 @@ static void split_set_pte(struct cpa_data *cpa, pte_t *pte, unsigned long pfn,
if (!cpa->force_static_prot)
goto set;
- prot = static_protections(ref_prot, address, pfn, npg, CPA_PROTECT);
+ /* Hand in lpsize = 0 to enforce the protection mechanism */
+ prot = static_protections(ref_prot, address, pfn, npg, 0, CPA_PROTECT);
if (pgprot_val(prot) == pgprot_val(ref_prot))
goto set;
@@ -1503,7 +1512,8 @@ static int __change_page_attr(struct cpa_data *cpa, int primary)
pgprot_val(new_prot) |= pgprot_val(cpa->mask_set);
cpa_inc_4k_install();
- new_prot = static_protections(new_prot, address, pfn, 1,
+ /* Hand in lpsize = 0 to enforce the protection mechanism */
+ new_prot = static_protections(new_prot, address, pfn, 1, 0,
CPA_PROTECT);
new_prot = pgprot_clear_protnone_bits(new_prot);
diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c
index 24b079e..c9ef6a7 100644
--- a/arch/x86/power/cpu.c
+++ b/arch/x86/power/cpu.c
@@ -12,6 +12,7 @@
#include <linux/smp.h>
#include <linux/perf_event.h>
#include <linux/tboot.h>
+#include <linux/dmi.h>
#include <asm/pgtable.h>
#include <asm/proto.h>
@@ -23,7 +24,7 @@
#include <asm/debugreg.h>
#include <asm/cpu.h>
#include <asm/mmu_context.h>
-#include <linux/dmi.h>
+#include <asm/cpu_device_id.h>
#ifdef CONFIG_X86_32
__visible unsigned long saved_context_ebx;
@@ -397,15 +398,14 @@ static int __init bsp_pm_check_init(void)
core_initcall(bsp_pm_check_init);
-static int msr_init_context(const u32 *msr_id, const int total_num)
+static int msr_build_context(const u32 *msr_id, const int num)
{
- int i = 0;
+ struct saved_msrs *saved_msrs = &saved_context.saved_msrs;
struct saved_msr *msr_array;
+ int total_num;
+ int i, j;
- if (saved_context.saved_msrs.array || saved_context.saved_msrs.num > 0) {
- pr_err("x86/pm: MSR quirk already applied, please check your DMI match table.\n");
- return -EINVAL;
- }
+ total_num = saved_msrs->num + num;
msr_array = kmalloc_array(total_num, sizeof(struct saved_msr), GFP_KERNEL);
if (!msr_array) {
@@ -413,19 +413,30 @@ static int msr_init_context(const u32 *msr_id, const int total_num)
return -ENOMEM;
}
- for (i = 0; i < total_num; i++) {
- msr_array[i].info.msr_no = msr_id[i];
+ if (saved_msrs->array) {
+ /*
+ * Multiple callbacks can invoke this function, so copy any
+ * MSR save requests from previous invocations.
+ */
+ memcpy(msr_array, saved_msrs->array,
+ sizeof(struct saved_msr) * saved_msrs->num);
+
+ kfree(saved_msrs->array);
+ }
+
+ for (i = saved_msrs->num, j = 0; i < total_num; i++, j++) {
+ msr_array[i].info.msr_no = msr_id[j];
msr_array[i].valid = false;
msr_array[i].info.reg.q = 0;
}
- saved_context.saved_msrs.num = total_num;
- saved_context.saved_msrs.array = msr_array;
+ saved_msrs->num = total_num;
+ saved_msrs->array = msr_array;
return 0;
}
/*
- * The following section is a quirk framework for problematic BIOSen:
+ * The following sections are a quirk framework for problematic BIOSen:
* Sometimes MSRs are modified by the BIOSen after suspended to
* RAM, this might cause unexpected behavior after wakeup.
* Thus we save/restore these specified MSRs across suspend/resume
@@ -440,7 +451,7 @@ static int msr_initialize_bdw(const struct dmi_system_id *d)
u32 bdw_msr_id[] = { MSR_IA32_THERM_CONTROL };
pr_info("x86/pm: %s detected, MSR saving is needed during suspending.\n", d->ident);
- return msr_init_context(bdw_msr_id, ARRAY_SIZE(bdw_msr_id));
+ return msr_build_context(bdw_msr_id, ARRAY_SIZE(bdw_msr_id));
}
static const struct dmi_system_id msr_save_dmi_table[] = {
@@ -455,9 +466,58 @@ static const struct dmi_system_id msr_save_dmi_table[] = {
{}
};
+static int msr_save_cpuid_features(const struct x86_cpu_id *c)
+{
+ u32 cpuid_msr_id[] = {
+ MSR_AMD64_CPUID_FN_1,
+ };
+
+ pr_info("x86/pm: family %#hx cpu detected, MSR saving is needed during suspending.\n",
+ c->family);
+
+ return msr_build_context(cpuid_msr_id, ARRAY_SIZE(cpuid_msr_id));
+}
+
+static const struct x86_cpu_id msr_save_cpu_table[] = {
+ {
+ .vendor = X86_VENDOR_AMD,
+ .family = 0x15,
+ .model = X86_MODEL_ANY,
+ .feature = X86_FEATURE_ANY,
+ .driver_data = (kernel_ulong_t)msr_save_cpuid_features,
+ },
+ {
+ .vendor = X86_VENDOR_AMD,
+ .family = 0x16,
+ .model = X86_MODEL_ANY,
+ .feature = X86_FEATURE_ANY,
+ .driver_data = (kernel_ulong_t)msr_save_cpuid_features,
+ },
+ {}
+};
+
+typedef int (*pm_cpu_match_t)(const struct x86_cpu_id *);
+static int pm_cpu_check(const struct x86_cpu_id *c)
+{
+ const struct x86_cpu_id *m;
+ int ret = 0;
+
+ m = x86_match_cpu(msr_save_cpu_table);
+ if (m) {
+ pm_cpu_match_t fn;
+
+ fn = (pm_cpu_match_t)m->driver_data;
+ ret = fn(m);
+ }
+
+ return ret;
+}
+
static int pm_check_save_msr(void)
{
dmi_check_system(msr_save_dmi_table);
+ pm_cpu_check(msr_save_cpu_table);
+
return 0;
}
diff --git a/drivers/atm/Kconfig b/drivers/atm/Kconfig
index 2e2efa5..8c37294 100644
--- a/drivers/atm/Kconfig
+++ b/drivers/atm/Kconfig
@@ -200,7 +200,7 @@
make the card work).
config ATM_NICSTAR_USE_IDT77105
- bool "Use IDT77015 PHY driver (25Mbps)"
+ bool "Use IDT77105 PHY driver (25Mbps)"
depends on ATM_NICSTAR
help
Support for the PHYsical layer chip in ForeRunner LE25 cards. In
diff --git a/drivers/auxdisplay/ht16k33.c b/drivers/auxdisplay/ht16k33.c
index 9c0bb77..a2fcde5 100644
--- a/drivers/auxdisplay/ht16k33.c
+++ b/drivers/auxdisplay/ht16k33.c
@@ -74,7 +74,7 @@ struct ht16k33_priv {
struct ht16k33_fbdev fbdev;
};
-static struct fb_fix_screeninfo ht16k33_fb_fix = {
+static const struct fb_fix_screeninfo ht16k33_fb_fix = {
.id = DRIVER_NAME,
.type = FB_TYPE_PACKED_PIXELS,
.visual = FB_VISUAL_MONO10,
@@ -85,7 +85,7 @@ static struct fb_fix_screeninfo ht16k33_fb_fix = {
.accel = FB_ACCEL_NONE,
};
-static struct fb_var_screeninfo ht16k33_fb_var = {
+static const struct fb_var_screeninfo ht16k33_fb_var = {
.xres = HT16K33_MATRIX_LED_MAX_ROWS,
.yres = HT16K33_MATRIX_LED_MAX_COLS,
.xres_virtual = HT16K33_MATRIX_LED_MAX_ROWS,
diff --git a/drivers/bcma/driver_mips.c b/drivers/bcma/driver_mips.c
index 27e9686..87760aa 100644
--- a/drivers/bcma/driver_mips.c
+++ b/drivers/bcma/driver_mips.c
@@ -59,22 +59,6 @@ static inline void mips_write32(struct bcma_drv_mips *mcore,
bcma_write32(mcore->core, offset, value);
}
-static const u32 ipsflag_irq_mask[] = {
- 0,
- BCMA_MIPS_IPSFLAG_IRQ1,
- BCMA_MIPS_IPSFLAG_IRQ2,
- BCMA_MIPS_IPSFLAG_IRQ3,
- BCMA_MIPS_IPSFLAG_IRQ4,
-};
-
-static const u32 ipsflag_irq_shift[] = {
- 0,
- BCMA_MIPS_IPSFLAG_IRQ1_SHIFT,
- BCMA_MIPS_IPSFLAG_IRQ2_SHIFT,
- BCMA_MIPS_IPSFLAG_IRQ3_SHIFT,
- BCMA_MIPS_IPSFLAG_IRQ4_SHIFT,
-};
-
static u32 bcma_core_mips_irqflag(struct bcma_device *dev)
{
u32 flag;
diff --git a/drivers/bcma/driver_pci.c b/drivers/bcma/driver_pci.c
index f499a46..12b2cc9 100644
--- a/drivers/bcma/driver_pci.c
+++ b/drivers/bcma/driver_pci.c
@@ -78,7 +78,7 @@ static u16 bcma_pcie_mdio_read(struct bcma_drv_pci *pc, u16 device, u8 address)
v |= (address << BCMA_CORE_PCI_MDIODATA_REGADDR_SHF_OLD);
}
- v = BCMA_CORE_PCI_MDIODATA_START;
+ v |= BCMA_CORE_PCI_MDIODATA_START;
v |= BCMA_CORE_PCI_MDIODATA_READ;
v |= BCMA_CORE_PCI_MDIODATA_TA;
@@ -121,7 +121,7 @@ static void bcma_pcie_mdio_write(struct bcma_drv_pci *pc, u16 device,
v |= (address << BCMA_CORE_PCI_MDIODATA_REGADDR_SHF_OLD);
}
- v = BCMA_CORE_PCI_MDIODATA_START;
+ v |= BCMA_CORE_PCI_MDIODATA_START;
v |= BCMA_CORE_PCI_MDIODATA_WRITE;
v |= BCMA_CORE_PCI_MDIODATA_TA;
v |= data;
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 9bd4ddd..5b24876 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -322,6 +322,8 @@ static int drbd_thread_setup(void *arg)
thi->name[0],
resource->name);
+ allow_kernel_signal(DRBD_SIGKILL);
+ allow_kernel_signal(SIGXCPU);
restart:
retval = thi->function(thi);
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 3327192..c8fb886 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -3038,6 +3038,17 @@ static bool rbd_obj_advance_read(struct rbd_obj_request *obj_req, int *result)
}
return true;
case RBD_OBJ_READ_PARENT:
+ /*
+ * The parent image is read only up to the overlap -- zero-fill
+ * from the overlap to the end of the request.
+ */
+ if (!*result) {
+ u32 obj_overlap = rbd_obj_img_extents_bytes(obj_req);
+
+ if (obj_overlap < obj_req->ex.oe_len)
+ rbd_obj_zero_range(obj_req, obj_overlap,
+ obj_req->ex.oe_len - obj_overlap);
+ }
return true;
default:
BUG();
diff --git a/drivers/bluetooth/btbcm.c b/drivers/bluetooth/btbcm.c
index 124ef0a..2d2e6d8 100644
--- a/drivers/bluetooth/btbcm.c
+++ b/drivers/bluetooth/btbcm.c
@@ -23,6 +23,7 @@
#define BDADDR_BCM43430A0 (&(bdaddr_t) {{0xac, 0x1f, 0x12, 0xa0, 0x43, 0x43}})
#define BDADDR_BCM4324B3 (&(bdaddr_t) {{0x00, 0x00, 0x00, 0xb3, 0x24, 0x43}})
#define BDADDR_BCM4330B1 (&(bdaddr_t) {{0x00, 0x00, 0x00, 0xb1, 0x30, 0x43}})
+#define BDADDR_BCM4345C5 (&(bdaddr_t) {{0xac, 0x1f, 0x00, 0xc5, 0x45, 0x43}})
#define BDADDR_BCM43341B (&(bdaddr_t) {{0xac, 0x1f, 0x00, 0x1b, 0x34, 0x43}})
int btbcm_check_bdaddr(struct hci_dev *hdev)
@@ -73,6 +74,7 @@ int btbcm_check_bdaddr(struct hci_dev *hdev)
!bacmp(&bda->bdaddr, BDADDR_BCM2076B1) ||
!bacmp(&bda->bdaddr, BDADDR_BCM4324B3) ||
!bacmp(&bda->bdaddr, BDADDR_BCM4330B1) ||
+ !bacmp(&bda->bdaddr, BDADDR_BCM4345C5) ||
!bacmp(&bda->bdaddr, BDADDR_BCM43430A0) ||
!bacmp(&bda->bdaddr, BDADDR_BCM43341B)) {
bt_dev_info(hdev, "BCM: Using default device address (%pMR)",
@@ -332,6 +334,7 @@ static const struct bcm_subver_table bcm_uart_subver_table[] = {
{ 0x2122, "BCM4343A0" }, /* 001.001.034 */
{ 0x2209, "BCM43430A1" }, /* 001.002.009 */
{ 0x6119, "BCM4345C0" }, /* 003.001.025 */
+ { 0x6606, "BCM4345C5" }, /* 003.006.006 */
{ 0x230f, "BCM4356A2" }, /* 001.003.015 */
{ 0x220e, "BCM20702A1" }, /* 001.002.014 */
{ 0x4217, "BCM4329B1" }, /* 002.002.023 */
diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
index 0875470..8cc21ad 100644
--- a/drivers/bluetooth/btqca.c
+++ b/drivers/bluetooth/btqca.c
@@ -106,8 +106,9 @@ int qca_send_pre_shutdown_cmd(struct hci_dev *hdev)
bt_dev_dbg(hdev, "QCA pre shutdown cmd");
- skb = __hci_cmd_sync(hdev, QCA_PRE_SHUTDOWN_CMD, 0,
- NULL, HCI_INIT_TIMEOUT);
+ skb = __hci_cmd_sync_ev(hdev, QCA_PRE_SHUTDOWN_CMD, 0,
+ NULL, HCI_EV_CMD_COMPLETE, HCI_INIT_TIMEOUT);
+
if (IS_ERR(skb)) {
err = PTR_ERR(skb);
bt_dev_err(hdev, "QCA preshutdown_cmd failed (%d)", err);
diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index 4f75a9b..bf3c02b 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -178,6 +178,27 @@ static const struct id_table *btrtl_match_ic(u16 lmp_subver, u16 hci_rev,
return &ic_id_table[i];
}
+static struct sk_buff *btrtl_read_local_version(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+
+ skb = __hci_cmd_sync(hdev, HCI_OP_READ_LOCAL_VERSION, 0, NULL,
+ HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb)) {
+ rtl_dev_err(hdev, "HCI_OP_READ_LOCAL_VERSION failed (%ld)",
+ PTR_ERR(skb));
+ return skb;
+ }
+
+ if (skb->len != sizeof(struct hci_rp_read_local_version)) {
+ rtl_dev_err(hdev, "HCI_OP_READ_LOCAL_VERSION event length mismatch");
+ kfree_skb(skb);
+ return ERR_PTR(-EIO);
+ }
+
+ return skb;
+}
+
static int rtl_read_rom_version(struct hci_dev *hdev, u8 *version)
{
struct rtl_rom_version_evt *rom_version;
@@ -186,19 +207,19 @@ static int rtl_read_rom_version(struct hci_dev *hdev, u8 *version)
/* Read RTL ROM version command */
skb = __hci_cmd_sync(hdev, 0xfc6d, 0, NULL, HCI_INIT_TIMEOUT);
if (IS_ERR(skb)) {
- rtl_dev_err(hdev, "Read ROM version failed (%ld)\n",
+ rtl_dev_err(hdev, "Read ROM version failed (%ld)",
PTR_ERR(skb));
return PTR_ERR(skb);
}
if (skb->len != sizeof(*rom_version)) {
- rtl_dev_err(hdev, "RTL version event length mismatch\n");
+ rtl_dev_err(hdev, "version event length mismatch");
kfree_skb(skb);
return -EIO;
}
rom_version = (struct rtl_rom_version_evt *)skb->data;
- rtl_dev_info(hdev, "rom_version status=%x version=%x\n",
+ rtl_dev_info(hdev, "rom_version status=%x version=%x",
rom_version->status, rom_version->version);
*version = rom_version->version;
@@ -242,7 +263,7 @@ static int rtlbt_parse_firmware(struct hci_dev *hdev,
fwptr = btrtl_dev->fw_data + btrtl_dev->fw_len - sizeof(extension_sig);
if (memcmp(fwptr, extension_sig, sizeof(extension_sig)) != 0) {
- rtl_dev_err(hdev, "extension section signature mismatch\n");
+ rtl_dev_err(hdev, "extension section signature mismatch");
return -EINVAL;
}
@@ -263,7 +284,7 @@ static int rtlbt_parse_firmware(struct hci_dev *hdev,
break;
if (length == 0) {
- rtl_dev_err(hdev, "found instruction with length 0\n");
+ rtl_dev_err(hdev, "found instruction with length 0");
return -EINVAL;
}
@@ -276,7 +297,7 @@ static int rtlbt_parse_firmware(struct hci_dev *hdev,
}
if (project_id < 0) {
- rtl_dev_err(hdev, "failed to find version instruction\n");
+ rtl_dev_err(hdev, "failed to find version instruction");
return -EINVAL;
}
@@ -287,13 +308,13 @@ static int rtlbt_parse_firmware(struct hci_dev *hdev,
}
if (i >= ARRAY_SIZE(project_id_to_lmp_subver)) {
- rtl_dev_err(hdev, "unknown project id %d\n", project_id);
+ rtl_dev_err(hdev, "unknown project id %d", project_id);
return -EINVAL;
}
if (btrtl_dev->ic_info->lmp_subver !=
project_id_to_lmp_subver[i].lmp_subver) {
- rtl_dev_err(hdev, "firmware is for %x but this is a %x\n",
+ rtl_dev_err(hdev, "firmware is for %x but this is a %x",
project_id_to_lmp_subver[i].lmp_subver,
btrtl_dev->ic_info->lmp_subver);
return -EINVAL;
@@ -301,7 +322,7 @@ static int rtlbt_parse_firmware(struct hci_dev *hdev,
epatch_info = (struct rtl_epatch_header *)btrtl_dev->fw_data;
if (memcmp(epatch_info->signature, RTL_EPATCH_SIGNATURE, 8) != 0) {
- rtl_dev_err(hdev, "bad EPATCH signature\n");
+ rtl_dev_err(hdev, "bad EPATCH signature");
return -EINVAL;
}
@@ -368,6 +389,8 @@ static int rtl_download_firmware(struct hci_dev *hdev,
int frag_len = RTL_FRAG_LEN;
int ret = 0;
int i;
+ struct sk_buff *skb;
+ struct hci_rp_read_local_version *rp;
dl_cmd = kmalloc(sizeof(struct rtl_download_cmd), GFP_KERNEL);
if (!dl_cmd)
@@ -378,7 +401,11 @@ static int rtl_download_firmware(struct hci_dev *hdev,
BT_DBG("download fw (%d/%d)", i, frag_num);
- dl_cmd->index = i;
+ if (i > 0x7f)
+ dl_cmd->index = (i & 0x7f) + 1;
+ else
+ dl_cmd->index = i;
+
if (i == (frag_num - 1)) {
dl_cmd->index |= 0x80; /* data end */
frag_len = fw_len % RTL_FRAG_LEN;
@@ -389,14 +416,14 @@ static int rtl_download_firmware(struct hci_dev *hdev,
skb = __hci_cmd_sync(hdev, 0xfc20, frag_len + 1, dl_cmd,
HCI_INIT_TIMEOUT);
if (IS_ERR(skb)) {
- rtl_dev_err(hdev, "download fw command failed (%ld)\n",
+ rtl_dev_err(hdev, "download fw command failed (%ld)",
PTR_ERR(skb));
ret = -PTR_ERR(skb);
goto out;
}
if (skb->len != sizeof(struct rtl_download_response)) {
- rtl_dev_err(hdev, "download fw event length mismatch\n");
+ rtl_dev_err(hdev, "download fw event length mismatch");
kfree_skb(skb);
ret = -EIO;
goto out;
@@ -406,6 +433,18 @@ static int rtl_download_firmware(struct hci_dev *hdev,
data += RTL_FRAG_LEN;
}
+ skb = btrtl_read_local_version(hdev);
+ if (IS_ERR(skb)) {
+ ret = PTR_ERR(skb);
+ rtl_dev_err(hdev, "read local version failed");
+ goto out;
+ }
+
+ rp = (struct hci_rp_read_local_version *)skb->data;
+ rtl_dev_info(hdev, "fw version 0x%04x%04x",
+ __le16_to_cpu(rp->hci_rev), __le16_to_cpu(rp->lmp_subver));
+ kfree_skb(skb);
+
out:
kfree(dl_cmd);
return ret;
@@ -416,7 +455,7 @@ static int rtl_load_file(struct hci_dev *hdev, const char *name, u8 **buff)
const struct firmware *fw;
int ret;
- rtl_dev_info(hdev, "rtl: loading %s\n", name);
+ rtl_dev_info(hdev, "loading %s", name);
ret = request_firmware(&fw, name, &hdev->dev);
if (ret < 0)
return ret;
@@ -440,7 +479,7 @@ static int btrtl_setup_rtl8723a(struct hci_dev *hdev,
* (which is only for RTL8723B and newer).
*/
if (!memcmp(btrtl_dev->fw_data, RTL_EPATCH_SIGNATURE, 8)) {
- rtl_dev_err(hdev, "unexpected EPATCH signature!\n");
+ rtl_dev_err(hdev, "unexpected EPATCH signature!");
return -EINVAL;
}
@@ -475,7 +514,7 @@ static int btrtl_setup_rtl8723b(struct hci_dev *hdev,
fw_data = tbuff;
}
- rtl_dev_info(hdev, "cfg_sz %d, total sz %d\n", btrtl_dev->cfg_len, ret);
+ rtl_dev_info(hdev, "cfg_sz %d, total sz %d", btrtl_dev->cfg_len, ret);
ret = rtl_download_firmware(hdev, fw_data, ret);
@@ -484,27 +523,6 @@ static int btrtl_setup_rtl8723b(struct hci_dev *hdev,
return ret;
}
-static struct sk_buff *btrtl_read_local_version(struct hci_dev *hdev)
-{
- struct sk_buff *skb;
-
- skb = __hci_cmd_sync(hdev, HCI_OP_READ_LOCAL_VERSION, 0, NULL,
- HCI_INIT_TIMEOUT);
- if (IS_ERR(skb)) {
- rtl_dev_err(hdev, "HCI_OP_READ_LOCAL_VERSION failed (%ld)\n",
- PTR_ERR(skb));
- return skb;
- }
-
- if (skb->len != sizeof(struct hci_rp_read_local_version)) {
- rtl_dev_err(hdev, "HCI_OP_READ_LOCAL_VERSION event length mismatch\n");
- kfree_skb(skb);
- return ERR_PTR(-EIO);
- }
-
- return skb;
-}
-
void btrtl_free(struct btrtl_device_info *btrtl_dev)
{
kfree(btrtl_dev->fw_data);
@@ -537,7 +555,7 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
}
resp = (struct hci_rp_read_local_version *)skb->data;
- rtl_dev_info(hdev, "rtl: examining hci_ver=%02x hci_rev=%04x lmp_ver=%02x lmp_subver=%04x\n",
+ rtl_dev_info(hdev, "examining hci_ver=%02x hci_rev=%04x lmp_ver=%02x lmp_subver=%04x",
resp->hci_ver, resp->hci_rev,
resp->lmp_ver, resp->lmp_subver);
@@ -550,7 +568,7 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
hdev->bus);
if (!btrtl_dev->ic_info) {
- rtl_dev_info(hdev, "rtl: unknown IC info, lmp subver %04x, hci rev %04x, hci ver %04x",
+ rtl_dev_info(hdev, "unknown IC info, lmp subver %04x, hci rev %04x, hci ver %04x",
lmp_subver, hci_rev, hci_ver);
return btrtl_dev;
}
@@ -564,7 +582,7 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
btrtl_dev->fw_len = rtl_load_file(hdev, btrtl_dev->ic_info->fw_name,
&btrtl_dev->fw_data);
if (btrtl_dev->fw_len < 0) {
- rtl_dev_err(hdev, "firmware file %s not found\n",
+ rtl_dev_err(hdev, "firmware file %s not found",
btrtl_dev->ic_info->fw_name);
ret = btrtl_dev->fw_len;
goto err_free;
@@ -582,7 +600,7 @@ struct btrtl_device_info *btrtl_initialize(struct hci_dev *hdev,
&btrtl_dev->cfg_data);
if (btrtl_dev->ic_info->config_needed &&
btrtl_dev->cfg_len <= 0) {
- rtl_dev_err(hdev, "mandatory config file %s not found\n",
+ rtl_dev_err(hdev, "mandatory config file %s not found",
btrtl_dev->ic_info->cfg_name);
ret = btrtl_dev->cfg_len;
goto err_free;
@@ -608,7 +626,7 @@ int btrtl_download_firmware(struct hci_dev *hdev,
* to a different value.
*/
if (!btrtl_dev->ic_info) {
- rtl_dev_info(hdev, "rtl: assuming no firmware upload needed\n");
+ rtl_dev_info(hdev, "assuming no firmware upload needed");
return 0;
}
@@ -622,7 +640,7 @@ int btrtl_download_firmware(struct hci_dev *hdev,
case RTL_ROM_LMP_8822B:
return btrtl_setup_rtl8723b(hdev, btrtl_dev);
default:
- rtl_dev_info(hdev, "rtl: assuming no firmware upload needed\n");
+ rtl_dev_info(hdev, "assuming no firmware upload needed");
return 0;
}
}
@@ -641,6 +659,11 @@ int btrtl_setup_realtek(struct hci_dev *hdev)
btrtl_free(btrtl_dev);
+ /* Enable controller to do both LE scan and BR/EDR inquiry
+ * simultaneously.
+ */
+ set_bit(HCI_QUIRK_SIMULTANEOUS_DISCOVERY, &hdev->quirks);
+
return ret;
}
EXPORT_SYMBOL_GPL(btrtl_setup_realtek);
@@ -714,18 +737,18 @@ int btrtl_get_uart_settings(struct hci_dev *hdev,
total_data_len = btrtl_dev->cfg_len - sizeof(*config);
if (total_data_len <= 0) {
- rtl_dev_warn(hdev, "no config loaded\n");
+ rtl_dev_warn(hdev, "no config loaded");
return -EINVAL;
}
config = (struct rtl_vendor_config *)btrtl_dev->cfg_data;
if (le32_to_cpu(config->signature) != RTL_CONFIG_MAGIC) {
- rtl_dev_err(hdev, "invalid config magic\n");
+ rtl_dev_err(hdev, "invalid config magic");
return -EINVAL;
}
if (total_data_len < le16_to_cpu(config->total_len)) {
- rtl_dev_err(hdev, "config is too short\n");
+ rtl_dev_err(hdev, "config is too short");
return -EINVAL;
}
@@ -735,7 +758,7 @@ int btrtl_get_uart_settings(struct hci_dev *hdev,
switch (le16_to_cpu(entry->offset)) {
case 0xc:
if (entry->len < sizeof(*device_baudrate)) {
- rtl_dev_err(hdev, "invalid UART config entry\n");
+ rtl_dev_err(hdev, "invalid UART config entry");
return -EINVAL;
}
@@ -752,7 +775,7 @@ int btrtl_get_uart_settings(struct hci_dev *hdev,
break;
default:
- rtl_dev_dbg(hdev, "skipping config entry 0x%x (len %u)\n",
+ rtl_dev_dbg(hdev, "skipping config entry 0x%x (len %u)",
le16_to_cpu(entry->offset), entry->len);
break;
};
@@ -761,13 +784,13 @@ int btrtl_get_uart_settings(struct hci_dev *hdev,
}
if (!found) {
- rtl_dev_err(hdev, "no UART config entry found\n");
+ rtl_dev_err(hdev, "no UART config entry found");
return -ENOENT;
}
- rtl_dev_dbg(hdev, "device baudrate = 0x%08x\n", *device_baudrate);
- rtl_dev_dbg(hdev, "controller baudrate = %u\n", *controller_baudrate);
- rtl_dev_dbg(hdev, "flow control %d\n", *flow_control);
+ rtl_dev_dbg(hdev, "device baudrate = 0x%08x", *device_baudrate);
+ rtl_dev_dbg(hdev, "controller baudrate = %u", *controller_baudrate);
+ rtl_dev_dbg(hdev, "flow control %d", *flow_control);
return 0;
}
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index 5cf0734..ed455de 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -435,6 +435,7 @@ static const struct dmi_system_id btusb_needs_reset_resume_table[] = {
#define BTUSB_OOB_WAKE_ENABLED 11
#define BTUSB_HW_RESET_ACTIVE 12
#define BTUSB_TX_WAIT_VND_EVT 13
+#define BTUSB_WAKEUP_DISABLE 14
struct btusb_data {
struct hci_dev *hdev;
@@ -523,6 +524,36 @@ static void btusb_intel_cmd_timeout(struct hci_dev *hdev)
gpiod_set_value_cansleep(reset_gpio, 0);
}
+static void btusb_rtl_cmd_timeout(struct hci_dev *hdev)
+{
+ struct btusb_data *data = hci_get_drvdata(hdev);
+ struct gpio_desc *reset_gpio = data->reset_gpio;
+
+ if (++data->cmd_timeout_cnt < 5)
+ return;
+
+ if (!reset_gpio) {
+ bt_dev_err(hdev, "No gpio to reset Realtek device, ignoring");
+ return;
+ }
+
+ /* Toggle the hard reset line. The Realtek device is going to
+ * yank itself off the USB and then replug. The cleanup is handled
+ * correctly on the way out (standard USB disconnect), and the new
+ * device is detected cleanly and bound to the driver again like
+ * it should be.
+ */
+ if (test_and_set_bit(BTUSB_HW_RESET_ACTIVE, &data->flags)) {
+ bt_dev_err(hdev, "last reset failed? Not resetting again");
+ return;
+ }
+
+ bt_dev_err(hdev, "Reset Realtek device via gpio");
+ gpiod_set_value_cansleep(reset_gpio, 0);
+ msleep(200);
+ gpiod_set_value_cansleep(reset_gpio, 1);
+}
+
static inline void btusb_free_frags(struct btusb_data *data)
{
unsigned long flags;
@@ -1175,6 +1206,13 @@ static int btusb_open(struct hci_dev *hdev)
*/
device_wakeup_enable(&data->udev->dev);
+ /* Disable device remote wakeup when host is suspended
+ * For Realtek chips, global suspend without
+ * SET_FEATURE (DEVICE_REMOTE_WAKEUP) can save more power in device.
+ */
+ if (test_bit(BTUSB_WAKEUP_DISABLE, &data->flags))
+ device_wakeup_disable(&data->udev->dev);
+
if (test_and_set_bit(BTUSB_INTR_RUNNING, &data->flags))
goto done;
@@ -1238,6 +1276,11 @@ static int btusb_close(struct hci_dev *hdev)
goto failed;
data->intf->needs_remote_wakeup = 0;
+
+ /* Enable remote wake up for auto-suspend */
+ if (test_bit(BTUSB_WAKEUP_DISABLE, &data->flags))
+ data->intf->needs_remote_wakeup = 1;
+
device_wakeup_disable(&data->udev->dev);
usb_autopm_put_interface(data->intf);
@@ -3770,12 +3813,13 @@ static int btusb_probe(struct usb_interface *intf,
if (id->driver_info & BTUSB_REALTEK) {
hdev->setup = btrtl_setup_realtek;
hdev->shutdown = btrtl_shutdown_realtek;
+ hdev->cmd_timeout = btusb_rtl_cmd_timeout;
- /* Realtek devices lose their updated firmware over suspend,
- * but the USB hub doesn't notice any status change.
- * Explicitly request a device reset on resume.
+ /* Realtek devices lose their updated firmware over global
+ * suspend that means host doesn't send SET_FEATURE
+ * (DEVICE_REMOTE_WAKEUP)
*/
- interface_to_usbdev(intf)->quirks |= USB_QUIRK_RESET_RESUME;
+ set_bit(BTUSB_WAKEUP_DISABLE, &data->flags);
}
#endif
@@ -3949,6 +3993,19 @@ static int btusb_suspend(struct usb_interface *intf, pm_message_t message)
enable_irq(data->oob_wake_irq);
}
+ /* For global suspend, Realtek devices lose the loaded fw
+ * in them. But for autosuspend, firmware should remain.
+ * Actually, it depends on whether the usb host sends
+ * set feature (enable wakeup) or not.
+ */
+ if (test_bit(BTUSB_WAKEUP_DISABLE, &data->flags)) {
+ if (PMSG_IS_AUTO(message) &&
+ device_can_wakeup(&data->udev->dev))
+ data->udev->do_remote_wakeup = 1;
+ else if (!PMSG_IS_AUTO(message))
+ data->udev->reset_resume = 1;
+ }
+
return 0;
}
diff --git a/drivers/bluetooth/hci_bcm.c b/drivers/bluetooth/hci_bcm.c
index ae2624f..7646636 100644
--- a/drivers/bluetooth/hci_bcm.c
+++ b/drivers/bluetooth/hci_bcm.c
@@ -260,7 +260,7 @@ static int bcm_gpio_set_power(struct bcm_device *dev, bool powered)
}
/* wait for device to power on and come out of reset */
- usleep_range(10000, 20000);
+ usleep_range(100000, 120000);
dev->res_enabled = powered;
@@ -824,6 +824,21 @@ static int bcm_resume(struct device *dev)
}
#endif
+/* Some firmware reports an IRQ which does not work (wrong pin in fw table?) */
+static const struct dmi_system_id bcm_broken_irq_dmi_table[] = {
+ {
+ .ident = "Meegopad T08",
+ .matches = {
+ DMI_EXACT_MATCH(DMI_BOARD_VENDOR,
+ "To be filled by OEM."),
+ DMI_EXACT_MATCH(DMI_BOARD_NAME, "T3 MRD"),
+ DMI_EXACT_MATCH(DMI_BOARD_VERSION, "V1.1"),
+ },
+ },
+ { }
+};
+
+#ifdef CONFIG_ACPI
static const struct acpi_gpio_params first_gpio = { 0, 0, false };
static const struct acpi_gpio_params second_gpio = { 1, 0, false };
static const struct acpi_gpio_params third_gpio = { 2, 0, false };
@@ -842,21 +857,6 @@ static const struct acpi_gpio_mapping acpi_bcm_int_first_gpios[] = {
{ },
};
-/* Some firmware reports an IRQ which does not work (wrong pin in fw table?) */
-static const struct dmi_system_id bcm_broken_irq_dmi_table[] = {
- {
- .ident = "Meegopad T08",
- .matches = {
- DMI_EXACT_MATCH(DMI_BOARD_VENDOR,
- "To be filled by OEM."),
- DMI_EXACT_MATCH(DMI_BOARD_NAME, "T3 MRD"),
- DMI_EXACT_MATCH(DMI_BOARD_VERSION, "V1.1"),
- },
- },
- { }
-};
-
-#ifdef CONFIG_ACPI
static int bcm_resource(struct acpi_resource *ares, void *data)
{
struct bcm_device *dev = data;
@@ -1419,6 +1419,7 @@ static void bcm_serdev_remove(struct serdev_device *serdev)
#ifdef CONFIG_OF
static const struct of_device_id bcm_bluetooth_of_match[] = {
{ .compatible = "brcm,bcm20702a1" },
+ { .compatible = "brcm,bcm4345c5" },
{ .compatible = "brcm,bcm4330-bt" },
{ .compatible = "brcm,bcm43438-bt" },
{ },
diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c
index 9a970fd..d33828f 100644
--- a/drivers/bluetooth/hci_qca.c
+++ b/drivers/bluetooth/hci_qca.c
@@ -502,26 +502,7 @@ static int qca_open(struct hci_uart *hu)
qca->tx_ibs_state = HCI_IBS_TX_ASLEEP;
qca->rx_ibs_state = HCI_IBS_RX_ASLEEP;
- /* clocks actually on, but we start votes off */
- qca->tx_vote = false;
- qca->rx_vote = false;
- qca->flags = 0;
-
- qca->ibs_sent_wacks = 0;
- qca->ibs_sent_slps = 0;
- qca->ibs_sent_wakes = 0;
- qca->ibs_recv_wacks = 0;
- qca->ibs_recv_slps = 0;
- qca->ibs_recv_wakes = 0;
qca->vote_last_jif = jiffies;
- qca->vote_on_ms = 0;
- qca->vote_off_ms = 0;
- qca->votes_on = 0;
- qca->votes_off = 0;
- qca->tx_votes_on = 0;
- qca->tx_votes_off = 0;
- qca->rx_votes_on = 0;
- qca->rx_votes_off = 0;
hu->priv = qca;
@@ -1261,6 +1242,11 @@ static int qca_setup(struct hci_uart *hu)
/* Patch downloading has to be done without IBS mode */
clear_bit(QCA_IBS_ENABLED, &qca->flags);
+ /* Enable controller to do both LE scan and BR/EDR inquiry
+ * simultaneously.
+ */
+ set_bit(HCI_QUIRK_SIMULTANEOUS_DISCOVERY, &hdev->quirks);
+
if (qca_is_wcn399x(soc_type)) {
bt_dev_info(hdev, "setting up wcn3990");
@@ -1326,7 +1312,7 @@ static int qca_setup(struct hci_uart *hu)
return ret;
}
-static struct hci_uart_proto qca_proto = {
+static const struct hci_uart_proto qca_proto = {
.id = HCI_UART_QCA,
.name = "QCA",
.manufacturer = 29,
@@ -1389,6 +1375,8 @@ static int qca_power_off(struct hci_dev *hdev)
/* Perform pre shutdown command */
qca_send_pre_shutdown_cmd(hdev);
+ usleep_range(8000, 10000);
+
qca_power_shutdown(hu);
return 0;
}
diff --git a/drivers/bus/hisi_lpc.c b/drivers/bus/hisi_lpc.c
index 19d7b6f..20c9571 100644
--- a/drivers/bus/hisi_lpc.c
+++ b/drivers/bus/hisi_lpc.c
@@ -456,6 +456,17 @@ struct hisi_lpc_acpi_cell {
size_t pdata_size;
};
+static void hisi_lpc_acpi_remove(struct device *hostdev)
+{
+ struct acpi_device *adev = ACPI_COMPANION(hostdev);
+ struct acpi_device *child;
+
+ device_for_each_child(hostdev, NULL, hisi_lpc_acpi_remove_subdev);
+
+ list_for_each_entry(child, &adev->children, node)
+ acpi_device_clear_enumerated(child);
+}
+
/*
* hisi_lpc_acpi_probe - probe children for ACPI FW
* @hostdev: LPC host device pointer
@@ -555,8 +566,7 @@ static int hisi_lpc_acpi_probe(struct device *hostdev)
return 0;
fail:
- device_for_each_child(hostdev, NULL,
- hisi_lpc_acpi_remove_subdev);
+ hisi_lpc_acpi_remove(hostdev);
return ret;
}
@@ -569,6 +579,10 @@ static int hisi_lpc_acpi_probe(struct device *dev)
{
return -ENODEV;
}
+
+static void hisi_lpc_acpi_remove(struct device *hostdev)
+{
+}
#endif // CONFIG_ACPI
/*
@@ -606,24 +620,27 @@ static int hisi_lpc_probe(struct platform_device *pdev)
range->fwnode = dev->fwnode;
range->flags = LOGIC_PIO_INDIRECT;
range->size = PIO_INDIRECT_SIZE;
+ range->hostdata = lpcdev;
+ range->ops = &hisi_lpc_ops;
+ lpcdev->io_host = range;
ret = logic_pio_register_range(range);
if (ret) {
dev_err(dev, "register IO range failed (%d)!\n", ret);
return ret;
}
- lpcdev->io_host = range;
/* register the LPC host PIO resources */
if (acpi_device)
ret = hisi_lpc_acpi_probe(dev);
else
ret = of_platform_populate(dev->of_node, NULL, NULL, dev);
- if (ret)
+ if (ret) {
+ logic_pio_unregister_range(range);
return ret;
+ }
- lpcdev->io_host->hostdata = lpcdev;
- lpcdev->io_host->ops = &hisi_lpc_ops;
+ dev_set_drvdata(dev, lpcdev);
io_end = lpcdev->io_host->io_start + lpcdev->io_host->size;
dev_info(dev, "registered range [%pa - %pa]\n",
@@ -632,6 +649,23 @@ static int hisi_lpc_probe(struct platform_device *pdev)
return ret;
}
+static int hisi_lpc_remove(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct acpi_device *acpi_device = ACPI_COMPANION(dev);
+ struct hisi_lpc_dev *lpcdev = dev_get_drvdata(dev);
+ struct logic_pio_hwaddr *range = lpcdev->io_host;
+
+ if (acpi_device)
+ hisi_lpc_acpi_remove(dev);
+ else
+ of_platform_depopulate(dev);
+
+ logic_pio_unregister_range(range);
+
+ return 0;
+}
+
static const struct of_device_id hisi_lpc_of_match[] = {
{ .compatible = "hisilicon,hip06-lpc", },
{ .compatible = "hisilicon,hip07-lpc", },
@@ -645,5 +679,6 @@ static struct platform_driver hisi_lpc_driver = {
.acpi_match_table = ACPI_PTR(hisi_lpc_acpi_match),
},
.probe = hisi_lpc_probe,
+ .remove = hisi_lpc_remove,
};
builtin_platform_driver(hisi_lpc_driver);
diff --git a/drivers/bus/ti-sysc.c b/drivers/bus/ti-sysc.c
index e6deabd..2db474a 100644
--- a/drivers/bus/ti-sysc.c
+++ b/drivers/bus/ti-sysc.c
@@ -949,7 +949,7 @@ static int sysc_best_idle_mode(u32 idlemodes, u32 *best_mode)
*best_mode = SYSC_IDLE_SMART_WKUP;
else if (idlemodes & BIT(SYSC_IDLE_SMART))
*best_mode = SYSC_IDLE_SMART;
- else if (idlemodes & SYSC_IDLE_FORCE)
+ else if (idlemodes & BIT(SYSC_IDLE_FORCE))
*best_mode = SYSC_IDLE_FORCE;
else
return -EINVAL;
@@ -1267,7 +1267,8 @@ static const struct sysc_revision_quirk sysc_revision_quirks[] = {
SYSC_QUIRK("control", 0, 0, 0x10, -1, 0x40000900, 0xffffffff, 0),
SYSC_QUIRK("cpgmac", 0, 0x1200, 0x1208, 0x1204, 0x4edb1902,
0xffff00f0, 0),
- SYSC_QUIRK("dcan", 0, 0, -1, -1, 0xffffffff, 0xffffffff, 0),
+ SYSC_QUIRK("dcan", 0, 0x20, -1, -1, 0xa3170504, 0xffffffff, 0),
+ SYSC_QUIRK("dcan", 0, 0x20, -1, -1, 0x4edb1902, 0xffffffff, 0),
SYSC_QUIRK("dmic", 0, 0, 0x10, -1, 0x50010000, 0xffffffff, 0),
SYSC_QUIRK("dwc3", 0, 0, 0x10, -1, 0x500a0200, 0xffffffff, 0),
SYSC_QUIRK("epwmss", 0, 0, 0x4, -1, 0x47400001, 0xffffffff, 0),
@@ -1692,10 +1693,7 @@ static int sysc_init_sysc_mask(struct sysc *ddata)
if (error)
return 0;
- if (val)
- ddata->cfg.sysc_val = val & ddata->cap->sysc_mask;
- else
- ddata->cfg.sysc_val = ddata->cap->sysc_mask;
+ ddata->cfg.sysc_val = val & ddata->cap->sysc_mask;
return 0;
}
@@ -2385,27 +2383,27 @@ static int sysc_probe(struct platform_device *pdev)
error = sysc_init_dts_quirks(ddata);
if (error)
- goto unprepare;
+ return error;
error = sysc_map_and_check_registers(ddata);
if (error)
- goto unprepare;
+ return error;
error = sysc_init_sysc_mask(ddata);
if (error)
- goto unprepare;
+ return error;
error = sysc_init_idlemodes(ddata);
if (error)
- goto unprepare;
+ return error;
error = sysc_init_syss_mask(ddata);
if (error)
- goto unprepare;
+ return error;
error = sysc_init_pdata(ddata);
if (error)
- goto unprepare;
+ return error;
sysc_init_early_quirks(ddata);
@@ -2415,7 +2413,7 @@ static int sysc_probe(struct platform_device *pdev)
error = sysc_init_resets(ddata);
if (error)
- return error;
+ goto unprepare;
error = sysc_init_module(ddata);
if (error)
diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index c099070..1c46bab 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -324,6 +324,25 @@ static struct clk_core *clk_core_lookup(const char *name)
return NULL;
}
+#ifdef CONFIG_OF
+static int of_parse_clkspec(const struct device_node *np, int index,
+ const char *name, struct of_phandle_args *out_args);
+static struct clk_hw *
+of_clk_get_hw_from_clkspec(struct of_phandle_args *clkspec);
+#else
+static inline int of_parse_clkspec(const struct device_node *np, int index,
+ const char *name,
+ struct of_phandle_args *out_args)
+{
+ return -ENOENT;
+}
+static inline struct clk_hw *
+of_clk_get_hw_from_clkspec(struct of_phandle_args *clkspec)
+{
+ return ERR_PTR(-ENOENT);
+}
+#endif
+
/**
* clk_core_get - Find the clk_core parent of a clk
* @core: clk to find parent of
@@ -355,8 +374,9 @@ static struct clk_core *clk_core_lookup(const char *name)
* };
*
* Returns: -ENOENT when the provider can't be found or the clk doesn't
- * exist in the provider. -EINVAL when the name can't be found. NULL when the
- * provider knows about the clk but it isn't provided on this system.
+ * exist in the provider or the name can't be found in the DT node or
+ * in a clkdev lookup. NULL when the provider knows about the clk but it
+ * isn't provided on this system.
* A valid clk_core pointer when the clk can be found in the provider.
*/
static struct clk_core *clk_core_get(struct clk_core *core, u8 p_index)
@@ -367,17 +387,19 @@ static struct clk_core *clk_core_get(struct clk_core *core, u8 p_index)
struct device *dev = core->dev;
const char *dev_id = dev ? dev_name(dev) : NULL;
struct device_node *np = core->of_node;
+ struct of_phandle_args clkspec;
- if (np && (name || index >= 0))
- hw = of_clk_get_hw(np, index, name);
-
- /*
- * If the DT search above couldn't find the provider or the provider
- * didn't know about this clk, fallback to looking up via clkdev based
- * clk_lookups
- */
- if (PTR_ERR(hw) == -ENOENT && name)
+ if (np && (name || index >= 0) &&
+ !of_parse_clkspec(np, index, name, &clkspec)) {
+ hw = of_clk_get_hw_from_clkspec(&clkspec);
+ of_node_put(clkspec.np);
+ } else if (name) {
+ /*
+ * If the DT search above couldn't find the provider fallback to
+ * looking up via clkdev based clk_lookups.
+ */
hw = clk_find_hw(dev_id, name);
+ }
if (IS_ERR(hw))
return ERR_CAST(hw);
@@ -401,7 +423,7 @@ static void clk_core_fill_parent_index(struct clk_core *core, u8 index)
parent = ERR_PTR(-EPROBE_DEFER);
} else {
parent = clk_core_get(core, index);
- if (IS_ERR(parent) && PTR_ERR(parent) == -ENOENT)
+ if (IS_ERR(parent) && PTR_ERR(parent) == -ENOENT && entry->name)
parent = clk_core_lookup(entry->name);
}
@@ -1632,7 +1654,8 @@ static int clk_fetch_parent_index(struct clk_core *core,
break;
/* Fallback to comparing globally unique names */
- if (!strcmp(parent->name, core->parents[i].name))
+ if (core->parents[i].name &&
+ !strcmp(parent->name, core->parents[i].name))
break;
}
diff --git a/drivers/clk/samsung/clk-exynos5-subcmu.c b/drivers/clk/samsung/clk-exynos5-subcmu.c
index 91db789..65c82d9 100644
--- a/drivers/clk/samsung/clk-exynos5-subcmu.c
+++ b/drivers/clk/samsung/clk-exynos5-subcmu.c
@@ -14,7 +14,7 @@
#include "clk-exynos5-subcmu.h"
static struct samsung_clk_provider *ctx;
-static const struct exynos5_subcmu_info *cmu;
+static const struct exynos5_subcmu_info **cmu;
static int nr_cmus;
static void exynos5_subcmu_clk_save(void __iomem *base,
@@ -56,17 +56,17 @@ static void exynos5_subcmu_defer_gate(struct samsung_clk_provider *ctx,
* when OF-core populates all device-tree nodes.
*/
void exynos5_subcmus_init(struct samsung_clk_provider *_ctx, int _nr_cmus,
- const struct exynos5_subcmu_info *_cmu)
+ const struct exynos5_subcmu_info **_cmu)
{
ctx = _ctx;
cmu = _cmu;
nr_cmus = _nr_cmus;
for (; _nr_cmus--; _cmu++) {
- exynos5_subcmu_defer_gate(ctx, _cmu->gate_clks,
- _cmu->nr_gate_clks);
- exynos5_subcmu_clk_save(ctx->reg_base, _cmu->suspend_regs,
- _cmu->nr_suspend_regs);
+ exynos5_subcmu_defer_gate(ctx, (*_cmu)->gate_clks,
+ (*_cmu)->nr_gate_clks);
+ exynos5_subcmu_clk_save(ctx->reg_base, (*_cmu)->suspend_regs,
+ (*_cmu)->nr_suspend_regs);
}
}
@@ -163,9 +163,9 @@ static int __init exynos5_clk_probe(struct platform_device *pdev)
if (of_property_read_string(np, "label", &name) < 0)
continue;
for (i = 0; i < nr_cmus; i++)
- if (strcmp(cmu[i].pd_name, name) == 0)
+ if (strcmp(cmu[i]->pd_name, name) == 0)
exynos5_clk_register_subcmu(&pdev->dev,
- &cmu[i], np);
+ cmu[i], np);
}
return 0;
}
diff --git a/drivers/clk/samsung/clk-exynos5-subcmu.h b/drivers/clk/samsung/clk-exynos5-subcmu.h
index 755ee8a..9ae5356 100644
--- a/drivers/clk/samsung/clk-exynos5-subcmu.h
+++ b/drivers/clk/samsung/clk-exynos5-subcmu.h
@@ -21,6 +21,6 @@ struct exynos5_subcmu_info {
};
void exynos5_subcmus_init(struct samsung_clk_provider *ctx, int nr_cmus,
- const struct exynos5_subcmu_info *cmu);
+ const struct exynos5_subcmu_info **cmu);
#endif
diff --git a/drivers/clk/samsung/clk-exynos5250.c b/drivers/clk/samsung/clk-exynos5250.c
index f2b8968..931c70a 100644
--- a/drivers/clk/samsung/clk-exynos5250.c
+++ b/drivers/clk/samsung/clk-exynos5250.c
@@ -681,6 +681,10 @@ static const struct exynos5_subcmu_info exynos5250_disp_subcmu = {
.pd_name = "DISP1",
};
+static const struct exynos5_subcmu_info *exynos5250_subcmus[] = {
+ &exynos5250_disp_subcmu,
+};
+
static const struct samsung_pll_rate_table vpll_24mhz_tbl[] __initconst = {
/* sorted in descending order */
/* PLL_36XX_RATE(rate, m, p, s, k) */
@@ -843,7 +847,8 @@ static void __init exynos5250_clk_init(struct device_node *np)
samsung_clk_sleep_init(reg_base, exynos5250_clk_regs,
ARRAY_SIZE(exynos5250_clk_regs));
- exynos5_subcmus_init(ctx, 1, &exynos5250_disp_subcmu);
+ exynos5_subcmus_init(ctx, ARRAY_SIZE(exynos5250_subcmus),
+ exynos5250_subcmus);
samsung_clk_of_add_provider(np, ctx);
diff --git a/drivers/clk/samsung/clk-exynos5420.c b/drivers/clk/samsung/clk-exynos5420.c
index 01bca5a..7670cc5 100644
--- a/drivers/clk/samsung/clk-exynos5420.c
+++ b/drivers/clk/samsung/clk-exynos5420.c
@@ -534,8 +534,6 @@ static const struct samsung_gate_clock exynos5800_gate_clks[] __initconst = {
GATE_BUS_TOP, 24, 0, 0),
GATE(CLK_ACLK432_SCALER, "aclk432_scaler", "mout_user_aclk432_scaler",
GATE_BUS_TOP, 27, CLK_IS_CRITICAL, 0),
- GATE(CLK_MAU_EPLL, "mau_epll", "mout_user_mau_epll",
- SRC_MASK_TOP7, 20, CLK_SET_RATE_PARENT, 0),
};
static const struct samsung_mux_clock exynos5420_mux_clks[] __initconst = {
@@ -577,8 +575,13 @@ static const struct samsung_div_clock exynos5420_div_clks[] __initconst = {
static const struct samsung_gate_clock exynos5420_gate_clks[] __initconst = {
GATE(CLK_SECKEY, "seckey", "aclk66_psgen", GATE_BUS_PERIS1, 1, 0, 0),
+ /* Maudio Block */
GATE(CLK_MAU_EPLL, "mau_epll", "mout_mau_epll_clk",
SRC_MASK_TOP7, 20, CLK_SET_RATE_PARENT, 0),
+ GATE(CLK_SCLK_MAUDIO0, "sclk_maudio0", "dout_maudio0",
+ GATE_TOP_SCLK_MAU, 0, CLK_SET_RATE_PARENT, 0),
+ GATE(CLK_SCLK_MAUPCM0, "sclk_maupcm0", "dout_maupcm0",
+ GATE_TOP_SCLK_MAU, 1, CLK_SET_RATE_PARENT, 0),
};
static const struct samsung_mux_clock exynos5x_mux_clks[] __initconst = {
@@ -890,9 +893,6 @@ static const struct samsung_div_clock exynos5x_div_clks[] __initconst = {
/* GSCL Block */
DIV(0, "dout_gscl_blk_333", "aclk333_432_gscl", DIV2_RATIO0, 6, 2),
- /* MSCL Block */
- DIV(0, "dout_mscl_blk", "aclk400_mscl", DIV2_RATIO0, 28, 2),
-
/* PSGEN */
DIV(0, "dout_gen_blk", "mout_user_aclk266", DIV2_RATIO0, 8, 1),
DIV(0, "dout_jpg_blk", "aclk166", DIV2_RATIO0, 20, 1),
@@ -1017,12 +1017,6 @@ static const struct samsung_gate_clock exynos5x_gate_clks[] __initconst = {
GATE(CLK_SCLK_DP1, "sclk_dp1", "dout_dp1",
GATE_TOP_SCLK_DISP1, 20, CLK_SET_RATE_PARENT, 0),
- /* Maudio Block */
- GATE(CLK_SCLK_MAUDIO0, "sclk_maudio0", "dout_maudio0",
- GATE_TOP_SCLK_MAU, 0, CLK_SET_RATE_PARENT, 0),
- GATE(CLK_SCLK_MAUPCM0, "sclk_maupcm0", "dout_maupcm0",
- GATE_TOP_SCLK_MAU, 1, CLK_SET_RATE_PARENT, 0),
-
/* FSYS Block */
GATE(CLK_TSI, "tsi", "aclk200_fsys", GATE_BUS_FSYS0, 0, 0, 0),
GATE(CLK_PDMA0, "pdma0", "aclk200_fsys", GATE_BUS_FSYS0, 1, 0, 0),
@@ -1162,17 +1156,6 @@ static const struct samsung_gate_clock exynos5x_gate_clks[] __initconst = {
GATE(CLK_FIMC_LITE3, "fimc_lite3", "aclk333_432_gscl",
GATE_IP_GSCL1, 17, 0, 0),
- /* MSCL Block */
- GATE(CLK_MSCL0, "mscl0", "aclk400_mscl", GATE_IP_MSCL, 0, 0, 0),
- GATE(CLK_MSCL1, "mscl1", "aclk400_mscl", GATE_IP_MSCL, 1, 0, 0),
- GATE(CLK_MSCL2, "mscl2", "aclk400_mscl", GATE_IP_MSCL, 2, 0, 0),
- GATE(CLK_SMMU_MSCL0, "smmu_mscl0", "dout_mscl_blk",
- GATE_IP_MSCL, 8, 0, 0),
- GATE(CLK_SMMU_MSCL1, "smmu_mscl1", "dout_mscl_blk",
- GATE_IP_MSCL, 9, 0, 0),
- GATE(CLK_SMMU_MSCL2, "smmu_mscl2", "dout_mscl_blk",
- GATE_IP_MSCL, 10, 0, 0),
-
/* ISP */
GATE(CLK_SCLK_UART_ISP, "sclk_uart_isp", "dout_uart_isp",
GATE_TOP_SCLK_ISP, 0, CLK_SET_RATE_PARENT, 0),
@@ -1281,32 +1264,103 @@ static struct exynos5_subcmu_reg_dump exynos5x_mfc_suspend_regs[] = {
{ DIV4_RATIO, 0, 0x3 }, /* DIV dout_mfc_blk */
};
-static const struct exynos5_subcmu_info exynos5x_subcmus[] = {
- {
- .div_clks = exynos5x_disp_div_clks,
- .nr_div_clks = ARRAY_SIZE(exynos5x_disp_div_clks),
- .gate_clks = exynos5x_disp_gate_clks,
- .nr_gate_clks = ARRAY_SIZE(exynos5x_disp_gate_clks),
- .suspend_regs = exynos5x_disp_suspend_regs,
- .nr_suspend_regs = ARRAY_SIZE(exynos5x_disp_suspend_regs),
- .pd_name = "DISP",
- }, {
- .div_clks = exynos5x_gsc_div_clks,
- .nr_div_clks = ARRAY_SIZE(exynos5x_gsc_div_clks),
- .gate_clks = exynos5x_gsc_gate_clks,
- .nr_gate_clks = ARRAY_SIZE(exynos5x_gsc_gate_clks),
- .suspend_regs = exynos5x_gsc_suspend_regs,
- .nr_suspend_regs = ARRAY_SIZE(exynos5x_gsc_suspend_regs),
- .pd_name = "GSC",
- }, {
- .div_clks = exynos5x_mfc_div_clks,
- .nr_div_clks = ARRAY_SIZE(exynos5x_mfc_div_clks),
- .gate_clks = exynos5x_mfc_gate_clks,
- .nr_gate_clks = ARRAY_SIZE(exynos5x_mfc_gate_clks),
- .suspend_regs = exynos5x_mfc_suspend_regs,
- .nr_suspend_regs = ARRAY_SIZE(exynos5x_mfc_suspend_regs),
- .pd_name = "MFC",
- },
+static const struct samsung_gate_clock exynos5x_mscl_gate_clks[] __initconst = {
+ /* MSCL Block */
+ GATE(CLK_MSCL0, "mscl0", "aclk400_mscl", GATE_IP_MSCL, 0, 0, 0),
+ GATE(CLK_MSCL1, "mscl1", "aclk400_mscl", GATE_IP_MSCL, 1, 0, 0),
+ GATE(CLK_MSCL2, "mscl2", "aclk400_mscl", GATE_IP_MSCL, 2, 0, 0),
+ GATE(CLK_SMMU_MSCL0, "smmu_mscl0", "dout_mscl_blk",
+ GATE_IP_MSCL, 8, 0, 0),
+ GATE(CLK_SMMU_MSCL1, "smmu_mscl1", "dout_mscl_blk",
+ GATE_IP_MSCL, 9, 0, 0),
+ GATE(CLK_SMMU_MSCL2, "smmu_mscl2", "dout_mscl_blk",
+ GATE_IP_MSCL, 10, 0, 0),
+};
+
+static const struct samsung_div_clock exynos5x_mscl_div_clks[] __initconst = {
+ DIV(0, "dout_mscl_blk", "aclk400_mscl", DIV2_RATIO0, 28, 2),
+};
+
+static struct exynos5_subcmu_reg_dump exynos5x_mscl_suspend_regs[] = {
+ { GATE_IP_MSCL, 0xffffffff, 0xffffffff }, /* MSCL gates */
+ { SRC_TOP3, 0, BIT(4) }, /* MUX mout_user_aclk400_mscl */
+ { DIV2_RATIO0, 0, 0x30000000 }, /* DIV dout_mscl_blk */
+};
+
+static const struct samsung_gate_clock exynos5800_mau_gate_clks[] __initconst = {
+ GATE(CLK_MAU_EPLL, "mau_epll", "mout_user_mau_epll",
+ SRC_MASK_TOP7, 20, CLK_SET_RATE_PARENT, 0),
+ GATE(CLK_SCLK_MAUDIO0, "sclk_maudio0", "dout_maudio0",
+ GATE_TOP_SCLK_MAU, 0, CLK_SET_RATE_PARENT, 0),
+ GATE(CLK_SCLK_MAUPCM0, "sclk_maupcm0", "dout_maupcm0",
+ GATE_TOP_SCLK_MAU, 1, CLK_SET_RATE_PARENT, 0),
+};
+
+static struct exynos5_subcmu_reg_dump exynos5800_mau_suspend_regs[] = {
+ { SRC_TOP9, 0, BIT(8) }, /* MUX mout_user_mau_epll */
+};
+
+static const struct exynos5_subcmu_info exynos5x_disp_subcmu = {
+ .div_clks = exynos5x_disp_div_clks,
+ .nr_div_clks = ARRAY_SIZE(exynos5x_disp_div_clks),
+ .gate_clks = exynos5x_disp_gate_clks,
+ .nr_gate_clks = ARRAY_SIZE(exynos5x_disp_gate_clks),
+ .suspend_regs = exynos5x_disp_suspend_regs,
+ .nr_suspend_regs = ARRAY_SIZE(exynos5x_disp_suspend_regs),
+ .pd_name = "DISP",
+};
+
+static const struct exynos5_subcmu_info exynos5x_gsc_subcmu = {
+ .div_clks = exynos5x_gsc_div_clks,
+ .nr_div_clks = ARRAY_SIZE(exynos5x_gsc_div_clks),
+ .gate_clks = exynos5x_gsc_gate_clks,
+ .nr_gate_clks = ARRAY_SIZE(exynos5x_gsc_gate_clks),
+ .suspend_regs = exynos5x_gsc_suspend_regs,
+ .nr_suspend_regs = ARRAY_SIZE(exynos5x_gsc_suspend_regs),
+ .pd_name = "GSC",
+};
+
+static const struct exynos5_subcmu_info exynos5x_mfc_subcmu = {
+ .div_clks = exynos5x_mfc_div_clks,
+ .nr_div_clks = ARRAY_SIZE(exynos5x_mfc_div_clks),
+ .gate_clks = exynos5x_mfc_gate_clks,
+ .nr_gate_clks = ARRAY_SIZE(exynos5x_mfc_gate_clks),
+ .suspend_regs = exynos5x_mfc_suspend_regs,
+ .nr_suspend_regs = ARRAY_SIZE(exynos5x_mfc_suspend_regs),
+ .pd_name = "MFC",
+};
+
+static const struct exynos5_subcmu_info exynos5x_mscl_subcmu = {
+ .div_clks = exynos5x_mscl_div_clks,
+ .nr_div_clks = ARRAY_SIZE(exynos5x_mscl_div_clks),
+ .gate_clks = exynos5x_mscl_gate_clks,
+ .nr_gate_clks = ARRAY_SIZE(exynos5x_mscl_gate_clks),
+ .suspend_regs = exynos5x_mscl_suspend_regs,
+ .nr_suspend_regs = ARRAY_SIZE(exynos5x_mscl_suspend_regs),
+ .pd_name = "MSC",
+};
+
+static const struct exynos5_subcmu_info exynos5800_mau_subcmu = {
+ .gate_clks = exynos5800_mau_gate_clks,
+ .nr_gate_clks = ARRAY_SIZE(exynos5800_mau_gate_clks),
+ .suspend_regs = exynos5800_mau_suspend_regs,
+ .nr_suspend_regs = ARRAY_SIZE(exynos5800_mau_suspend_regs),
+ .pd_name = "MAU",
+};
+
+static const struct exynos5_subcmu_info *exynos5x_subcmus[] = {
+ &exynos5x_disp_subcmu,
+ &exynos5x_gsc_subcmu,
+ &exynos5x_mfc_subcmu,
+ &exynos5x_mscl_subcmu,
+};
+
+static const struct exynos5_subcmu_info *exynos5800_subcmus[] = {
+ &exynos5x_disp_subcmu,
+ &exynos5x_gsc_subcmu,
+ &exynos5x_mfc_subcmu,
+ &exynos5x_mscl_subcmu,
+ &exynos5800_mau_subcmu,
};
static const struct samsung_pll_rate_table exynos5420_pll2550x_24mhz_tbl[] __initconst = {
@@ -1539,11 +1593,17 @@ static void __init exynos5x_clk_init(struct device_node *np,
samsung_clk_extended_sleep_init(reg_base,
exynos5x_clk_regs, ARRAY_SIZE(exynos5x_clk_regs),
exynos5420_set_clksrc, ARRAY_SIZE(exynos5420_set_clksrc));
- if (soc == EXYNOS5800)
+
+ if (soc == EXYNOS5800) {
samsung_clk_sleep_init(reg_base, exynos5800_clk_regs,
ARRAY_SIZE(exynos5800_clk_regs));
- exynos5_subcmus_init(ctx, ARRAY_SIZE(exynos5x_subcmus),
- exynos5x_subcmus);
+
+ exynos5_subcmus_init(ctx, ARRAY_SIZE(exynos5800_subcmus),
+ exynos5800_subcmus);
+ } else {
+ exynos5_subcmus_init(ctx, ARRAY_SIZE(exynos5x_subcmus),
+ exynos5x_subcmus);
+ }
samsung_clk_of_add_provider(np, ctx);
}
diff --git a/drivers/clk/socfpga/clk-periph-s10.c b/drivers/clk/socfpga/clk-periph-s10.c
index 5c50e72..1a191ee 100644
--- a/drivers/clk/socfpga/clk-periph-s10.c
+++ b/drivers/clk/socfpga/clk-periph-s10.c
@@ -38,7 +38,7 @@ static unsigned long clk_peri_cnt_clk_recalc_rate(struct clk_hw *hwclk,
if (socfpgaclk->fixed_div) {
div = socfpgaclk->fixed_div;
} else {
- if (!socfpgaclk->bypass_reg)
+ if (socfpgaclk->hw.reg)
div = ((readl(socfpgaclk->hw.reg) & 0x7ff) + 1);
}
diff --git a/drivers/crypto/ccp/ccp-dev.c b/drivers/crypto/ccp/ccp-dev.c
index f79eede..edefa66 100644
--- a/drivers/crypto/ccp/ccp-dev.c
+++ b/drivers/crypto/ccp/ccp-dev.c
@@ -540,6 +540,10 @@ int ccp_dev_suspend(struct sp_device *sp, pm_message_t state)
unsigned long flags;
unsigned int i;
+ /* If there's no device there's nothing to do */
+ if (!ccp)
+ return 0;
+
spin_lock_irqsave(&ccp->cmd_lock, flags);
ccp->suspending = 1;
@@ -564,6 +568,10 @@ int ccp_dev_resume(struct sp_device *sp)
unsigned long flags;
unsigned int i;
+ /* If there's no device there's nothing to do */
+ if (!ccp)
+ return 0;
+
spin_lock_irqsave(&ccp->cmd_lock, flags);
ccp->suspending = 0;
diff --git a/drivers/crypto/chelsio/chtls/chtls_main.c b/drivers/crypto/chelsio/chtls/chtls_main.c
index 635bb4b..e6df5b9 100644
--- a/drivers/crypto/chelsio/chtls/chtls_main.c
+++ b/drivers/crypto/chelsio/chtls/chtls_main.c
@@ -474,7 +474,8 @@ static int chtls_getsockopt(struct sock *sk, int level, int optname,
struct tls_context *ctx = tls_get_ctx(sk);
if (level != SOL_TLS)
- return ctx->getsockopt(sk, level, optname, optval, optlen);
+ return ctx->sk_proto->getsockopt(sk, level,
+ optname, optval, optlen);
return do_chtls_getsockopt(sk, optval, optlen);
}
@@ -541,7 +542,8 @@ static int chtls_setsockopt(struct sock *sk, int level, int optname,
struct tls_context *ctx = tls_get_ctx(sk);
if (level != SOL_TLS)
- return ctx->setsockopt(sk, level, optname, optval, optlen);
+ return ctx->sk_proto->setsockopt(sk, level,
+ optname, optval, optlen);
return do_chtls_setsockopt(sk, optname, optval, optlen);
}
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 23e0a35..ad72b3f 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -1163,6 +1163,7 @@ static int fsl_dma_chan_probe(struct fsldma_device *fdev,
switch (chan->feature & FSL_DMA_IP_MASK) {
case FSL_DMA_IP_85XX:
chan->toggle_ext_pause = fsl_chan_toggle_ext_pause;
+ /* Fall through */
case FSL_DMA_IP_83XX:
chan->toggle_ext_start = fsl_chan_toggle_ext_start;
chan->set_src_loop_size = fsl_chan_set_src_loop_size;
diff --git a/drivers/fpga/altera-ps-spi.c b/drivers/fpga/altera-ps-spi.c
index a13f224..0221dee 100644
--- a/drivers/fpga/altera-ps-spi.c
+++ b/drivers/fpga/altera-ps-spi.c
@@ -210,7 +210,7 @@ static int altera_ps_write_complete(struct fpga_manager *mgr,
return -EIO;
}
- if (!IS_ERR(conf->confd)) {
+ if (conf->confd) {
if (!gpiod_get_raw_value_cansleep(conf->confd)) {
dev_err(&mgr->dev, "CONF_DONE is inactive!\n");
return -EIO;
@@ -289,10 +289,13 @@ static int altera_ps_probe(struct spi_device *spi)
return PTR_ERR(conf->status);
}
- conf->confd = devm_gpiod_get(&spi->dev, "confd", GPIOD_IN);
+ conf->confd = devm_gpiod_get_optional(&spi->dev, "confd", GPIOD_IN);
if (IS_ERR(conf->confd)) {
- dev_warn(&spi->dev, "Not using confd gpio: %ld\n",
- PTR_ERR(conf->confd));
+ dev_err(&spi->dev, "Failed to get confd gpio: %ld\n",
+ PTR_ERR(conf->confd));
+ return PTR_ERR(conf->confd);
+ } else if (!conf->confd) {
+ dev_warn(&spi->dev, "Not using confd gpio");
}
/* Register manager with unique name */
diff --git a/drivers/fsi/fsi-scom.c b/drivers/fsi/fsi-scom.c
index 343153d..004dc03 100644
--- a/drivers/fsi/fsi-scom.c
+++ b/drivers/fsi/fsi-scom.c
@@ -38,8 +38,7 @@
#define SCOM_STATUS_PIB_RESP_MASK 0x00007000
#define SCOM_STATUS_PIB_RESP_SHIFT 12
-#define SCOM_STATUS_ANY_ERR (SCOM_STATUS_ERR_SUMMARY | \
- SCOM_STATUS_PROTECTION | \
+#define SCOM_STATUS_ANY_ERR (SCOM_STATUS_PROTECTION | \
SCOM_STATUS_PARITY | \
SCOM_STATUS_PIB_ABORT | \
SCOM_STATUS_PIB_RESP_MASK)
@@ -251,11 +250,6 @@ static int handle_fsi2pib_status(struct scom_device *scom, uint32_t status)
/* Return -EBUSY on PIB abort to force a retry */
if (status & SCOM_STATUS_PIB_ABORT)
return -EBUSY;
- if (status & SCOM_STATUS_ERR_SUMMARY) {
- fsi_device_write(scom->fsi_dev, SCOM_FSI2PIB_RESET_REG, &dummy,
- sizeof(uint32_t));
- return -EIO;
- }
return 0;
}
diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
index 567fb98..9762dd6 100644
--- a/drivers/gpio/gpiolib-of.c
+++ b/drivers/gpio/gpiolib-of.c
@@ -363,7 +363,7 @@ struct gpio_desc *of_find_gpio(struct device *dev, const char *con_id,
/* Special handling for SPI GPIOs if used */
if (IS_ERR(desc))
desc = of_find_spi_gpio(dev, con_id, &of_flags);
- if (IS_ERR(desc)) {
+ if (IS_ERR(desc) && PTR_ERR(desc) != -EPROBE_DEFER) {
/* This quirk looks up flags and all */
desc = of_find_spi_cs_gpio(dev, con_id, idx, flags);
if (!IS_ERR(desc))
diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
index f497003..cca7490 100644
--- a/drivers/gpio/gpiolib.c
+++ b/drivers/gpio/gpiolib.c
@@ -1091,9 +1091,11 @@ static long gpio_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
if (test_bit(FLAG_ACTIVE_LOW, &desc->flags))
lineinfo.flags |= GPIOLINE_FLAG_ACTIVE_LOW;
if (test_bit(FLAG_OPEN_DRAIN, &desc->flags))
- lineinfo.flags |= GPIOLINE_FLAG_OPEN_DRAIN;
+ lineinfo.flags |= (GPIOLINE_FLAG_OPEN_DRAIN |
+ GPIOLINE_FLAG_IS_OUT);
if (test_bit(FLAG_OPEN_SOURCE, &desc->flags))
- lineinfo.flags |= GPIOLINE_FLAG_OPEN_SOURCE;
+ lineinfo.flags |= (GPIOLINE_FLAG_OPEN_SOURCE |
+ GPIOLINE_FLAG_IS_OUT);
if (copy_to_user(ip, &lineinfo, sizeof(lineinfo)))
return -EFAULT;
@@ -1371,21 +1373,13 @@ int gpiochip_add_data_with_key(struct gpio_chip *chip, void *data,
if (status)
goto err_remove_from_list;
- status = gpiochip_irqchip_init_valid_mask(chip);
+ status = gpiochip_alloc_valid_mask(chip);
if (status)
goto err_remove_from_list;
- status = gpiochip_alloc_valid_mask(chip);
- if (status)
- goto err_remove_irqchip_mask;
-
- status = gpiochip_add_irqchip(chip, lock_key, request_key);
- if (status)
- goto err_free_gpiochip_mask;
-
status = of_gpiochip_add(chip);
if (status)
- goto err_remove_chip;
+ goto err_free_gpiochip_mask;
status = gpiochip_init_valid_mask(chip);
if (status)
@@ -1411,6 +1405,14 @@ int gpiochip_add_data_with_key(struct gpio_chip *chip, void *data,
machine_gpiochip_add(chip);
+ status = gpiochip_irqchip_init_valid_mask(chip);
+ if (status)
+ goto err_remove_acpi_chip;
+
+ status = gpiochip_add_irqchip(chip, lock_key, request_key);
+ if (status)
+ goto err_remove_irqchip_mask;
+
/*
* By first adding the chardev, and then adding the device,
* we get a device node entry in sysfs under
@@ -1422,21 +1424,21 @@ int gpiochip_add_data_with_key(struct gpio_chip *chip, void *data,
if (gpiolib_initialized) {
status = gpiochip_setup_dev(gdev);
if (status)
- goto err_remove_acpi_chip;
+ goto err_remove_irqchip;
}
return 0;
+err_remove_irqchip:
+ gpiochip_irqchip_remove(chip);
+err_remove_irqchip_mask:
+ gpiochip_irqchip_free_valid_mask(chip);
err_remove_acpi_chip:
acpi_gpiochip_remove(chip);
err_remove_of_chip:
gpiochip_free_hogs(chip);
of_gpiochip_remove(chip);
-err_remove_chip:
- gpiochip_irqchip_remove(chip);
err_free_gpiochip_mask:
gpiochip_free_valid_mask(chip);
-err_remove_irqchip_mask:
- gpiochip_irqchip_free_valid_mask(chip);
err_remove_from_list:
spin_lock_irqsave(&gpio_lock, flags);
list_del(&gdev->list);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c
index 9b384a9..3e35a8f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atpx_handler.c
@@ -574,6 +574,7 @@ static const struct amdgpu_px_quirk amdgpu_px_quirk_list[] = {
{ 0x1002, 0x6900, 0x1002, 0x0124, AMDGPU_PX_QUIRK_FORCE_ATPX },
{ 0x1002, 0x6900, 0x1028, 0x0812, AMDGPU_PX_QUIRK_FORCE_ATPX },
{ 0x1002, 0x6900, 0x1028, 0x0813, AMDGPU_PX_QUIRK_FORCE_ATPX },
+ { 0x1002, 0x699f, 0x1028, 0x0814, AMDGPU_PX_QUIRK_FORCE_ATPX },
{ 0x1002, 0x6900, 0x1025, 0x125A, AMDGPU_PX_QUIRK_FORCE_ATPX },
{ 0x1002, 0x6900, 0x17AA, 0x3806, AMDGPU_PX_QUIRK_FORCE_ATPX },
{ 0, 0, 0, 0, 0 },
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 4e4094f..8b26c97 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1143,6 +1143,9 @@ static int amdgpu_cs_process_syncobj_out_dep(struct amdgpu_cs_parser *p,
num_deps = chunk->length_dw * 4 /
sizeof(struct drm_amdgpu_cs_chunk_sem);
+ if (p->post_deps)
+ return -EINVAL;
+
p->post_deps = kmalloc_array(num_deps, sizeof(*p->post_deps),
GFP_KERNEL);
p->num_post_deps = 0;
@@ -1166,8 +1169,7 @@ static int amdgpu_cs_process_syncobj_out_dep(struct amdgpu_cs_parser *p,
static int amdgpu_cs_process_syncobj_timeline_out_dep(struct amdgpu_cs_parser *p,
- struct amdgpu_cs_chunk
- *chunk)
+ struct amdgpu_cs_chunk *chunk)
{
struct drm_amdgpu_cs_chunk_syncobj *syncobj_deps;
unsigned num_deps;
@@ -1177,6 +1179,9 @@ static int amdgpu_cs_process_syncobj_timeline_out_dep(struct amdgpu_cs_parser *p
num_deps = chunk->length_dw * 4 /
sizeof(struct drm_amdgpu_cs_chunk_syncobj);
+ if (p->post_deps)
+ return -EINVAL;
+
p->post_deps = kmalloc_array(num_deps, sizeof(*p->post_deps),
GFP_KERNEL);
p->num_post_deps = 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
index f539a2a..7398b48 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
@@ -534,21 +534,24 @@ int amdgpu_ctx_wait_prev_fence(struct amdgpu_ctx *ctx,
struct drm_sched_entity *entity)
{
struct amdgpu_ctx_entity *centity = to_amdgpu_ctx_entity(entity);
- unsigned idx = centity->sequence & (amdgpu_sched_jobs - 1);
- struct dma_fence *other = centity->fences[idx];
+ struct dma_fence *other;
+ unsigned idx;
+ long r;
- if (other) {
- signed long r;
- r = dma_fence_wait(other, true);
- if (r < 0) {
- if (r != -ERESTARTSYS)
- DRM_ERROR("Error (%ld) waiting for fence!\n", r);
+ spin_lock(&ctx->ring_lock);
+ idx = centity->sequence & (amdgpu_sched_jobs - 1);
+ other = dma_fence_get(centity->fences[idx]);
+ spin_unlock(&ctx->ring_lock);
- return r;
- }
- }
+ if (!other)
+ return 0;
- return 0;
+ r = dma_fence_wait(other, true);
+ if (r < 0 && r != -ERESTARTSYS)
+ DRM_ERROR("Error (%ld) waiting for fence!\n", r);
+
+ dma_fence_put(other);
+ return r;
}
void amdgpu_ctx_mgr_init(struct amdgpu_ctx_mgr *mgr)
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
index 04b8ac4..c066e1d 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
@@ -596,14 +596,18 @@ static void gfx_v9_0_check_if_need_gfxoff(struct amdgpu_device *adev)
case CHIP_VEGA20:
break;
case CHIP_RAVEN:
- if (adev->rev_id >= 0x8 || adev->pdev->device == 0x15d8)
- break;
- if ((adev->gfx.rlc_fw_version != 106 &&
- adev->gfx.rlc_fw_version < 531) ||
- (adev->gfx.rlc_fw_version == 53815) ||
- (adev->gfx.rlc_feature_version < 1) ||
- !adev->gfx.rlc.is_rlc_v2_1)
+ if (!(adev->rev_id >= 0x8 || adev->pdev->device == 0x15d8)
+ &&((adev->gfx.rlc_fw_version != 106 &&
+ adev->gfx.rlc_fw_version < 531) ||
+ (adev->gfx.rlc_fw_version == 53815) ||
+ (adev->gfx.rlc_feature_version < 1) ||
+ !adev->gfx.rlc.is_rlc_v2_1))
adev->pm.pp_feature &= ~PP_GFXOFF_MASK;
+
+ if (adev->pm.pp_feature & PP_GFXOFF_MASK)
+ adev->pg_flags |= AMD_PG_SUPPORT_GFX_PG |
+ AMD_PG_SUPPORT_CP |
+ AMD_PG_SUPPORT_RLC_SMU_HS;
break;
default:
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/nv.c b/drivers/gpu/drm/amd/amdgpu/nv.c
index 662612f..9922bce 100644
--- a/drivers/gpu/drm/amd/amdgpu/nv.c
+++ b/drivers/gpu/drm/amd/amdgpu/nv.c
@@ -552,7 +552,6 @@ static int nv_common_early_init(void *handle)
AMD_CG_SUPPORT_BIF_LS;
adev->pg_flags = AMD_PG_SUPPORT_VCN |
AMD_PG_SUPPORT_VCN_DPG |
- AMD_PG_SUPPORT_MMHUB |
AMD_PG_SUPPORT_ATHUB;
adev->external_rev_id = adev->rev_id + 0x1;
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index 2326541..04fbf05 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -992,11 +992,6 @@ static int soc15_common_early_init(void *handle)
adev->pg_flags = AMD_PG_SUPPORT_SDMA | AMD_PG_SUPPORT_VCN;
}
-
- if (adev->pm.pp_feature & PP_GFXOFF_MASK)
- adev->pg_flags |= AMD_PG_SUPPORT_GFX_PG |
- AMD_PG_SUPPORT_CP |
- AMD_PG_SUPPORT_RLC_SMU_HS;
break;
default:
/* FIXME: not supported yet */
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 4a29f72..45be7a2 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -3131,13 +3131,25 @@ static enum dc_color_depth
convert_color_depth_from_display_info(const struct drm_connector *connector,
const struct drm_connector_state *state)
{
- uint32_t bpc = connector->display_info.bpc;
+ uint8_t bpc = (uint8_t)connector->display_info.bpc;
+
+ /* Assume 8 bpc by default if no bpc is specified. */
+ bpc = bpc ? bpc : 8;
if (!state)
state = connector->state;
if (state) {
- bpc = state->max_bpc;
+ /*
+ * Cap display bpc based on the user requested value.
+ *
+ * The value for state->max_bpc may not correctly updated
+ * depending on when the connector gets added to the state
+ * or if this was called outside of atomic check, so it
+ * can't be used directly.
+ */
+ bpc = min(bpc, state->max_requested_bpc);
+
/* Round down to the nearest even number. */
bpc = bpc - (bpc & 1);
}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c
index f27c6fb..90c4e87 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c
@@ -2101,7 +2101,11 @@ static int vega20_get_gpu_power(struct pp_hwmgr *hwmgr,
if (ret)
return ret;
- *query = metrics_table.CurrSocketPower << 8;
+ /* For the 40.46 release, they changed the value name */
+ if (hwmgr->smu_version == 0x282e00)
+ *query = metrics_table.AverageSocketPower << 8;
+ else
+ *query = metrics_table.CurrSocketPower << 8;
return ret;
}
@@ -2349,12 +2353,16 @@ static int vega20_force_dpm_highest(struct pp_hwmgr *hwmgr)
data->dpm_table.soc_table.dpm_state.soft_max_level =
data->dpm_table.soc_table.dpm_levels[soft_level].value;
- ret = vega20_upload_dpm_min_level(hwmgr, 0xFFFFFFFF);
+ ret = vega20_upload_dpm_min_level(hwmgr, FEATURE_DPM_GFXCLK_MASK |
+ FEATURE_DPM_UCLK_MASK |
+ FEATURE_DPM_SOCCLK_MASK);
PP_ASSERT_WITH_CODE(!ret,
"Failed to upload boot level to highest!",
return ret);
- ret = vega20_upload_dpm_max_level(hwmgr, 0xFFFFFFFF);
+ ret = vega20_upload_dpm_max_level(hwmgr, FEATURE_DPM_GFXCLK_MASK |
+ FEATURE_DPM_UCLK_MASK |
+ FEATURE_DPM_SOCCLK_MASK);
PP_ASSERT_WITH_CODE(!ret,
"Failed to upload dpm max level to highest!",
return ret);
@@ -2387,12 +2395,16 @@ static int vega20_force_dpm_lowest(struct pp_hwmgr *hwmgr)
data->dpm_table.soc_table.dpm_state.soft_max_level =
data->dpm_table.soc_table.dpm_levels[soft_level].value;
- ret = vega20_upload_dpm_min_level(hwmgr, 0xFFFFFFFF);
+ ret = vega20_upload_dpm_min_level(hwmgr, FEATURE_DPM_GFXCLK_MASK |
+ FEATURE_DPM_UCLK_MASK |
+ FEATURE_DPM_SOCCLK_MASK);
PP_ASSERT_WITH_CODE(!ret,
"Failed to upload boot level to highest!",
return ret);
- ret = vega20_upload_dpm_max_level(hwmgr, 0xFFFFFFFF);
+ ret = vega20_upload_dpm_max_level(hwmgr, FEATURE_DPM_GFXCLK_MASK |
+ FEATURE_DPM_UCLK_MASK |
+ FEATURE_DPM_SOCCLK_MASK);
PP_ASSERT_WITH_CODE(!ret,
"Failed to upload dpm max level to highest!",
return ret);
@@ -2403,14 +2415,54 @@ static int vega20_force_dpm_lowest(struct pp_hwmgr *hwmgr)
static int vega20_unforce_dpm_levels(struct pp_hwmgr *hwmgr)
{
+ struct vega20_hwmgr *data =
+ (struct vega20_hwmgr *)(hwmgr->backend);
+ uint32_t soft_min_level, soft_max_level;
int ret = 0;
- ret = vega20_upload_dpm_min_level(hwmgr, 0xFFFFFFFF);
+ /* gfxclk soft min/max settings */
+ soft_min_level =
+ vega20_find_lowest_dpm_level(&(data->dpm_table.gfx_table));
+ soft_max_level =
+ vega20_find_highest_dpm_level(&(data->dpm_table.gfx_table));
+
+ data->dpm_table.gfx_table.dpm_state.soft_min_level =
+ data->dpm_table.gfx_table.dpm_levels[soft_min_level].value;
+ data->dpm_table.gfx_table.dpm_state.soft_max_level =
+ data->dpm_table.gfx_table.dpm_levels[soft_max_level].value;
+
+ /* uclk soft min/max settings */
+ soft_min_level =
+ vega20_find_lowest_dpm_level(&(data->dpm_table.mem_table));
+ soft_max_level =
+ vega20_find_highest_dpm_level(&(data->dpm_table.mem_table));
+
+ data->dpm_table.mem_table.dpm_state.soft_min_level =
+ data->dpm_table.mem_table.dpm_levels[soft_min_level].value;
+ data->dpm_table.mem_table.dpm_state.soft_max_level =
+ data->dpm_table.mem_table.dpm_levels[soft_max_level].value;
+
+ /* socclk soft min/max settings */
+ soft_min_level =
+ vega20_find_lowest_dpm_level(&(data->dpm_table.soc_table));
+ soft_max_level =
+ vega20_find_highest_dpm_level(&(data->dpm_table.soc_table));
+
+ data->dpm_table.soc_table.dpm_state.soft_min_level =
+ data->dpm_table.soc_table.dpm_levels[soft_min_level].value;
+ data->dpm_table.soc_table.dpm_state.soft_max_level =
+ data->dpm_table.soc_table.dpm_levels[soft_max_level].value;
+
+ ret = vega20_upload_dpm_min_level(hwmgr, FEATURE_DPM_GFXCLK_MASK |
+ FEATURE_DPM_UCLK_MASK |
+ FEATURE_DPM_SOCCLK_MASK);
PP_ASSERT_WITH_CODE(!ret,
"Failed to upload DPM Bootup Levels!",
return ret);
- ret = vega20_upload_dpm_max_level(hwmgr, 0xFFFFFFFF);
+ ret = vega20_upload_dpm_max_level(hwmgr, FEATURE_DPM_GFXCLK_MASK |
+ FEATURE_DPM_UCLK_MASK |
+ FEATURE_DPM_SOCCLK_MASK);
PP_ASSERT_WITH_CODE(!ret,
"Failed to upload DPM Max Levels!",
return ret);
diff --git a/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h
index a0f52c8..a78b2e29 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h
@@ -907,8 +907,6 @@ struct smu_funcs
((smu)->funcs->register_irq_handler ? (smu)->funcs->register_irq_handler(smu) : 0)
#define smu_set_azalia_d3_pme(smu) \
((smu)->funcs->set_azalia_d3_pme ? (smu)->funcs->set_azalia_d3_pme((smu)) : 0)
-#define smu_get_uclk_dpm_states(smu, clocks_in_khz, num_states) \
- ((smu)->ppt_funcs->get_uclk_dpm_states ? (smu)->ppt_funcs->get_uclk_dpm_states((smu), (clocks_in_khz), (num_states)) : 0)
#define smu_get_max_sustainable_clocks_by_dc(smu, max_clocks) \
((smu)->funcs->get_max_sustainable_clocks_by_dc ? (smu)->funcs->get_max_sustainable_clocks_by_dc((smu), (max_clocks)) : 0)
#define smu_get_uclk_dpm_states(smu, clocks_in_khz, num_states) \
diff --git a/drivers/gpu/drm/amd/powerplay/smu_v11_0.c b/drivers/gpu/drm/amd/powerplay/smu_v11_0.c
index 5fde5cf..5309796 100644
--- a/drivers/gpu/drm/amd/powerplay/smu_v11_0.c
+++ b/drivers/gpu/drm/amd/powerplay/smu_v11_0.c
@@ -326,7 +326,8 @@ static int smu_v11_0_setup_pptable(struct smu_context *smu)
struct amdgpu_device *adev = smu->adev;
const struct smc_firmware_header_v1_0 *hdr;
int ret, index;
- uint32_t size;
+ uint32_t size = 0;
+ uint16_t atom_table_size;
uint8_t frev, crev;
void *table;
uint16_t version_major, version_minor;
@@ -354,10 +355,11 @@ static int smu_v11_0_setup_pptable(struct smu_context *smu)
index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1,
powerplayinfo);
- ret = smu_get_atom_data_table(smu, index, (uint16_t *)&size, &frev, &crev,
+ ret = smu_get_atom_data_table(smu, index, &atom_table_size, &frev, &crev,
(uint8_t **)&table);
if (ret)
return ret;
+ size = atom_table_size;
}
if (!smu->smu_table.power_play_table)
diff --git a/drivers/gpu/drm/amd/powerplay/vega20_ppt.c b/drivers/gpu/drm/amd/powerplay/vega20_ppt.c
index dd6fd1c..6a14497 100644
--- a/drivers/gpu/drm/amd/powerplay/vega20_ppt.c
+++ b/drivers/gpu/drm/amd/powerplay/vega20_ppt.c
@@ -3050,6 +3050,7 @@ static int vega20_get_fan_speed_percent(struct smu_context *smu,
static int vega20_get_gpu_power(struct smu_context *smu, uint32_t *value)
{
+ uint32_t smu_version;
int ret = 0;
SmuMetrics_t metrics;
@@ -3060,7 +3061,15 @@ static int vega20_get_gpu_power(struct smu_context *smu, uint32_t *value)
if (ret)
return ret;
- *value = metrics.CurrSocketPower << 8;
+ ret = smu_get_smc_version(smu, NULL, &smu_version);
+ if (ret)
+ return ret;
+
+ /* For the 40.46 release, they changed the value name */
+ if (smu_version == 0x282e00)
+ *value = metrics.AverageSocketPower << 8;
+ else
+ *value = metrics.CurrSocketPower << 8;
return 0;
}
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_dev.c b/drivers/gpu/drm/arm/display/komeda/komeda_dev.c
index 5a11898..9d4d507 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_dev.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_dev.c
@@ -8,6 +8,7 @@
#include <linux/iommu.h>
#include <linux/of_device.h>
#include <linux/of_graph.h>
+#include <linux/of_reserved_mem.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#ifdef CONFIG_DEBUG_FS
@@ -126,7 +127,7 @@ static int komeda_parse_pipe_dt(struct komeda_dev *mdev, struct device_node *np)
pipe->of_output_port =
of_graph_get_port_by_id(np, KOMEDA_OF_PORT_OUTPUT);
- pipe->of_node = np;
+ pipe->of_node = of_node_get(np);
return 0;
}
@@ -143,6 +144,12 @@ static int komeda_parse_dt(struct device *dev, struct komeda_dev *mdev)
return mdev->irq;
}
+ /* Get the optional framebuffer memory resource */
+ ret = of_reserved_mem_device_init(dev);
+ if (ret && ret != -ENODEV)
+ return ret;
+ ret = 0;
+
for_each_available_child_of_node(np, child) {
if (of_node_cmp(child->name, "pipeline") == 0) {
ret = komeda_parse_pipe_dt(mdev, child);
@@ -289,6 +296,8 @@ void komeda_dev_destroy(struct komeda_dev *mdev)
mdev->n_pipelines = 0;
+ of_reserved_mem_device_release(dev);
+
if (funcs && funcs->cleanup)
funcs->cleanup(mdev);
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.c b/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.c
index cd4d9f5..c9a1edb 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.c
@@ -35,6 +35,25 @@ komeda_get_format_caps(struct komeda_format_caps_table *table,
return NULL;
}
+u32 komeda_get_afbc_format_bpp(const struct drm_format_info *info, u64 modifier)
+{
+ u32 bpp;
+
+ switch (info->format) {
+ case DRM_FORMAT_YUV420_8BIT:
+ bpp = 12;
+ break;
+ case DRM_FORMAT_YUV420_10BIT:
+ bpp = 15;
+ break;
+ default:
+ bpp = info->cpp[0] * 8;
+ break;
+ }
+
+ return bpp;
+}
+
/* Two assumptions
* 1. RGB always has YTR
* 2. Tiled RGB always has SC
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h b/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h
index 3631910..32273cf 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h
@@ -97,6 +97,9 @@ const struct komeda_format_caps *
komeda_get_format_caps(struct komeda_format_caps_table *table,
u32 fourcc, u64 modifier);
+u32 komeda_get_afbc_format_bpp(const struct drm_format_info *info,
+ u64 modifier);
+
u32 *komeda_get_layer_fourcc_list(struct komeda_format_caps_table *table,
u32 layer_type, u32 *n_fmts);
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c b/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c
index 3b0a70e..1b01a62 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c
@@ -43,7 +43,7 @@ komeda_fb_afbc_size_check(struct komeda_fb *kfb, struct drm_file *file,
struct drm_framebuffer *fb = &kfb->base;
const struct drm_format_info *info = fb->format;
struct drm_gem_object *obj;
- u32 alignment_w = 0, alignment_h = 0, alignment_header, n_blocks;
+ u32 alignment_w = 0, alignment_h = 0, alignment_header, n_blocks, bpp;
u64 min_size;
obj = drm_gem_object_lookup(file, mode_cmd->handles[0]);
@@ -88,8 +88,9 @@ komeda_fb_afbc_size_check(struct komeda_fb *kfb, struct drm_file *file,
kfb->offset_payload = ALIGN(n_blocks * AFBC_HEADER_SIZE,
alignment_header);
+ bpp = komeda_get_afbc_format_bpp(info, fb->modifier);
kfb->afbc_size = kfb->offset_payload + n_blocks *
- ALIGN(info->cpp[0] * AFBC_SUPERBLK_PIXELS,
+ ALIGN(bpp * AFBC_SUPERBLK_PIXELS / 8,
AFBC_SUPERBLK_ALIGNMENT);
min_size = kfb->afbc_size + fb->offsets[0];
if (min_size > obj->size) {
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_kms.c b/drivers/gpu/drm/arm/display/komeda/komeda_kms.c
index 419a8b0..69d9e26 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_kms.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_kms.c
@@ -14,6 +14,7 @@
#include <drm/drm_gem_cma_helper.h>
#include <drm/drm_gem_framebuffer_helper.h>
#include <drm/drm_irq.h>
+#include <drm/drm_probe_helper.h>
#include <drm/drm_vblank.h>
#include "komeda_dev.h"
@@ -146,7 +147,6 @@ static int komeda_crtc_normalize_zpos(struct drm_crtc *crtc,
struct komeda_crtc_state *kcrtc_st = to_kcrtc_st(crtc_st);
struct komeda_plane_state *kplane_st;
struct drm_plane_state *plane_st;
- struct drm_framebuffer *fb;
struct drm_plane *plane;
struct list_head zorder_list;
int order = 0, err;
@@ -172,7 +172,6 @@ static int komeda_crtc_normalize_zpos(struct drm_crtc *crtc,
list_for_each_entry(kplane_st, &zorder_list, zlist_node) {
plane_st = &kplane_st->base;
- fb = plane_st->fb;
plane = plane_st->plane;
plane_st->normalized_zpos = order++;
@@ -205,7 +204,7 @@ static int komeda_kms_check(struct drm_device *dev,
struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
- struct drm_crtc_state *old_crtc_st, *new_crtc_st;
+ struct drm_crtc_state *new_crtc_st;
int i, err;
err = drm_atomic_helper_check_modeset(dev, state);
@@ -216,7 +215,7 @@ static int komeda_kms_check(struct drm_device *dev,
* so need to add all affected_planes (even unchanged) to
* drm_atomic_state.
*/
- for_each_oldnew_crtc_in_state(state, crtc, old_crtc_st, new_crtc_st, i) {
+ for_each_new_crtc_in_state(state, crtc, new_crtc_st, i) {
err = drm_atomic_add_affected_planes(state, crtc);
if (err)
return err;
@@ -307,24 +306,33 @@ struct komeda_kms_dev *komeda_kms_attach(struct komeda_dev *mdev)
komeda_kms_irq_handler, IRQF_SHARED,
drm->driver->name, drm);
if (err)
- goto cleanup_mode_config;
+ goto free_component_binding;
err = mdev->funcs->enable_irq(mdev);
if (err)
- goto cleanup_mode_config;
+ goto free_component_binding;
drm->irq_enabled = true;
+ drm_kms_helper_poll_init(drm);
+
err = drm_dev_register(drm, 0);
if (err)
- goto cleanup_mode_config;
+ goto free_interrupts;
return kms;
-cleanup_mode_config:
+free_interrupts:
+ drm_kms_helper_poll_fini(drm);
drm->irq_enabled = false;
+ mdev->funcs->disable_irq(mdev);
+free_component_binding:
+ component_unbind_all(mdev->dev, drm);
+cleanup_mode_config:
drm_mode_config_cleanup(drm);
komeda_kms_cleanup_private_objs(kms);
+ drm->dev_private = NULL;
+ drm_dev_put(drm);
free_kms:
kfree(kms);
return ERR_PTR(err);
@@ -335,12 +343,14 @@ void komeda_kms_detach(struct komeda_kms_dev *kms)
struct drm_device *drm = &kms->base;
struct komeda_dev *mdev = drm->dev_private;
+ drm_dev_unregister(drm);
+ drm_kms_helper_poll_fini(drm);
+ drm_atomic_helper_shutdown(drm);
drm->irq_enabled = false;
mdev->funcs->disable_irq(mdev);
- drm_dev_unregister(drm);
component_unbind_all(mdev->dev, drm);
- komeda_kms_cleanup_private_objs(kms);
drm_mode_config_cleanup(drm);
+ komeda_kms_cleanup_private_objs(kms);
drm->dev_private = NULL;
drm_dev_put(drm);
}
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h b/drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h
index a90bcbb..14b6831 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_pipeline.h
@@ -480,6 +480,7 @@ void komeda_pipeline_dump_register(struct komeda_pipeline *pipe,
struct seq_file *sf);
/* component APIs */
+extern __printf(10, 11)
struct komeda_component *
komeda_component_add(struct komeda_pipeline *pipe,
size_t comp_sz, u32 id, u32 hw_id,
diff --git a/drivers/gpu/drm/arm/display/komeda/komeda_wb_connector.c b/drivers/gpu/drm/arm/display/komeda/komeda_wb_connector.c
index 617e1f7..2851cac 100644
--- a/drivers/gpu/drm/arm/display/komeda/komeda_wb_connector.c
+++ b/drivers/gpu/drm/arm/display/komeda/komeda_wb_connector.c
@@ -148,7 +148,7 @@ static int komeda_wb_connector_add(struct komeda_kms_dev *kms,
if (!kcrtc->master->wb_layer)
return 0;
- kwb_conn = kzalloc(sizeof(*wb_conn), GFP_KERNEL);
+ kwb_conn = kzalloc(sizeof(*kwb_conn), GFP_KERNEL);
if (!kwb_conn)
return -ENOMEM;
diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c
index 7925a17..1cb1fa7 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi.c
+++ b/drivers/gpu/drm/i915/display/intel_ddi.c
@@ -1465,8 +1465,8 @@ static void ddi_dotclock_get(struct intel_crtc_state *pipe_config)
else if (intel_crtc_has_dp_encoder(pipe_config))
dotclock = intel_dotclock_calculate(pipe_config->port_clock,
&pipe_config->dp_m_n);
- else if (pipe_config->has_hdmi_sink && pipe_config->pipe_bpp == 36)
- dotclock = pipe_config->port_clock * 2 / 3;
+ else if (pipe_config->has_hdmi_sink && pipe_config->pipe_bpp > 24)
+ dotclock = pipe_config->port_clock * 24 / pipe_config->pipe_bpp;
else
dotclock = pipe_config->port_clock;
diff --git a/drivers/gpu/drm/i915/display/intel_dp_mst.c b/drivers/gpu/drm/i915/display/intel_dp_mst.c
index 60652eb..18e4cba 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_mst.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_mst.c
@@ -539,7 +539,15 @@ static struct drm_connector *intel_dp_add_mst_connector(struct drm_dp_mst_topolo
intel_attach_force_audio_property(connector);
intel_attach_broadcast_rgb_property(connector);
- drm_connector_attach_max_bpc_property(connector, 6, 12);
+
+ /*
+ * Reuse the prop from the SST connector because we're
+ * not allowed to create new props after device registration.
+ */
+ connector->max_bpc_property =
+ intel_dp->attached_connector->base.max_bpc_property;
+ if (connector->max_bpc_property)
+ drm_connector_attach_max_bpc_property(connector, 6, 12);
return connector;
diff --git a/drivers/gpu/drm/i915/display/intel_vdsc.c b/drivers/gpu/drm/i915/display/intel_vdsc.c
index ffec807..f413904 100644
--- a/drivers/gpu/drm/i915/display/intel_vdsc.c
+++ b/drivers/gpu/drm/i915/display/intel_vdsc.c
@@ -541,7 +541,7 @@ static void intel_configure_pps_for_dsc_encoder(struct intel_encoder *encoder,
pps_val |= DSC_PIC_HEIGHT(vdsc_cfg->pic_height) |
DSC_PIC_WIDTH(vdsc_cfg->pic_width / num_vdsc_instances);
DRM_INFO("PPS2 = 0x%08x\n", pps_val);
- if (encoder->type == INTEL_OUTPUT_EDP) {
+ if (cpu_transcoder == TRANSCODER_EDP) {
I915_WRITE(DSCA_PICTURE_PARAMETER_SET_2, pps_val);
/*
* If 2 VDSC instances are needed, configure PPS for second
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index f62e339..bac1ee9 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -1598,6 +1598,12 @@ static int i915_driver_init_hw(struct drm_i915_private *dev_priv)
pci_set_master(pdev);
+ /*
+ * We don't have a max segment size, so set it to the max so sg's
+ * debugging layer doesn't complain
+ */
+ dma_set_max_seg_size(&pdev->dev, UINT_MAX);
+
/* overlay on gen2 is broken and can't address above 1G */
if (IS_GEN(dev_priv, 2)) {
ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(30));
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index 94d3992..724627a 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -101,6 +101,9 @@ static struct _balloon_info_ bl_info;
static void vgt_deballoon_space(struct i915_ggtt *ggtt,
struct drm_mm_node *node)
{
+ if (!drm_mm_node_allocated(node))
+ return;
+
DRM_DEBUG_DRIVER("deballoon space: range [0x%llx - 0x%llx] %llu KiB.\n",
node->start,
node->start + node->size,
diff --git a/drivers/gpu/drm/i915/intel_drv.h b/drivers/gpu/drm/i915/intel_drv.h
index 1d58f7e..f119798 100644
--- a/drivers/gpu/drm/i915/intel_drv.h
+++ b/drivers/gpu/drm/i915/intel_drv.h
@@ -829,7 +829,7 @@ struct intel_crtc_state {
/*
* Frequence the dpll for the port should run at. Differs from the
- * adjusted dotclock e.g. for DP or 12bpc hdmi mode. This is also
+ * adjusted dotclock e.g. for DP or 10/12bpc hdmi mode. This is also
* already multiplied by pixel_multiplier.
*/
int port_clock;
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
index 95fdbd0..945bc20 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_drv.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
@@ -17,6 +17,7 @@
#include <linux/of_address.h>
#include <linux/of_platform.h>
#include <linux/pm_runtime.h>
+#include <linux/dma-mapping.h>
#include "mtk_drm_crtc.h"
#include "mtk_drm_ddp.h"
@@ -213,6 +214,7 @@ static int mtk_drm_kms_init(struct drm_device *drm)
struct mtk_drm_private *private = drm->dev_private;
struct platform_device *pdev;
struct device_node *np;
+ struct device *dma_dev;
int ret;
if (!iommu_present(&platform_bus_type))
@@ -275,7 +277,29 @@ static int mtk_drm_kms_init(struct drm_device *drm)
goto err_component_unbind;
}
- private->dma_dev = &pdev->dev;
+ dma_dev = &pdev->dev;
+ private->dma_dev = dma_dev;
+
+ /*
+ * Configure the DMA segment size to make sure we get contiguous IOVA
+ * when importing PRIME buffers.
+ */
+ if (!dma_dev->dma_parms) {
+ private->dma_parms_allocated = true;
+ dma_dev->dma_parms =
+ devm_kzalloc(drm->dev, sizeof(*dma_dev->dma_parms),
+ GFP_KERNEL);
+ }
+ if (!dma_dev->dma_parms) {
+ ret = -ENOMEM;
+ goto err_component_unbind;
+ }
+
+ ret = dma_set_max_seg_size(dma_dev, (unsigned int)DMA_BIT_MASK(32));
+ if (ret) {
+ dev_err(dma_dev, "Failed to set DMA segment size\n");
+ goto err_unset_dma_parms;
+ }
/*
* We don't use the drm_irq_install() helpers provided by the DRM
@@ -285,13 +309,16 @@ static int mtk_drm_kms_init(struct drm_device *drm)
drm->irq_enabled = true;
ret = drm_vblank_init(drm, MAX_CRTC);
if (ret < 0)
- goto err_component_unbind;
+ goto err_unset_dma_parms;
drm_kms_helper_poll_init(drm);
drm_mode_config_reset(drm);
return 0;
+err_unset_dma_parms:
+ if (private->dma_parms_allocated)
+ dma_dev->dma_parms = NULL;
err_component_unbind:
component_unbind_all(drm->dev, drm);
err_config_cleanup:
@@ -302,9 +329,14 @@ static int mtk_drm_kms_init(struct drm_device *drm)
static void mtk_drm_kms_deinit(struct drm_device *drm)
{
+ struct mtk_drm_private *private = drm->dev_private;
+
drm_kms_helper_poll_fini(drm);
drm_atomic_helper_shutdown(drm);
+ if (private->dma_parms_allocated)
+ private->dma_dev->dma_parms = NULL;
+
component_unbind_all(drm->dev, drm);
drm_mode_config_cleanup(drm);
}
@@ -320,6 +352,18 @@ static const struct file_operations mtk_drm_fops = {
.compat_ioctl = drm_compat_ioctl,
};
+/*
+ * We need to override this because the device used to import the memory is
+ * not dev->dev, as drm_gem_prime_import() expects.
+ */
+struct drm_gem_object *mtk_drm_gem_prime_import(struct drm_device *dev,
+ struct dma_buf *dma_buf)
+{
+ struct mtk_drm_private *private = dev->dev_private;
+
+ return drm_gem_prime_import_dev(dev, dma_buf, private->dma_dev);
+}
+
static struct drm_driver mtk_drm_driver = {
.driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME |
DRIVER_ATOMIC,
@@ -331,7 +375,7 @@ static struct drm_driver mtk_drm_driver = {
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
.gem_prime_export = drm_gem_prime_export,
- .gem_prime_import = drm_gem_prime_import,
+ .gem_prime_import = mtk_drm_gem_prime_import,
.gem_prime_get_sg_table = mtk_gem_prime_get_sg_table,
.gem_prime_import_sg_table = mtk_gem_prime_import_sg_table,
.gem_prime_mmap = mtk_drm_gem_mmap_buf,
@@ -524,12 +568,15 @@ static int mtk_drm_probe(struct platform_device *pdev)
comp = devm_kzalloc(dev, sizeof(*comp), GFP_KERNEL);
if (!comp) {
ret = -ENOMEM;
+ of_node_put(node);
goto err_node;
}
ret = mtk_ddp_comp_init(dev, node, comp, comp_id, NULL);
- if (ret)
+ if (ret) {
+ of_node_put(node);
goto err_node;
+ }
private->ddp_comp[comp_id] = comp;
}
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.h b/drivers/gpu/drm/mediatek/mtk_drm_drv.h
index 598ff3e..e03fea1 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_drv.h
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.h
@@ -51,6 +51,8 @@ struct mtk_drm_private {
} commit;
struct drm_atomic_state *suspend_state;
+
+ bool dma_parms_allocated;
};
extern struct platform_driver mtk_ddp_driver;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.c b/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.c
index b4e7404..a11637b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.c
@@ -40,8 +40,7 @@ nvkm_i2c_aux_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs, int num)
u8 *ptr = msg->buf;
while (remaining) {
- u8 cnt = (remaining > 16) ? 16 : remaining;
- u8 cmd;
+ u8 cnt, retries, cmd;
if (msg->flags & I2C_M_RD)
cmd = 1;
@@ -51,10 +50,19 @@ nvkm_i2c_aux_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs, int num)
if (mcnt || remaining > 16)
cmd |= 4; /* MOT */
- ret = aux->func->xfer(aux, true, cmd, msg->addr, ptr, &cnt);
- if (ret < 0) {
- nvkm_i2c_aux_release(aux);
- return ret;
+ for (retries = 0, cnt = 0;
+ retries < 32 && !cnt;
+ retries++) {
+ cnt = min_t(u8, remaining, 16);
+ ret = aux->func->xfer(aux, true, cmd,
+ msg->addr, ptr, &cnt);
+ if (ret < 0)
+ goto out;
+ }
+ if (!cnt) {
+ AUX_TRACE(aux, "no data after 32 retries");
+ ret = -EIO;
+ goto out;
}
ptr += cnt;
@@ -64,8 +72,10 @@ nvkm_i2c_aux_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs, int num)
msg++;
}
+ ret = num;
+out:
nvkm_i2c_aux_release(aux);
- return num;
+ return ret;
}
static u32
diff --git a/drivers/gpu/drm/omapdrm/dss/output.c b/drivers/gpu/drm/omapdrm/dss/output.c
index de0f882f..14b41de 100644
--- a/drivers/gpu/drm/omapdrm/dss/output.c
+++ b/drivers/gpu/drm/omapdrm/dss/output.c
@@ -4,6 +4,7 @@
* Author: Archit Taneja <archit@ti.com>
*/
+#include <linux/bitops.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
@@ -20,7 +21,8 @@ int omapdss_device_init_output(struct omap_dss_device *out)
{
struct device_node *remote_node;
- remote_node = of_graph_get_remote_node(out->dev->of_node, 0, 0);
+ remote_node = of_graph_get_remote_node(out->dev->of_node,
+ ffs(out->of_ports) - 1, 0);
if (!remote_node) {
dev_dbg(out->dev, "failed to find video sink\n");
return 0;
diff --git a/drivers/gpu/drm/omapdrm/omap_drv.c b/drivers/gpu/drm/omapdrm/omap_drv.c
index 288c59d..1bad0a2 100644
--- a/drivers/gpu/drm/omapdrm/omap_drv.c
+++ b/drivers/gpu/drm/omapdrm/omap_drv.c
@@ -669,7 +669,7 @@ static int pdev_probe(struct platform_device *pdev)
if (omapdss_is_initialized() == false)
return -EPROBE_DEFER;
- ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
+ ret = dma_coerce_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
if (ret) {
dev_err(&pdev->dev, "Failed to set the DMA mask\n");
return ret;
diff --git a/drivers/gpu/drm/qxl/qxl_drv.c b/drivers/gpu/drm/qxl/qxl_drv.c
index f33e349..952201c 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.c
+++ b/drivers/gpu/drm/qxl/qxl_drv.c
@@ -59,6 +59,11 @@ module_param_named(num_heads, qxl_num_crtc, int, 0400);
static struct drm_driver qxl_driver;
static struct pci_driver qxl_pci_driver;
+static bool is_vga(struct pci_dev *pdev)
+{
+ return pdev->class == PCI_CLASS_DISPLAY_VGA << 8;
+}
+
static int
qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
@@ -83,9 +88,17 @@ qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ret)
goto disable_pci;
+ if (is_vga(pdev)) {
+ ret = vga_get_interruptible(pdev, VGA_RSRC_LEGACY_IO);
+ if (ret) {
+ DRM_ERROR("can't get legacy vga ioports\n");
+ goto disable_pci;
+ }
+ }
+
ret = qxl_device_init(qdev, &qxl_driver, pdev);
if (ret)
- goto disable_pci;
+ goto put_vga;
ret = qxl_modeset_init(qdev);
if (ret)
@@ -105,6 +118,9 @@ qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
qxl_modeset_fini(qdev);
unload:
qxl_device_fini(qdev);
+put_vga:
+ if (is_vga(pdev))
+ vga_put(pdev, VGA_RSRC_LEGACY_IO);
disable_pci:
pci_disable_device(pdev);
free_dev:
@@ -122,6 +138,8 @@ qxl_pci_remove(struct pci_dev *pdev)
qxl_modeset_fini(qdev);
qxl_device_fini(qdev);
+ if (is_vga(pdev))
+ vga_put(pdev, VGA_RSRC_LEGACY_IO);
dev->dev_private = NULL;
kfree(qdev);
diff --git a/drivers/gpu/drm/rcar-du/rcar_lvds.c b/drivers/gpu/drm/rcar-du/rcar_lvds.c
index 1c62578..082d02c 100644
--- a/drivers/gpu/drm/rcar-du/rcar_lvds.c
+++ b/drivers/gpu/drm/rcar-du/rcar_lvds.c
@@ -673,10 +673,8 @@ static int rcar_lvds_parse_dt_companion(struct rcar_lvds *lvds)
/* Locate the companion LVDS encoder for dual-link operation, if any. */
companion = of_parse_phandle(dev->of_node, "renesas,companion", 0);
- if (!companion) {
- dev_err(dev, "Companion LVDS encoder not found\n");
- return -ENXIO;
- }
+ if (!companion)
+ return 0;
/*
* Sanity check: the companion encoder must have the same compatible
diff --git a/drivers/gpu/drm/sun4i/sun4i_tcon.c b/drivers/gpu/drm/sun4i/sun4i_tcon.c
index 64c43ee..df0cc8f 100644
--- a/drivers/gpu/drm/sun4i/sun4i_tcon.c
+++ b/drivers/gpu/drm/sun4i/sun4i_tcon.c
@@ -314,6 +314,7 @@ static void sun4i_tcon0_mode_set_dithering(struct sun4i_tcon *tcon,
/* R and B components are only 5 bits deep */
val |= SUN4I_TCON0_FRM_CTL_MODE_R;
val |= SUN4I_TCON0_FRM_CTL_MODE_B;
+ /* Fall through */
case MEDIA_BUS_FMT_RGB666_1X18:
case MEDIA_BUS_FMT_RGB666_1X7X3_SPWG:
/* Fall through: enable dithering */
diff --git a/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c b/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c
index a1fc8b5..b889ad3 100644
--- a/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c
+++ b/drivers/gpu/drm/sun4i/sun6i_mipi_dsi.c
@@ -993,6 +993,7 @@ static ssize_t sun6i_dsi_transfer(struct mipi_dsi_host *host,
ret = sun6i_dsi_dcs_read(dsi, msg);
break;
}
+ /* Else, fall through */
default:
ret = -EINVAL;
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index b2da313..09b5265 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -204,6 +204,7 @@ int virtio_gpu_object_get_sg_table(struct virtio_gpu_device *qdev,
.interruptible = false,
.no_wait_gpu = false
};
+ size_t max_segment;
/* wtf swapping */
if (bo->pages)
@@ -215,8 +216,13 @@ int virtio_gpu_object_get_sg_table(struct virtio_gpu_device *qdev,
if (!bo->pages)
goto out;
- ret = sg_alloc_table_from_pages(bo->pages, pages, nr_pages, 0,
- nr_pages << PAGE_SHIFT, GFP_KERNEL);
+ max_segment = virtio_max_dma_size(qdev->vdev);
+ max_segment &= PAGE_MASK;
+ if (max_segment > SCATTERLIST_MAX_SEGMENT)
+ max_segment = SCATTERLIST_MAX_SEGMENT;
+ ret = __sg_alloc_table_from_pages(bo->pages, pages, nr_pages, 0,
+ nr_pages << PAGE_SHIFT,
+ max_segment, GFP_KERNEL);
if (ret)
goto out;
return 0;
diff --git a/drivers/hid/hid-cp2112.c b/drivers/hid/hid-cp2112.c
index 2310c96c..db1b55d 100644
--- a/drivers/hid/hid-cp2112.c
+++ b/drivers/hid/hid-cp2112.c
@@ -1153,8 +1153,6 @@ static unsigned int cp2112_gpio_irq_startup(struct irq_data *d)
INIT_DELAYED_WORK(&dev->gpio_poll_worker, cp2112_gpio_poll_callback);
- cp2112_gpio_direction_input(gc, d->hwirq);
-
if (!dev->gpio_poll) {
dev->gpio_poll = true;
schedule_delayed_work(&dev->gpio_poll_worker, 0);
@@ -1204,6 +1202,12 @@ static int __maybe_unused cp2112_allocate_irq(struct cp2112_device *dev,
return PTR_ERR(dev->desc[pin]);
}
+ ret = cp2112_gpio_direction_input(&dev->gc, pin);
+ if (ret < 0) {
+ dev_err(dev->gc.parent, "Failed to set GPIO to input dir\n");
+ goto err_desc;
+ }
+
ret = gpiochip_lock_as_irq(&dev->gc, pin);
if (ret) {
dev_err(dev->gc.parent, "Failed to lock GPIO as interrupt\n");
diff --git a/drivers/hid/hid-logitech-hidpp.c b/drivers/hid/hid-logitech-hidpp.c
index 21268c9..0179f7e 100644
--- a/drivers/hid/hid-logitech-hidpp.c
+++ b/drivers/hid/hid-logitech-hidpp.c
@@ -3749,30 +3749,8 @@ static const struct hid_device_id hidpp_devices[] = {
{ L27MHZ_DEVICE(HID_ANY_ID) },
- { /* Logitech G203/Prodigy Gaming Mouse */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC084) },
- { /* Logitech G302 Gaming Mouse */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC07F) },
- { /* Logitech G303 Gaming Mouse */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC080) },
- { /* Logitech G400 Gaming Mouse */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC07E) },
{ /* Logitech G403 Wireless Gaming Mouse over USB */
HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC082) },
- { /* Logitech G403 Gaming Mouse */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC083) },
- { /* Logitech G403 Hero Gaming Mouse over USB */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC08F) },
- { /* Logitech G502 Proteus Core Gaming Mouse */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC07D) },
- { /* Logitech G502 Proteus Spectrum Gaming Mouse over USB */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC332) },
- { /* Logitech G502 Hero Gaming Mouse over USB */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC08B) },
- { /* Logitech G700 Gaming Mouse over USB */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC06B) },
- { /* Logitech G700s Gaming Mouse over USB */
- HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC07C) },
{ /* Logitech G703 Gaming Mouse over USB */
HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, 0xC087) },
{ /* Logitech G703 Hero Gaming Mouse over USB */
diff --git a/drivers/hid/hid-microsoft.c b/drivers/hid/hid-microsoft.c
index 8b3a922..2cf8385 100644
--- a/drivers/hid/hid-microsoft.c
+++ b/drivers/hid/hid-microsoft.c
@@ -303,7 +303,7 @@ static void ms_ff_worker(struct work_struct *work)
r->magnitude[MAGNITUDE_WEAK] = ms->weak; /* right actuator */
ret = hid_hw_output_report(hdev, (__u8 *)r, sizeof(*r));
- if (ret)
+ if (ret < 0)
hid_warn(hdev, "failed to send FF report\n");
}
diff --git a/drivers/hid/intel-ish-hid/ipc/hw-ish.h b/drivers/hid/intel-ish-hid/ipc/hw-ish.h
index 1065692..5792a10 100644
--- a/drivers/hid/intel-ish-hid/ipc/hw-ish.h
+++ b/drivers/hid/intel-ish-hid/ipc/hw-ish.h
@@ -24,6 +24,7 @@
#define ICL_MOBILE_DEVICE_ID 0x34FC
#define SPT_H_DEVICE_ID 0xA135
#define CML_LP_DEVICE_ID 0x02FC
+#define EHL_Ax_DEVICE_ID 0x4BB3
#define REVISION_ID_CHT_A0 0x6
#define REVISION_ID_CHT_Ax_SI 0x0
diff --git a/drivers/hid/intel-ish-hid/ipc/pci-ish.c b/drivers/hid/intel-ish-hid/ipc/pci-ish.c
index aa80b4d..279567b 100644
--- a/drivers/hid/intel-ish-hid/ipc/pci-ish.c
+++ b/drivers/hid/intel-ish-hid/ipc/pci-ish.c
@@ -33,6 +33,7 @@ static const struct pci_device_id ish_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, ICL_MOBILE_DEVICE_ID)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, SPT_H_DEVICE_ID)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, CML_LP_DEVICE_ID)},
+ {PCI_DEVICE(PCI_VENDOR_ID_INTEL, EHL_Ax_DEVICE_ID)},
{0, }
};
MODULE_DEVICE_TABLE(pci, ish_pci_tbl);
diff --git a/drivers/hid/wacom_wac.c b/drivers/hid/wacom_wac.c
index 7a8ddc9..1713235 100644
--- a/drivers/hid/wacom_wac.c
+++ b/drivers/hid/wacom_wac.c
@@ -846,6 +846,8 @@ static int wacom_intuos_general(struct wacom_wac *wacom)
y >>= 1;
distance >>= 1;
}
+ if (features->type == INTUOSHT2)
+ distance = features->distance_max - distance;
input_report_abs(input, ABS_X, x);
input_report_abs(input, ABS_Y, y);
input_report_abs(input, ABS_DISTANCE, distance);
@@ -1059,7 +1061,7 @@ static int wacom_remote_irq(struct wacom_wac *wacom_wac, size_t len)
input_report_key(input, BTN_BASE2, (data[11] & 0x02));
if (data[12] & 0x80)
- input_report_abs(input, ABS_WHEEL, (data[12] & 0x7f));
+ input_report_abs(input, ABS_WHEEL, (data[12] & 0x7f) - 1);
else
input_report_abs(input, ABS_WHEEL, 0);
@@ -1290,7 +1292,8 @@ static void wacom_intuos_pro2_bt_pen(struct wacom_wac *wacom)
}
if (wacom->tool[0]) {
input_report_abs(pen_input, ABS_PRESSURE, get_unaligned_le16(&frame[5]));
- if (wacom->features.type == INTUOSP2_BT) {
+ if (wacom->features.type == INTUOSP2_BT ||
+ wacom->features.type == INTUOSP2S_BT) {
input_report_abs(pen_input, ABS_DISTANCE,
range ? frame[13] : wacom->features.distance_max);
} else {
diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index 5f9505a..23f358c 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -26,7 +26,7 @@
static unsigned long virt_to_hvpfn(void *addr)
{
- unsigned long paddr;
+ phys_addr_t paddr;
if (is_vmalloc_addr(addr))
paddr = page_to_phys(vmalloc_to_page(addr)) +
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 362e70e..fb16a62 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -146,8 +146,6 @@ struct hv_context {
*/
u64 guestid;
- void *tsc_page;
-
struct hv_per_cpu_context __percpu *cpu_context;
/*
diff --git a/drivers/hwtracing/intel_th/pci.c b/drivers/hwtracing/intel_th/pci.c
index c0378c3..91dfeba 100644
--- a/drivers/hwtracing/intel_th/pci.c
+++ b/drivers/hwtracing/intel_th/pci.c
@@ -165,6 +165,11 @@ static const struct pci_device_id intel_th_pci_id_table[] = {
.driver_data = (kernel_ulong_t)0,
},
{
+ /* Lewisburg PCH */
+ PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xa226),
+ .driver_data = (kernel_ulong_t)0,
+ },
+ {
/* Gemini Lake */
PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x318e),
.driver_data = (kernel_ulong_t)&intel_th_2x,
@@ -199,6 +204,11 @@ static const struct pci_device_id intel_th_pci_id_table[] = {
PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x45c5),
.driver_data = (kernel_ulong_t)&intel_th_2x,
},
+ {
+ /* Tiger Lake PCH */
+ PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xa0a6),
+ .driver_data = (kernel_ulong_t)&intel_th_2x,
+ },
{ 0 },
};
diff --git a/drivers/hwtracing/stm/core.c b/drivers/hwtracing/stm/core.c
index e55b902..181e7ff 100644
--- a/drivers/hwtracing/stm/core.c
+++ b/drivers/hwtracing/stm/core.c
@@ -1276,7 +1276,6 @@ int stm_source_register_device(struct device *parent,
err:
put_device(&src->dev);
- kfree(src);
return err;
}
diff --git a/drivers/i2c/busses/i2c-bcm-iproc.c b/drivers/i2c/busses/i2c-bcm-iproc.c
index d7fd76b..19ef2b0 100644
--- a/drivers/i2c/busses/i2c-bcm-iproc.c
+++ b/drivers/i2c/busses/i2c-bcm-iproc.c
@@ -790,7 +790,10 @@ static int bcm_iproc_i2c_xfer(struct i2c_adapter *adapter,
static uint32_t bcm_iproc_i2c_functionality(struct i2c_adapter *adap)
{
- u32 val = I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
+ u32 val;
+
+ /* We do not support the SMBUS Quick command */
+ val = I2C_FUNC_I2C | (I2C_FUNC_SMBUS_EMUL & ~I2C_FUNC_SMBUS_QUICK);
if (adap->algo->reg_slave)
val |= I2C_FUNC_SLAVE;
diff --git a/drivers/i2c/busses/i2c-designware-slave.c b/drivers/i2c/busses/i2c-designware-slave.c
index e7f9305..f5f0017 100644
--- a/drivers/i2c/busses/i2c-designware-slave.c
+++ b/drivers/i2c/busses/i2c-designware-slave.c
@@ -94,6 +94,7 @@ static int i2c_dw_unreg_slave(struct i2c_client *slave)
dev->disable_int(dev);
dev->disable(dev);
+ synchronize_irq(dev->irq);
dev->slave = NULL;
pm_runtime_put(dev->dev);
diff --git a/drivers/i2c/busses/i2c-i801.c b/drivers/i2c/busses/i2c-i801.c
index f295693..2e08b47 100644
--- a/drivers/i2c/busses/i2c-i801.c
+++ b/drivers/i2c/busses/i2c-i801.c
@@ -1194,19 +1194,28 @@ static acpi_status check_acpi_smo88xx_device(acpi_handle obj_handle,
int i;
status = acpi_get_object_info(obj_handle, &info);
- if (!ACPI_SUCCESS(status) || !(info->valid & ACPI_VALID_HID))
+ if (ACPI_FAILURE(status))
return AE_OK;
+ if (!(info->valid & ACPI_VALID_HID))
+ goto smo88xx_not_found;
+
hid = info->hardware_id.string;
if (!hid)
- return AE_OK;
+ goto smo88xx_not_found;
i = match_string(acpi_smo8800_ids, ARRAY_SIZE(acpi_smo8800_ids), hid);
if (i < 0)
- return AE_OK;
+ goto smo88xx_not_found;
+
+ kfree(info);
*((bool *)return_value) = true;
return AE_CTRL_TERMINATE;
+
+smo88xx_not_found:
+ kfree(info);
+ return AE_OK;
}
static bool is_dell_system_with_lis3lv02d(void)
diff --git a/drivers/i2c/busses/i2c-mt65xx.c b/drivers/i2c/busses/i2c-mt65xx.c
index 252edb4..29eae1b 100644
--- a/drivers/i2c/busses/i2c-mt65xx.c
+++ b/drivers/i2c/busses/i2c-mt65xx.c
@@ -234,6 +234,10 @@ static const struct i2c_adapter_quirks mt7622_i2c_quirks = {
.max_num_msgs = 255,
};
+static const struct i2c_adapter_quirks mt8183_i2c_quirks = {
+ .flags = I2C_AQ_NO_ZERO_LEN,
+};
+
static const struct mtk_i2c_compatible mt2712_compat = {
.regs = mt_i2c_regs_v1,
.pmic_i2c = 0,
@@ -298,6 +302,7 @@ static const struct mtk_i2c_compatible mt8173_compat = {
};
static const struct mtk_i2c_compatible mt8183_compat = {
+ .quirks = &mt8183_i2c_quirks,
.regs = mt_i2c_regs_v2,
.pmic_i2c = 0,
.dcm = 0,
@@ -870,7 +875,11 @@ static irqreturn_t mtk_i2c_irq(int irqno, void *dev_id)
static u32 mtk_i2c_functionality(struct i2c_adapter *adap)
{
- return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
+ if (adap->quirks->flags & I2C_AQ_NO_ZERO_LEN)
+ return I2C_FUNC_I2C |
+ (I2C_FUNC_SMBUS_EMUL & ~I2C_FUNC_SMBUS_QUICK);
+ else
+ return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
}
static const struct i2c_algorithm mtk_i2c_algorithm = {
diff --git a/drivers/i2c/busses/i2c-piix4.c b/drivers/i2c/busses/i2c-piix4.c
index c46c4bd..cba325e 100644
--- a/drivers/i2c/busses/i2c-piix4.c
+++ b/drivers/i2c/busses/i2c-piix4.c
@@ -91,7 +91,7 @@
#define SB800_PIIX4_PORT_IDX_MASK 0x06
#define SB800_PIIX4_PORT_IDX_SHIFT 1
-/* On kerncz, SmBus0Sel is at bit 20:19 of PMx00 DecodeEn */
+/* On kerncz and Hudson2, SmBus0Sel is at bit 20:19 of PMx00 DecodeEn */
#define SB800_PIIX4_PORT_IDX_KERNCZ 0x02
#define SB800_PIIX4_PORT_IDX_MASK_KERNCZ 0x18
#define SB800_PIIX4_PORT_IDX_SHIFT_KERNCZ 3
@@ -358,18 +358,16 @@ static int piix4_setup_sb800(struct pci_dev *PIIX4_dev,
/* Find which register is used for port selection */
if (PIIX4_dev->vendor == PCI_VENDOR_ID_AMD ||
PIIX4_dev->vendor == PCI_VENDOR_ID_HYGON) {
- switch (PIIX4_dev->device) {
- case PCI_DEVICE_ID_AMD_KERNCZ_SMBUS:
+ if (PIIX4_dev->device == PCI_DEVICE_ID_AMD_KERNCZ_SMBUS ||
+ (PIIX4_dev->device == PCI_DEVICE_ID_AMD_HUDSON2_SMBUS &&
+ PIIX4_dev->revision >= 0x1F)) {
piix4_port_sel_sb800 = SB800_PIIX4_PORT_IDX_KERNCZ;
piix4_port_mask_sb800 = SB800_PIIX4_PORT_IDX_MASK_KERNCZ;
piix4_port_shift_sb800 = SB800_PIIX4_PORT_IDX_SHIFT_KERNCZ;
- break;
- case PCI_DEVICE_ID_AMD_HUDSON2_SMBUS:
- default:
+ } else {
piix4_port_sel_sb800 = SB800_PIIX4_PORT_IDX_ALT;
piix4_port_mask_sb800 = SB800_PIIX4_PORT_IDX_MASK;
piix4_port_shift_sb800 = SB800_PIIX4_PORT_IDX_SHIFT;
- break;
}
} else {
if (!request_muxed_region(SB800_PIIX4_SMB_IDX, 2,
diff --git a/drivers/i2c/i2c-core-base.c b/drivers/i2c/i2c-core-base.c
index f26ed49..9c440fa 100644
--- a/drivers/i2c/i2c-core-base.c
+++ b/drivers/i2c/i2c-core-base.c
@@ -832,7 +832,7 @@ EXPORT_SYMBOL_GPL(i2c_new_device);
*/
void i2c_unregister_device(struct i2c_client *client)
{
- if (!client)
+ if (IS_ERR_OR_NULL(client))
return;
if (client->dev.of_node) {
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 19f1730..a68d0cc 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -4724,10 +4724,14 @@ static int __init cma_init(void)
if (ret)
goto err;
- cma_configfs_init();
+ ret = cma_configfs_init();
+ if (ret)
+ goto err_ib;
return 0;
+err_ib:
+ ib_unregister_client(&cma_client);
err:
unregister_netdevice_notifier(&cma_nb);
ib_sa_unregister_client(&sa_client);
diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c
index b798907..af8c85d 100644
--- a/drivers/infiniband/core/counters.c
+++ b/drivers/infiniband/core/counters.c
@@ -149,13 +149,11 @@ static bool auto_mode_match(struct ib_qp *qp, struct rdma_counter *counter,
struct auto_mode_param *param = &counter->mode.param;
bool match = true;
- if (rdma_is_kernel_res(&counter->res) != rdma_is_kernel_res(&qp->res))
+ if (!rdma_is_visible_in_pid_ns(&qp->res))
return false;
- /* Ensure that counter belong to right PID */
- if (!rdma_is_kernel_res(&counter->res) &&
- !rdma_is_kernel_res(&qp->res) &&
- (task_pid_vnr(counter->res.task) != current->pid))
+ /* Ensure that counter belongs to the right PID */
+ if (task_pid_nr(counter->res.task) != task_pid_nr(qp->res.task))
return false;
if (auto_mask & RDMA_COUNTER_MASK_QP_TYPE)
@@ -424,7 +422,7 @@ static struct ib_qp *rdma_counter_get_qp(struct ib_device *dev, u32 qp_num)
return qp;
err:
- rdma_restrack_put(&qp->res);
+ rdma_restrack_put(res);
return NULL;
}
diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 87d40d1..020c269 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -382,8 +382,7 @@ static int fill_res_info(struct sk_buff *msg, struct ib_device *device)
for (i = 0; i < RDMA_RESTRACK_MAX; i++) {
if (!names[i])
continue;
- curr = rdma_restrack_count(device, i,
- task_active_pid_ns(current));
+ curr = rdma_restrack_count(device, i);
ret = fill_res_info_entry(msg, names[i], curr);
if (ret)
goto err;
diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c
index bddff42..a07665f 100644
--- a/drivers/infiniband/core/restrack.c
+++ b/drivers/infiniband/core/restrack.c
@@ -107,10 +107,8 @@ void rdma_restrack_clean(struct ib_device *dev)
* rdma_restrack_count() - the current usage of specific object
* @dev: IB device
* @type: actual type of object to operate
- * @ns: PID namespace
*/
-int rdma_restrack_count(struct ib_device *dev, enum rdma_restrack_type type,
- struct pid_namespace *ns)
+int rdma_restrack_count(struct ib_device *dev, enum rdma_restrack_type type)
{
struct rdma_restrack_root *rt = &dev->res[type];
struct rdma_restrack_entry *e;
@@ -119,10 +117,9 @@ int rdma_restrack_count(struct ib_device *dev, enum rdma_restrack_type type,
xa_lock(&rt->xa);
xas_for_each(&xas, e, U32_MAX) {
- if (ns == &init_pid_ns ||
- (!rdma_is_kernel_res(e) &&
- ns == task_active_pid_ns(e->task)))
- cnt++;
+ if (!rdma_is_visible_in_pid_ns(e))
+ continue;
+ cnt++;
}
xa_unlock(&rt->xa);
return cnt;
@@ -360,5 +357,7 @@ bool rdma_is_visible_in_pid_ns(struct rdma_restrack_entry *res)
*/
if (rdma_is_kernel_res(res))
return task_active_pid_ns(current) == &init_pid_ns;
- return task_active_pid_ns(current) == task_active_pid_ns(res->task);
+
+ /* PID 0 means that resource is not found in current namespace */
+ return task_pid_vnr(res->task);
}
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 08da840..5655366 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -379,14 +379,9 @@ EXPORT_SYMBOL(ib_umem_release);
int ib_umem_page_count(struct ib_umem *umem)
{
- int i;
- int n;
+ int i, n = 0;
struct scatterlist *sg;
- if (umem->is_odp)
- return ib_umem_num_pages(umem);
-
- n = 0;
for_each_sg(umem->sg_head.sgl, sg, umem->nmap, i)
n += sg_dma_len(sg) >> PAGE_SHIFT;
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
index 48b04d2..60c8f76 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
+++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c
@@ -136,6 +136,13 @@ static int __send_message(struct bnxt_qplib_rcfw *rcfw, struct cmdq_base *req,
spin_unlock_irqrestore(&cmdq->lock, flags);
return -EBUSY;
}
+
+ size = req->cmd_size;
+ /* change the cmd_size to the number of 16byte cmdq unit.
+ * req->cmd_size is modified here
+ */
+ bnxt_qplib_set_cmd_slots(req);
+
memset(resp, 0, sizeof(*resp));
crsqe->resp = (struct creq_qp_event *)resp;
crsqe->resp->cookie = req->cookie;
@@ -150,7 +157,6 @@ static int __send_message(struct bnxt_qplib_rcfw *rcfw, struct cmdq_base *req,
cmdq_ptr = (struct bnxt_qplib_cmdqe **)cmdq->pbl_ptr;
preq = (u8 *)req;
- size = req->cmd_size * BNXT_QPLIB_CMDQE_UNITS;
do {
/* Locate the next cmdq slot */
sw_prod = HWQ_CMP(cmdq->prod, cmdq);
diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h
index 2138533..dfeadc1 100644
--- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h
+++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h
@@ -55,9 +55,7 @@
do { \
memset(&(req), 0, sizeof((req))); \
(req).opcode = CMDQ_BASE_OPCODE_##CMD; \
- (req).cmd_size = (sizeof((req)) + \
- BNXT_QPLIB_CMDQE_UNITS - 1) / \
- BNXT_QPLIB_CMDQE_UNITS; \
+ (req).cmd_size = sizeof((req)); \
(req).flags = cpu_to_le16(cmd_flags); \
} while (0)
@@ -95,6 +93,13 @@ static inline u32 bnxt_qplib_cmdqe_cnt_per_pg(u32 depth)
BNXT_QPLIB_CMDQE_UNITS);
}
+/* Set the cmd_size to a factor of CMDQE unit */
+static inline void bnxt_qplib_set_cmd_slots(struct cmdq_base *req)
+{
+ req->cmd_size = (req->cmd_size + BNXT_QPLIB_CMDQE_UNITS - 1) /
+ BNXT_QPLIB_CMDQE_UNITS;
+}
+
#define MAX_CMDQ_IDX(depth) ((depth) - 1)
static inline u32 bnxt_qplib_max_cmdq_idx_per_pg(u32 depth)
diff --git a/drivers/infiniband/hw/hfi1/fault.c b/drivers/infiniband/hw/hfi1/fault.c
index 93613e5..986c121 100644
--- a/drivers/infiniband/hw/hfi1/fault.c
+++ b/drivers/infiniband/hw/hfi1/fault.c
@@ -141,12 +141,14 @@ static ssize_t fault_opcodes_write(struct file *file, const char __user *buf,
if (!data)
return -ENOMEM;
copy = min(len, datalen - 1);
- if (copy_from_user(data, buf, copy))
- return -EFAULT;
+ if (copy_from_user(data, buf, copy)) {
+ ret = -EFAULT;
+ goto free_data;
+ }
ret = debugfs_file_get(file->f_path.dentry);
if (unlikely(ret))
- return ret;
+ goto free_data;
ptr = data;
token = ptr;
for (ptr = data; *ptr; ptr = end + 1, token = ptr) {
@@ -195,6 +197,7 @@ static ssize_t fault_opcodes_write(struct file *file, const char __user *buf,
ret = len;
debugfs_file_put(file->f_path.dentry);
+free_data:
kfree(data);
return ret;
}
@@ -214,7 +217,7 @@ static ssize_t fault_opcodes_read(struct file *file, char __user *buf,
return -ENOMEM;
ret = debugfs_file_get(file->f_path.dentry);
if (unlikely(ret))
- return ret;
+ goto free_data;
bit = find_first_bit(fault->opcodes, bitsize);
while (bit < bitsize) {
zero = find_next_zero_bit(fault->opcodes, bitsize, bit);
@@ -232,6 +235,7 @@ static ssize_t fault_opcodes_read(struct file *file, char __user *buf,
data[size - 1] = '\n';
data[size] = '\0';
ret = simple_read_from_buffer(buf, len, pos, data, size);
+free_data:
kfree(data);
return ret;
}
diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c
index 996fc29..6141f4e 100644
--- a/drivers/infiniband/hw/hfi1/tid_rdma.c
+++ b/drivers/infiniband/hw/hfi1/tid_rdma.c
@@ -2574,18 +2574,9 @@ void hfi1_kern_read_tid_flow_free(struct rvt_qp *qp)
hfi1_kern_clear_hw_flow(priv->rcd, qp);
}
-static bool tid_rdma_tid_err(struct hfi1_ctxtdata *rcd,
- struct hfi1_packet *packet, u8 rcv_type,
- u8 opcode)
+static bool tid_rdma_tid_err(struct hfi1_packet *packet, u8 rcv_type)
{
struct rvt_qp *qp = packet->qp;
- struct hfi1_qp_priv *qpriv = qp->priv;
- u32 ipsn;
- struct ib_other_headers *ohdr = packet->ohdr;
- struct rvt_ack_entry *e;
- struct tid_rdma_request *req;
- struct rvt_dev_info *rdi = ib_to_rvt(qp->ibqp.device);
- u32 i;
if (rcv_type >= RHF_RCV_TYPE_IB)
goto done;
@@ -2602,41 +2593,9 @@ static bool tid_rdma_tid_err(struct hfi1_ctxtdata *rcd,
if (rcv_type == RHF_RCV_TYPE_EAGER) {
hfi1_restart_rc(qp, qp->s_last_psn + 1, 1);
hfi1_schedule_send(qp);
- goto done_unlock;
}
- /*
- * For TID READ response, error out QP after freeing the tid
- * resources.
- */
- if (opcode == TID_OP(READ_RESP)) {
- ipsn = mask_psn(be32_to_cpu(ohdr->u.tid_rdma.r_rsp.verbs_psn));
- if (cmp_psn(ipsn, qp->s_last_psn) > 0 &&
- cmp_psn(ipsn, qp->s_psn) < 0) {
- hfi1_kern_read_tid_flow_free(qp);
- spin_unlock(&qp->s_lock);
- rvt_rc_error(qp, IB_WC_LOC_QP_OP_ERR);
- goto done;
- }
- goto done_unlock;
- }
-
- /*
- * Error out the qp for TID RDMA WRITE
- */
- hfi1_kern_clear_hw_flow(qpriv->rcd, qp);
- for (i = 0; i < rvt_max_atomic(rdi); i++) {
- e = &qp->s_ack_queue[i];
- if (e->opcode == TID_OP(WRITE_REQ)) {
- req = ack_to_tid_req(e);
- hfi1_kern_exp_rcv_clear_all(req);
- }
- }
- spin_unlock(&qp->s_lock);
- rvt_rc_error(qp, IB_WC_LOC_LEN_ERR);
- goto done;
-
-done_unlock:
+ /* Since no payload is delivered, just drop the packet */
spin_unlock(&qp->s_lock);
done:
return true;
@@ -2687,12 +2646,12 @@ static bool handle_read_kdeth_eflags(struct hfi1_ctxtdata *rcd,
u32 fpsn;
lockdep_assert_held(&qp->r_lock);
+ spin_lock(&qp->s_lock);
/* If the psn is out of valid range, drop the packet */
if (cmp_psn(ibpsn, qp->s_last_psn) < 0 ||
cmp_psn(ibpsn, qp->s_psn) > 0)
- return ret;
+ goto s_unlock;
- spin_lock(&qp->s_lock);
/*
* Note that NAKs implicitly ACK outstanding SEND and RDMA write
* requests and implicitly NAK RDMA read and atomic requests issued
@@ -2740,9 +2699,12 @@ static bool handle_read_kdeth_eflags(struct hfi1_ctxtdata *rcd,
wqe = do_rc_completion(qp, wqe, ibp);
if (qp->s_acked == qp->s_tail)
- break;
+ goto s_unlock;
}
+ if (qp->s_acked == qp->s_tail)
+ goto s_unlock;
+
/* Handle the eflags for the request */
if (wqe->wr.opcode != IB_WR_TID_RDMA_READ)
goto s_unlock;
@@ -2922,7 +2884,7 @@ bool hfi1_handle_kdeth_eflags(struct hfi1_ctxtdata *rcd,
if (lnh == HFI1_LRH_GRH)
goto r_unlock;
- if (tid_rdma_tid_err(rcd, packet, rcv_type, opcode))
+ if (tid_rdma_tid_err(packet, rcv_type))
goto r_unlock;
}
@@ -2942,8 +2904,15 @@ bool hfi1_handle_kdeth_eflags(struct hfi1_ctxtdata *rcd,
*/
spin_lock(&qp->s_lock);
qpriv = qp->priv;
+ if (qpriv->r_tid_tail == HFI1_QP_WQE_INVALID ||
+ qpriv->r_tid_tail == qpriv->r_tid_head)
+ goto unlock;
e = &qp->s_ack_queue[qpriv->r_tid_tail];
+ if (e->opcode != TID_OP(WRITE_REQ))
+ goto unlock;
req = ack_to_tid_req(e);
+ if (req->comp_seg == req->cur_seg)
+ goto unlock;
flow = &req->flows[req->clear_tail];
trace_hfi1_eflags_err_write(qp, rcv_type, rte, psn);
trace_hfi1_rsp_handle_kdeth_eflags(qp, psn);
@@ -4509,7 +4478,7 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
struct rvt_swqe *wqe;
struct tid_rdma_request *req;
struct tid_rdma_flow *flow;
- u32 aeth, psn, req_psn, ack_psn, resync_psn, ack_kpsn;
+ u32 aeth, psn, req_psn, ack_psn, flpsn, resync_psn, ack_kpsn;
unsigned long flags;
u16 fidx;
@@ -4538,6 +4507,9 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
ack_kpsn--;
}
+ if (unlikely(qp->s_acked == qp->s_tail))
+ goto ack_op_err;
+
wqe = rvt_get_swqe_ptr(qp, qp->s_acked);
if (wqe->wr.opcode != IB_WR_TID_RDMA_WRITE)
@@ -4550,7 +4522,8 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
trace_hfi1_tid_flow_rcv_tid_ack(qp, req->acked_tail, flow);
/* Drop stale ACK/NAK */
- if (cmp_psn(psn, full_flow_psn(flow, flow->flow_state.spsn)) < 0)
+ if (cmp_psn(psn, full_flow_psn(flow, flow->flow_state.spsn)) < 0 ||
+ cmp_psn(req_psn, flow->flow_state.resp_ib_psn) < 0)
goto ack_op_err;
while (cmp_psn(ack_kpsn,
@@ -4712,7 +4685,12 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
switch ((aeth >> IB_AETH_CREDIT_SHIFT) &
IB_AETH_CREDIT_MASK) {
case 0: /* PSN sequence error */
+ if (!req->flows)
+ break;
flow = &req->flows[req->acked_tail];
+ flpsn = full_flow_psn(flow, flow->flow_state.lpsn);
+ if (cmp_psn(psn, flpsn) > 0)
+ break;
trace_hfi1_tid_flow_rcv_tid_ack(qp, req->acked_tail,
flow);
req->r_ack_psn = mask_psn(be32_to_cpu(ohdr->bth[2]));
diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index 68c9514..5707911 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -1677,8 +1677,6 @@ static int mlx4_ib_alloc_pv_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
tx_buf_size, DMA_TO_DEVICE);
kfree(tun_qp->tx_ring[i].buf.addr);
}
- kfree(tun_qp->tx_ring);
- tun_qp->tx_ring = NULL;
i = MLX4_NUM_TUNNEL_BUFS;
err:
while (i > 0) {
@@ -1687,6 +1685,8 @@ static int mlx4_ib_alloc_pv_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
rx_buf_size, DMA_FROM_DEVICE);
kfree(tun_qp->ring[i].addr);
}
+ kfree(tun_qp->tx_ring);
+ tun_qp->tx_ring = NULL;
kfree(tun_qp->ring);
tun_qp->ring = NULL;
return -ENOMEM;
diff --git a/drivers/infiniband/hw/mlx5/cmd.c b/drivers/infiniband/hw/mlx5/cmd.c
index 6c86450..4937947 100644
--- a/drivers/infiniband/hw/mlx5/cmd.c
+++ b/drivers/infiniband/hw/mlx5/cmd.c
@@ -186,136 +186,6 @@ int mlx5_cmd_dealloc_memic(struct mlx5_dm *dm, phys_addr_t addr, u64 length)
return err;
}
-int mlx5_cmd_alloc_sw_icm(struct mlx5_dm *dm, int type, u64 length,
- u16 uid, phys_addr_t *addr, u32 *obj_id)
-{
- struct mlx5_core_dev *dev = dm->dev;
- u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {};
- u32 in[MLX5_ST_SZ_DW(create_sw_icm_in)] = {};
- unsigned long *block_map;
- u64 icm_start_addr;
- u32 log_icm_size;
- u32 num_blocks;
- u32 max_blocks;
- u64 block_idx;
- void *sw_icm;
- int ret;
-
- MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
- MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
- MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_OBJ_TYPE_SW_ICM);
- MLX5_SET(general_obj_in_cmd_hdr, in, uid, uid);
-
- switch (type) {
- case MLX5_IB_UAPI_DM_TYPE_STEERING_SW_ICM:
- icm_start_addr = MLX5_CAP64_DEV_MEM(dev,
- steering_sw_icm_start_address);
- log_icm_size = MLX5_CAP_DEV_MEM(dev, log_steering_sw_icm_size);
- block_map = dm->steering_sw_icm_alloc_blocks;
- break;
- case MLX5_IB_UAPI_DM_TYPE_HEADER_MODIFY_SW_ICM:
- icm_start_addr = MLX5_CAP64_DEV_MEM(dev,
- header_modify_sw_icm_start_address);
- log_icm_size = MLX5_CAP_DEV_MEM(dev,
- log_header_modify_sw_icm_size);
- block_map = dm->header_modify_sw_icm_alloc_blocks;
- break;
- default:
- return -EINVAL;
- }
-
- num_blocks = (length + MLX5_SW_ICM_BLOCK_SIZE(dev) - 1) >>
- MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
- max_blocks = BIT(log_icm_size - MLX5_LOG_SW_ICM_BLOCK_SIZE(dev));
- spin_lock(&dm->lock);
- block_idx = bitmap_find_next_zero_area(block_map,
- max_blocks,
- 0,
- num_blocks, 0);
-
- if (block_idx < max_blocks)
- bitmap_set(block_map,
- block_idx, num_blocks);
-
- spin_unlock(&dm->lock);
-
- if (block_idx >= max_blocks)
- return -ENOMEM;
-
- sw_icm = MLX5_ADDR_OF(create_sw_icm_in, in, sw_icm);
- icm_start_addr += block_idx << MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
- MLX5_SET64(sw_icm, sw_icm, sw_icm_start_addr,
- icm_start_addr);
- MLX5_SET(sw_icm, sw_icm, log_sw_icm_size, ilog2(length));
-
- ret = mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
- if (ret) {
- spin_lock(&dm->lock);
- bitmap_clear(block_map,
- block_idx, num_blocks);
- spin_unlock(&dm->lock);
-
- return ret;
- }
-
- *addr = icm_start_addr;
- *obj_id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
-
- return 0;
-}
-
-int mlx5_cmd_dealloc_sw_icm(struct mlx5_dm *dm, int type, u64 length,
- u16 uid, phys_addr_t addr, u32 obj_id)
-{
- struct mlx5_core_dev *dev = dm->dev;
- u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {};
- u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {};
- unsigned long *block_map;
- u32 num_blocks;
- u64 start_idx;
- int err;
-
- num_blocks = (length + MLX5_SW_ICM_BLOCK_SIZE(dev) - 1) >>
- MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
-
- switch (type) {
- case MLX5_IB_UAPI_DM_TYPE_STEERING_SW_ICM:
- start_idx =
- (addr - MLX5_CAP64_DEV_MEM(
- dev, steering_sw_icm_start_address)) >>
- MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
- block_map = dm->steering_sw_icm_alloc_blocks;
- break;
- case MLX5_IB_UAPI_DM_TYPE_HEADER_MODIFY_SW_ICM:
- start_idx =
- (addr -
- MLX5_CAP64_DEV_MEM(
- dev, header_modify_sw_icm_start_address)) >>
- MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
- block_map = dm->header_modify_sw_icm_alloc_blocks;
- break;
- default:
- return -EINVAL;
- }
-
- MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
- MLX5_CMD_OP_DESTROY_GENERAL_OBJECT);
- MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_OBJ_TYPE_SW_ICM);
- MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, obj_id);
- MLX5_SET(general_obj_in_cmd_hdr, in, uid, uid);
-
- err = mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
- if (err)
- return err;
-
- spin_lock(&dm->lock);
- bitmap_clear(block_map,
- start_idx, num_blocks);
- spin_unlock(&dm->lock);
-
- return 0;
-}
-
int mlx5_cmd_query_ext_ppcnt_counters(struct mlx5_core_dev *dev, void *out)
{
u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
diff --git a/drivers/infiniband/hw/mlx5/cmd.h b/drivers/infiniband/hw/mlx5/cmd.h
index 0572dcb..169cab4 100644
--- a/drivers/infiniband/hw/mlx5/cmd.h
+++ b/drivers/infiniband/hw/mlx5/cmd.h
@@ -65,8 +65,4 @@ int mlx5_cmd_alloc_q_counter(struct mlx5_core_dev *dev, u16 *counter_id,
u16 uid);
int mlx5_cmd_mad_ifc(struct mlx5_core_dev *dev, const void *inb, void *outb,
u16 opmod, u8 port);
-int mlx5_cmd_alloc_sw_icm(struct mlx5_dm *dm, int type, u64 length,
- u16 uid, phys_addr_t *addr, u32 *obj_id);
-int mlx5_cmd_dealloc_sw_icm(struct mlx5_dm *dm, int type, u64 length,
- u16 uid, phys_addr_t addr, u32 obj_id);
#endif /* MLX5_IB_CMD_H */
diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index af5bbb3..25b6482 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -922,6 +922,7 @@ static bool devx_is_general_cmd(void *in, struct mlx5_ib_dev *dev)
case MLX5_CMD_OP_QUERY_CONG_STATUS:
case MLX5_CMD_OP_QUERY_CONG_PARAMS:
case MLX5_CMD_OP_QUERY_CONG_STATISTICS:
+ case MLX5_CMD_OP_QUERY_LAG:
return true;
default:
return false;
diff --git a/drivers/infiniband/hw/mlx5/flow.c b/drivers/infiniband/hw/mlx5/flow.c
index b884135..1c8f04a 100644
--- a/drivers/infiniband/hw/mlx5/flow.c
+++ b/drivers/infiniband/hw/mlx5/flow.c
@@ -322,11 +322,11 @@ void mlx5_ib_destroy_flow_action_raw(struct mlx5_ib_flow_action *maction)
switch (maction->flow_action_raw.sub_type) {
case MLX5_IB_FLOW_ACTION_MODIFY_HEADER:
mlx5_modify_header_dealloc(maction->flow_action_raw.dev->mdev,
- maction->flow_action_raw.action_id);
+ maction->flow_action_raw.modify_hdr);
break;
case MLX5_IB_FLOW_ACTION_PACKET_REFORMAT:
mlx5_packet_reformat_dealloc(maction->flow_action_raw.dev->mdev,
- maction->flow_action_raw.action_id);
+ maction->flow_action_raw.pkt_reformat);
break;
case MLX5_IB_FLOW_ACTION_DECAP:
break;
@@ -352,10 +352,11 @@ mlx5_ib_create_modify_header(struct mlx5_ib_dev *dev,
if (!maction)
return ERR_PTR(-ENOMEM);
- ret = mlx5_modify_header_alloc(dev->mdev, namespace, num_actions, in,
- &maction->flow_action_raw.action_id);
+ maction->flow_action_raw.modify_hdr =
+ mlx5_modify_header_alloc(dev->mdev, namespace, num_actions, in);
- if (ret) {
+ if (IS_ERR(maction->flow_action_raw.modify_hdr)) {
+ ret = PTR_ERR(maction->flow_action_raw.modify_hdr);
kfree(maction);
return ERR_PTR(ret);
}
@@ -479,11 +480,13 @@ static int mlx5_ib_flow_action_create_packet_reformat_ctx(
if (ret)
return ret;
- ret = mlx5_packet_reformat_alloc(dev->mdev, prm_prt, len,
- in, namespace,
- &maction->flow_action_raw.action_id);
- if (ret)
+ maction->flow_action_raw.pkt_reformat =
+ mlx5_packet_reformat_alloc(dev->mdev, prm_prt, len,
+ in, namespace);
+ if (IS_ERR(maction->flow_action_raw.pkt_reformat)) {
+ ret = PTR_ERR(maction->flow_action_raw.pkt_reformat);
return ret;
+ }
maction->flow_action_raw.sub_type =
MLX5_IB_FLOW_ACTION_PACKET_REFORMAT;
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index e12a440..4e9f150 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1023,7 +1023,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
props->timestamp_mask = 0x7FFFFFFFFFFFFFFFULL;
if (IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING)) {
- if (MLX5_CAP_GEN(mdev, pg))
+ if (dev->odp_caps.general_caps & IB_ODP_SUPPORT)
props->device_cap_flags |= IB_DEVICE_ON_DEMAND_PAGING;
props->odp_caps = dev->odp_caps;
}
@@ -2280,6 +2280,7 @@ static inline int check_dm_type_support(struct mlx5_ib_dev *dev,
return -EOPNOTSUPP;
break;
case MLX5_IB_UAPI_DM_TYPE_STEERING_SW_ICM:
+ case MLX5_IB_UAPI_DM_TYPE_HEADER_MODIFY_SW_ICM:
if (!capable(CAP_SYS_RAWIO) ||
!capable(CAP_NET_RAW))
return -EPERM;
@@ -2344,20 +2345,20 @@ static int handle_alloc_dm_sw_icm(struct ib_ucontext *ctx,
struct uverbs_attr_bundle *attrs,
int type)
{
- struct mlx5_dm *dm_db = &to_mdev(ctx->device)->dm;
+ struct mlx5_core_dev *dev = to_mdev(ctx->device)->mdev;
u64 act_size;
int err;
/* Allocation size must a multiple of the basic block size
* and a power of 2.
*/
- act_size = round_up(attr->length, MLX5_SW_ICM_BLOCK_SIZE(dm_db->dev));
+ act_size = round_up(attr->length, MLX5_SW_ICM_BLOCK_SIZE(dev));
act_size = roundup_pow_of_two(act_size);
dm->size = act_size;
- err = mlx5_cmd_alloc_sw_icm(dm_db, type, act_size,
- to_mucontext(ctx)->devx_uid, &dm->dev_addr,
- &dm->icm_dm.obj_id);
+ err = mlx5_dm_sw_icm_alloc(dev, type, act_size,
+ to_mucontext(ctx)->devx_uid, &dm->dev_addr,
+ &dm->icm_dm.obj_id);
if (err)
return err;
@@ -2365,9 +2366,9 @@ static int handle_alloc_dm_sw_icm(struct ib_ucontext *ctx,
MLX5_IB_ATTR_ALLOC_DM_RESP_START_OFFSET,
&dm->dev_addr, sizeof(dm->dev_addr));
if (err)
- mlx5_cmd_dealloc_sw_icm(dm_db, type, dm->size,
- to_mucontext(ctx)->devx_uid,
- dm->dev_addr, dm->icm_dm.obj_id);
+ mlx5_dm_sw_icm_dealloc(dev, type, dm->size,
+ to_mucontext(ctx)->devx_uid, dm->dev_addr,
+ dm->icm_dm.obj_id);
return err;
}
@@ -2407,8 +2408,14 @@ struct ib_dm *mlx5_ib_alloc_dm(struct ib_device *ibdev,
attrs);
break;
case MLX5_IB_UAPI_DM_TYPE_STEERING_SW_ICM:
+ err = handle_alloc_dm_sw_icm(context, dm,
+ attr, attrs,
+ MLX5_SW_ICM_TYPE_STEERING);
+ break;
case MLX5_IB_UAPI_DM_TYPE_HEADER_MODIFY_SW_ICM:
- err = handle_alloc_dm_sw_icm(context, dm, attr, attrs, type);
+ err = handle_alloc_dm_sw_icm(context, dm,
+ attr, attrs,
+ MLX5_SW_ICM_TYPE_HEADER_MODIFY);
break;
default:
err = -EOPNOTSUPP;
@@ -2428,6 +2435,7 @@ int mlx5_ib_dealloc_dm(struct ib_dm *ibdm, struct uverbs_attr_bundle *attrs)
{
struct mlx5_ib_ucontext *ctx = rdma_udata_to_drv_context(
&attrs->driver_udata, struct mlx5_ib_ucontext, ibucontext);
+ struct mlx5_core_dev *dev = to_mdev(ibdm->device)->mdev;
struct mlx5_dm *dm_db = &to_mdev(ibdm->device)->dm;
struct mlx5_ib_dm *dm = to_mdm(ibdm);
u32 page_idx;
@@ -2439,19 +2447,23 @@ int mlx5_ib_dealloc_dm(struct ib_dm *ibdm, struct uverbs_attr_bundle *attrs)
if (ret)
return ret;
- page_idx = (dm->dev_addr -
- pci_resource_start(dm_db->dev->pdev, 0) -
- MLX5_CAP64_DEV_MEM(dm_db->dev,
- memic_bar_start_addr)) >>
- PAGE_SHIFT;
+ page_idx = (dm->dev_addr - pci_resource_start(dev->pdev, 0) -
+ MLX5_CAP64_DEV_MEM(dev, memic_bar_start_addr)) >>
+ PAGE_SHIFT;
bitmap_clear(ctx->dm_pages, page_idx,
DIV_ROUND_UP(dm->size, PAGE_SIZE));
break;
case MLX5_IB_UAPI_DM_TYPE_STEERING_SW_ICM:
+ ret = mlx5_dm_sw_icm_dealloc(dev, MLX5_SW_ICM_TYPE_STEERING,
+ dm->size, ctx->devx_uid, dm->dev_addr,
+ dm->icm_dm.obj_id);
+ if (ret)
+ return ret;
+ break;
case MLX5_IB_UAPI_DM_TYPE_HEADER_MODIFY_SW_ICM:
- ret = mlx5_cmd_dealloc_sw_icm(dm_db, dm->type, dm->size,
- ctx->devx_uid, dm->dev_addr,
- dm->icm_dm.obj_id);
+ ret = mlx5_dm_sw_icm_dealloc(dev, MLX5_SW_ICM_TYPE_HEADER_MODIFY,
+ dm->size, ctx->devx_uid, dm->dev_addr,
+ dm->icm_dm.obj_id);
if (ret)
return ret;
break;
@@ -2646,7 +2658,8 @@ int parse_flow_flow_action(struct mlx5_ib_flow_action *maction,
if (action->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)
return -EINVAL;
action->action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
- action->modify_id = maction->flow_action_raw.action_id;
+ action->modify_hdr =
+ maction->flow_action_raw.modify_hdr;
return 0;
}
if (maction->flow_action_raw.sub_type ==
@@ -2663,8 +2676,8 @@ int parse_flow_flow_action(struct mlx5_ib_flow_action *maction,
return -EINVAL;
action->action |=
MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
- action->reformat_id =
- maction->flow_action_raw.action_id;
+ action->pkt_reformat =
+ maction->flow_action_raw.pkt_reformat;
return 0;
}
/* fall through */
@@ -6096,8 +6109,6 @@ static struct ib_counters *mlx5_ib_create_counters(struct ib_device *device,
static void mlx5_ib_stage_init_cleanup(struct mlx5_ib_dev *dev)
{
- struct mlx5_core_dev *mdev = dev->mdev;
-
mlx5_ib_cleanup_multiport_master(dev);
if (IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING)) {
srcu_barrier(&dev->mr_srcu);
@@ -6105,29 +6116,11 @@ static void mlx5_ib_stage_init_cleanup(struct mlx5_ib_dev *dev)
}
WARN_ON(!bitmap_empty(dev->dm.memic_alloc_pages, MLX5_MAX_MEMIC_PAGES));
-
- WARN_ON(dev->dm.steering_sw_icm_alloc_blocks &&
- !bitmap_empty(
- dev->dm.steering_sw_icm_alloc_blocks,
- BIT(MLX5_CAP_DEV_MEM(mdev, log_steering_sw_icm_size) -
- MLX5_LOG_SW_ICM_BLOCK_SIZE(mdev))));
-
- kfree(dev->dm.steering_sw_icm_alloc_blocks);
-
- WARN_ON(dev->dm.header_modify_sw_icm_alloc_blocks &&
- !bitmap_empty(dev->dm.header_modify_sw_icm_alloc_blocks,
- BIT(MLX5_CAP_DEV_MEM(
- mdev, log_header_modify_sw_icm_size) -
- MLX5_LOG_SW_ICM_BLOCK_SIZE(mdev))));
-
- kfree(dev->dm.header_modify_sw_icm_alloc_blocks);
}
static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev)
{
struct mlx5_core_dev *mdev = dev->mdev;
- u64 header_modify_icm_blocks = 0;
- u64 steering_icm_blocks = 0;
int err;
int i;
@@ -6139,6 +6132,8 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev)
dev->port[i].roce.last_port_state = IB_PORT_DOWN;
}
+ mlx5_ib_internal_fill_odp_caps(dev);
+
err = mlx5_ib_init_multiport_master(dev);
if (err)
return err;
@@ -6172,51 +6167,17 @@ static int mlx5_ib_stage_init_init(struct mlx5_ib_dev *dev)
INIT_LIST_HEAD(&dev->qp_list);
spin_lock_init(&dev->reset_flow_resource_lock);
- if (MLX5_CAP_GEN_64(mdev, general_obj_types) &
- MLX5_GENERAL_OBJ_TYPES_CAP_SW_ICM) {
- if (MLX5_CAP64_DEV_MEM(mdev, steering_sw_icm_start_address)) {
- steering_icm_blocks =
- BIT(MLX5_CAP_DEV_MEM(mdev,
- log_steering_sw_icm_size) -
- MLX5_LOG_SW_ICM_BLOCK_SIZE(mdev));
-
- dev->dm.steering_sw_icm_alloc_blocks =
- kcalloc(BITS_TO_LONGS(steering_icm_blocks),
- sizeof(unsigned long), GFP_KERNEL);
- if (!dev->dm.steering_sw_icm_alloc_blocks)
- goto err_mp;
- }
-
- if (MLX5_CAP64_DEV_MEM(mdev,
- header_modify_sw_icm_start_address)) {
- header_modify_icm_blocks = BIT(
- MLX5_CAP_DEV_MEM(
- mdev, log_header_modify_sw_icm_size) -
- MLX5_LOG_SW_ICM_BLOCK_SIZE(mdev));
-
- dev->dm.header_modify_sw_icm_alloc_blocks =
- kcalloc(BITS_TO_LONGS(header_modify_icm_blocks),
- sizeof(unsigned long), GFP_KERNEL);
- if (!dev->dm.header_modify_sw_icm_alloc_blocks)
- goto err_dm;
- }
- }
-
spin_lock_init(&dev->dm.lock);
dev->dm.dev = mdev;
if (IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING)) {
err = init_srcu_struct(&dev->mr_srcu);
if (err)
- goto err_dm;
+ goto err_mp;
}
return 0;
-err_dm:
- kfree(dev->dm.steering_sw_icm_alloc_blocks);
- kfree(dev->dm.header_modify_sw_icm_alloc_blocks);
-
err_mp:
mlx5_ib_cleanup_multiport_master(dev);
@@ -6563,8 +6524,6 @@ static void mlx5_ib_stage_dev_res_cleanup(struct mlx5_ib_dev *dev)
static int mlx5_ib_stage_odp_init(struct mlx5_ib_dev *dev)
{
- mlx5_ib_internal_fill_odp_caps(dev);
-
return mlx5_ib_odp_init_one(dev);
}
diff --git a/drivers/infiniband/hw/mlx5/mem.c b/drivers/infiniband/hw/mlx5/mem.c
index fe1a76d..a40e0ab 100644
--- a/drivers/infiniband/hw/mlx5/mem.c
+++ b/drivers/infiniband/hw/mlx5/mem.c
@@ -57,9 +57,10 @@ void mlx5_ib_cont_pages(struct ib_umem *umem, u64 addr,
int entry;
if (umem->is_odp) {
- unsigned int page_shift = to_ib_umem_odp(umem)->page_shift;
+ struct ib_umem_odp *odp = to_ib_umem_odp(umem);
+ unsigned int page_shift = odp->page_shift;
- *ncont = ib_umem_page_count(umem);
+ *ncont = ib_umem_odp_num_pages(odp);
*count = *ncont << (page_shift - PAGE_SHIFT);
*shift = page_shift;
if (order)
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index f6a5345..125a507 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -868,7 +868,10 @@ struct mlx5_ib_flow_action {
struct {
struct mlx5_ib_dev *dev;
u32 sub_type;
- u32 action_id;
+ union {
+ struct mlx5_modify_hdr *modify_hdr;
+ struct mlx5_pkt_reformat *pkt_reformat;
+ };
} flow_action_raw;
};
};
@@ -881,8 +884,6 @@ struct mlx5_dm {
*/
spinlock_t lock;
DECLARE_BITMAP(memic_alloc_pages, MLX5_MAX_MEMIC_PAGES);
- unsigned long *steering_sw_icm_alloc_blocks;
- unsigned long *header_modify_sw_icm_alloc_blocks;
};
struct mlx5_read_counters_attr {
@@ -1475,4 +1476,18 @@ int bfregn_to_uar_index(struct mlx5_ib_dev *dev,
bool dyn_bfreg);
int mlx5_ib_qp_set_counter(struct ib_qp *qp, struct rdma_counter *counter);
+
+static inline bool mlx5_ib_can_use_umr(struct mlx5_ib_dev *dev,
+ bool do_modify_atomic)
+{
+ if (MLX5_CAP_GEN(dev->mdev, umr_modify_entity_size_disabled))
+ return false;
+
+ if (do_modify_atomic &&
+ MLX5_CAP_GEN(dev->mdev, atomic) &&
+ MLX5_CAP_GEN(dev->mdev, umr_modify_atomic_disabled))
+ return false;
+
+ return true;
+}
#endif /* MLX5_IB_H */
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index b74fad0..3401f5f 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1293,9 +1293,7 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
if (err < 0)
return ERR_PTR(err);
- use_umr = !MLX5_CAP_GEN(dev->mdev, umr_modify_entity_size_disabled) &&
- (!MLX5_CAP_GEN(dev->mdev, umr_modify_atomic_disabled) ||
- !MLX5_CAP_GEN(dev->mdev, atomic));
+ use_umr = mlx5_ib_can_use_umr(dev, true);
if (order <= mr_cache_max_order(dev) && use_umr) {
mr = alloc_mr_from_cache(pd, umem, virt_addr, length, ncont,
@@ -1448,7 +1446,8 @@ int mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start,
goto err;
}
- if (flags & IB_MR_REREG_TRANS && !use_umr_mtt_update(mr, addr, len)) {
+ if (!mlx5_ib_can_use_umr(dev, true) ||
+ (flags & IB_MR_REREG_TRANS && !use_umr_mtt_update(mr, addr, len))) {
/*
* UMR can't be used - MKey needs to be replaced.
*/
diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index 1d257d1..0a59912 100644
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -301,7 +301,8 @@ void mlx5_ib_internal_fill_odp_caps(struct mlx5_ib_dev *dev)
memset(caps, 0, sizeof(*caps));
- if (!MLX5_CAP_GEN(dev->mdev, pg))
+ if (!MLX5_CAP_GEN(dev->mdev, pg) ||
+ !mlx5_ib_can_use_umr(dev, true))
return;
caps->general_caps = IB_ODP_SUPPORT;
@@ -355,7 +356,8 @@ void mlx5_ib_internal_fill_odp_caps(struct mlx5_ib_dev *dev)
if (MLX5_CAP_GEN(dev->mdev, fixed_buffer_size) &&
MLX5_CAP_GEN(dev->mdev, null_mkey) &&
- MLX5_CAP_GEN(dev->mdev, umr_extended_translation_offset))
+ MLX5_CAP_GEN(dev->mdev, umr_extended_translation_offset) &&
+ !MLX5_CAP_GEN(dev->mdev, umr_indirect_mkey_disabled))
caps->general_caps |= IB_ODP_SUPPORT_IMPLICIT;
return;
@@ -1622,8 +1624,10 @@ int mlx5_ib_odp_init_one(struct mlx5_ib_dev *dev)
{
int ret = 0;
- if (dev->odp_caps.general_caps & IB_ODP_SUPPORT)
- ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_odp_ops);
+ if (!(dev->odp_caps.general_caps & IB_ODP_SUPPORT))
+ return ret;
+
+ ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_odp_ops);
if (dev->odp_caps.general_caps & IB_ODP_SUPPORT_IMPLICIT) {
ret = mlx5_cmd_null_mkey(dev->mdev, &dev->null_mkey);
@@ -1633,9 +1637,6 @@ int mlx5_ib_odp_init_one(struct mlx5_ib_dev *dev)
}
}
- if (!MLX5_CAP_GEN(dev->mdev, pg))
- return ret;
-
ret = mlx5_ib_create_pf_eq(dev, &dev->odp_pf_eq);
return ret;
@@ -1643,7 +1644,7 @@ int mlx5_ib_odp_init_one(struct mlx5_ib_dev *dev)
void mlx5_ib_odp_cleanup_one(struct mlx5_ib_dev *dev)
{
- if (!MLX5_CAP_GEN(dev->mdev, pg))
+ if (!(dev->odp_caps.general_caps & IB_ODP_SUPPORT))
return;
mlx5_ib_destroy_pf_eq(dev, &dev->odp_pf_eq);
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 379328b..72869ff 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -4162,7 +4162,7 @@ static u64 get_xlt_octo(u64 bytes)
MLX5_IB_UMR_OCTOWORD;
}
-static __be64 frwr_mkey_mask(void)
+static __be64 frwr_mkey_mask(bool atomic)
{
u64 result;
@@ -4175,10 +4175,12 @@ static __be64 frwr_mkey_mask(void)
MLX5_MKEY_MASK_LW |
MLX5_MKEY_MASK_RR |
MLX5_MKEY_MASK_RW |
- MLX5_MKEY_MASK_A |
MLX5_MKEY_MASK_SMALL_FENCE |
MLX5_MKEY_MASK_FREE;
+ if (atomic)
+ result |= MLX5_MKEY_MASK_A;
+
return cpu_to_be64(result);
}
@@ -4204,7 +4206,7 @@ static __be64 sig_mkey_mask(void)
}
static void set_reg_umr_seg(struct mlx5_wqe_umr_ctrl_seg *umr,
- struct mlx5_ib_mr *mr, u8 flags)
+ struct mlx5_ib_mr *mr, u8 flags, bool atomic)
{
int size = (mr->ndescs + mr->meta_ndescs) * mr->desc_size;
@@ -4212,7 +4214,7 @@ static void set_reg_umr_seg(struct mlx5_wqe_umr_ctrl_seg *umr,
umr->flags = flags;
umr->xlt_octowords = cpu_to_be16(get_xlt_octo(size));
- umr->mkey_mask = frwr_mkey_mask();
+ umr->mkey_mask = frwr_mkey_mask(atomic);
}
static void set_linv_umr_seg(struct mlx5_wqe_umr_ctrl_seg *umr)
@@ -4811,10 +4813,22 @@ static int set_reg_wr(struct mlx5_ib_qp *qp,
{
struct mlx5_ib_mr *mr = to_mmr(wr->mr);
struct mlx5_ib_pd *pd = to_mpd(qp->ibqp.pd);
+ struct mlx5_ib_dev *dev = to_mdev(pd->ibpd.device);
int mr_list_size = (mr->ndescs + mr->meta_ndescs) * mr->desc_size;
bool umr_inline = mr_list_size <= MLX5_IB_SQ_UMR_INLINE_THRESHOLD;
+ bool atomic = wr->access & IB_ACCESS_REMOTE_ATOMIC;
u8 flags = 0;
+ if (!mlx5_ib_can_use_umr(dev, atomic)) {
+ mlx5_ib_warn(to_mdev(qp->ibqp.device),
+ "Fast update of %s for MR is disabled\n",
+ (MLX5_CAP_GEN(dev->mdev,
+ umr_modify_entity_size_disabled)) ?
+ "entity size" :
+ "atomic access");
+ return -EINVAL;
+ }
+
if (unlikely(wr->wr.send_flags & IB_SEND_INLINE)) {
mlx5_ib_warn(to_mdev(qp->ibqp.device),
"Invalid IB_SEND_INLINE send flag\n");
@@ -4826,7 +4840,7 @@ static int set_reg_wr(struct mlx5_ib_qp *qp,
if (umr_inline)
flags |= MLX5_UMR_INLINE;
- set_reg_umr_seg(*seg, mr, flags);
+ set_reg_umr_seg(*seg, mr, flags, atomic);
*seg += sizeof(struct mlx5_wqe_umr_ctrl_seg);
*size += sizeof(struct mlx5_wqe_umr_ctrl_seg) / 16;
handle_post_send_edge(&qp->sq, seg, *size, cur_edge);
diff --git a/drivers/infiniband/hw/mlx5/srq_cmd.c b/drivers/infiniband/hw/mlx5/srq_cmd.c
index b0d0687..8fc3630 100644
--- a/drivers/infiniband/hw/mlx5/srq_cmd.c
+++ b/drivers/infiniband/hw/mlx5/srq_cmd.c
@@ -86,7 +86,7 @@ struct mlx5_core_srq *mlx5_cmd_get_srq(struct mlx5_ib_dev *dev, u32 srqn)
xa_lock(&table->array);
srq = xa_load(&table->array, srqn);
if (srq)
- atomic_inc(&srq->common.refcount);
+ refcount_inc(&srq->common.refcount);
xa_unlock(&table->array);
return srq;
@@ -592,7 +592,7 @@ int mlx5_cmd_create_srq(struct mlx5_ib_dev *dev, struct mlx5_core_srq *srq,
if (err)
return err;
- atomic_set(&srq->common.refcount, 1);
+ refcount_set(&srq->common.refcount, 1);
init_completion(&srq->common.free);
err = xa_err(xa_store_irq(&table->array, srq->srqn, srq, GFP_KERNEL));
@@ -675,7 +675,7 @@ static int srq_event_notifier(struct notifier_block *nb,
xa_lock(&table->array);
srq = xa_load(&table->array, srqn);
if (srq)
- atomic_inc(&srq->common.refcount);
+ refcount_inc(&srq->common.refcount);
xa_unlock(&table->array);
if (!srq)
diff --git a/drivers/infiniband/sw/siw/siw.h b/drivers/infiniband/sw/siw/siw.h
index 77b1aab..dba4535 100644
--- a/drivers/infiniband/sw/siw/siw.h
+++ b/drivers/infiniband/sw/siw/siw.h
@@ -138,9 +138,9 @@ struct siw_umem {
};
struct siw_pble {
- u64 addr; /* Address of assigned user buffer */
- u64 size; /* Size of this entry */
- u64 pbl_off; /* Total offset from start of PBL */
+ dma_addr_t addr; /* Address of assigned buffer */
+ unsigned int size; /* Size of this entry */
+ unsigned long pbl_off; /* Total offset from start of PBL */
};
struct siw_pbl {
@@ -734,7 +734,7 @@ static inline void siw_crc_skb(struct siw_rx_stream *srx, unsigned int len)
"MEM[0x%08x] %s: " fmt, mem->stag, __func__, ##__VA_ARGS__)
#define siw_dbg_cep(cep, fmt, ...) \
- ibdev_dbg(&cep->sdev->base_dev, "CEP[0x%p] %s: " fmt, \
+ ibdev_dbg(&cep->sdev->base_dev, "CEP[0x%pK] %s: " fmt, \
cep, __func__, ##__VA_ARGS__)
void siw_cq_flush(struct siw_cq *cq);
diff --git a/drivers/infiniband/sw/siw/siw_cm.c b/drivers/infiniband/sw/siw/siw_cm.c
index 9ce8a1b..8c1931a 100644
--- a/drivers/infiniband/sw/siw/siw_cm.c
+++ b/drivers/infiniband/sw/siw/siw_cm.c
@@ -355,8 +355,8 @@ static int siw_cm_upcall(struct siw_cep *cep, enum iw_cm_event_type reason,
getname_local(cep->sock, &event.local_addr);
getname_peer(cep->sock, &event.remote_addr);
}
- siw_dbg_cep(cep, "[QP %u]: id 0x%p, reason=%d, status=%d\n",
- cep->qp ? qp_id(cep->qp) : -1, id, reason, status);
+ siw_dbg_cep(cep, "[QP %u]: reason=%d, status=%d\n",
+ cep->qp ? qp_id(cep->qp) : UINT_MAX, reason, status);
return id->event_handler(id, &event);
}
@@ -947,8 +947,6 @@ static void siw_accept_newconn(struct siw_cep *cep)
siw_cep_get(new_cep);
new_s->sk->sk_user_data = new_cep;
- siw_dbg_cep(cep, "listen socket 0x%p, new 0x%p\n", s, new_s);
-
if (siw_tcp_nagle == false) {
int val = 1;
@@ -1011,7 +1009,8 @@ static void siw_cm_work_handler(struct work_struct *w)
cep = work->cep;
siw_dbg_cep(cep, "[QP %u]: work type: %d, state %d\n",
- cep->qp ? qp_id(cep->qp) : -1, work->type, cep->state);
+ cep->qp ? qp_id(cep->qp) : UINT_MAX,
+ work->type, cep->state);
siw_cep_set_inuse(cep);
@@ -1145,9 +1144,9 @@ static void siw_cm_work_handler(struct work_struct *w)
}
if (release_cep) {
siw_dbg_cep(cep,
- "release: timer=%s, QP[%u], id 0x%p\n",
+ "release: timer=%s, QP[%u]\n",
cep->mpa_timer ? "y" : "n",
- cep->qp ? qp_id(cep->qp) : -1, cep->cm_id);
+ cep->qp ? qp_id(cep->qp) : UINT_MAX);
siw_cancel_mpatimer(cep);
@@ -1211,8 +1210,8 @@ int siw_cm_queue_work(struct siw_cep *cep, enum siw_work_type type)
else
delay = MPAREP_TIMEOUT;
}
- siw_dbg_cep(cep, "[QP %u]: work type: %d, work 0x%p, timeout %lu\n",
- cep->qp ? qp_id(cep->qp) : -1, type, work, delay);
+ siw_dbg_cep(cep, "[QP %u]: work type: %d, timeout %lu\n",
+ cep->qp ? qp_id(cep->qp) : -1, type, delay);
queue_delayed_work(siw_cm_wq, &work->work, delay);
@@ -1376,16 +1375,16 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
}
if (v4)
siw_dbg_qp(qp,
- "id 0x%p, pd_len %d, laddr %pI4 %d, raddr %pI4 %d\n",
- id, pd_len,
+ "pd_len %d, laddr %pI4 %d, raddr %pI4 %d\n",
+ pd_len,
&((struct sockaddr_in *)(laddr))->sin_addr,
ntohs(((struct sockaddr_in *)(laddr))->sin_port),
&((struct sockaddr_in *)(raddr))->sin_addr,
ntohs(((struct sockaddr_in *)(raddr))->sin_port));
else
siw_dbg_qp(qp,
- "id 0x%p, pd_len %d, laddr %pI6 %d, raddr %pI6 %d\n",
- id, pd_len,
+ "pd_len %d, laddr %pI6 %d, raddr %pI6 %d\n",
+ pd_len,
&((struct sockaddr_in6 *)(laddr))->sin6_addr,
ntohs(((struct sockaddr_in6 *)(laddr))->sin6_port),
&((struct sockaddr_in6 *)(raddr))->sin6_addr,
@@ -1508,14 +1507,13 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
if (rv >= 0) {
rv = siw_cm_queue_work(cep, SIW_CM_WORK_MPATIMEOUT);
if (!rv) {
- siw_dbg_cep(cep, "id 0x%p, [QP %u]: exit\n", id,
- qp_id(qp));
+ siw_dbg_cep(cep, "[QP %u]: exit\n", qp_id(qp));
siw_cep_set_free(cep);
return 0;
}
}
error:
- siw_dbg_qp(qp, "failed: %d\n", rv);
+ siw_dbg(id->device, "failed: %d\n", rv);
if (cep) {
siw_socket_disassoc(s);
@@ -1540,7 +1538,8 @@ int siw_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
} else if (s) {
sock_release(s);
}
- siw_qp_put(qp);
+ if (qp)
+ siw_qp_put(qp);
return rv;
}
@@ -1580,7 +1579,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
siw_cancel_mpatimer(cep);
if (cep->state != SIW_EPSTATE_RECVD_MPAREQ) {
- siw_dbg_cep(cep, "id 0x%p: out of state\n", id);
+ siw_dbg_cep(cep, "out of state\n");
siw_cep_set_free(cep);
siw_cep_put(cep);
@@ -1601,7 +1600,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
up_write(&qp->state_lock);
goto error;
}
- siw_dbg_cep(cep, "id 0x%p\n", id);
+ siw_dbg_cep(cep, "[QP %d]\n", params->qpn);
if (try_gso && cep->mpa.hdr.params.bits & MPA_RR_FLAG_GSO_EXP) {
siw_dbg_cep(cep, "peer allows GSO on TX\n");
@@ -1611,8 +1610,8 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
params->ird > sdev->attrs.max_ird) {
siw_dbg_cep(
cep,
- "id 0x%p, [QP %u]: ord %d (max %d), ird %d (max %d)\n",
- id, qp_id(qp), params->ord, sdev->attrs.max_ord,
+ "[QP %u]: ord %d (max %d), ird %d (max %d)\n",
+ qp_id(qp), params->ord, sdev->attrs.max_ord,
params->ird, sdev->attrs.max_ird);
rv = -EINVAL;
up_write(&qp->state_lock);
@@ -1624,8 +1623,8 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
if (params->private_data_len > max_priv_data) {
siw_dbg_cep(
cep,
- "id 0x%p, [QP %u]: private data length: %d (max %d)\n",
- id, qp_id(qp), params->private_data_len, max_priv_data);
+ "[QP %u]: private data length: %d (max %d)\n",
+ qp_id(qp), params->private_data_len, max_priv_data);
rv = -EINVAL;
up_write(&qp->state_lock);
goto error;
@@ -1679,7 +1678,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
qp_attrs.flags = SIW_MPA_CRC;
qp_attrs.state = SIW_QP_STATE_RTS;
- siw_dbg_cep(cep, "id 0x%p, [QP%u]: moving to rts\n", id, qp_id(qp));
+ siw_dbg_cep(cep, "[QP%u]: moving to rts\n", qp_id(qp));
/* Associate QP with CEP */
siw_cep_get(cep);
@@ -1700,8 +1699,8 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
if (rv)
goto error;
- siw_dbg_cep(cep, "id 0x%p, [QP %u]: send mpa reply, %d byte pdata\n",
- id, qp_id(qp), params->private_data_len);
+ siw_dbg_cep(cep, "[QP %u]: send mpa reply, %d byte pdata\n",
+ qp_id(qp), params->private_data_len);
rv = siw_send_mpareqrep(cep, params->private_data,
params->private_data_len);
@@ -1759,14 +1758,14 @@ int siw_reject(struct iw_cm_id *id, const void *pdata, u8 pd_len)
siw_cancel_mpatimer(cep);
if (cep->state != SIW_EPSTATE_RECVD_MPAREQ) {
- siw_dbg_cep(cep, "id 0x%p: out of state\n", id);
+ siw_dbg_cep(cep, "out of state\n");
siw_cep_set_free(cep);
siw_cep_put(cep); /* put last reference */
return -ECONNRESET;
}
- siw_dbg_cep(cep, "id 0x%p, cep->state %d, pd_len %d\n", id, cep->state,
+ siw_dbg_cep(cep, "cep->state %d, pd_len %d\n", cep->state,
pd_len);
if (__mpa_rr_revision(cep->mpa.hdr.params.bits) >= MPA_REVISION_1) {
@@ -1804,14 +1803,14 @@ static int siw_listen_address(struct iw_cm_id *id, int backlog,
rv = kernel_setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (char *)&s_val,
sizeof(s_val));
if (rv) {
- siw_dbg(id->device, "id 0x%p: setsockopt error: %d\n", id, rv);
+ siw_dbg(id->device, "setsockopt error: %d\n", rv);
goto error;
}
rv = s->ops->bind(s, laddr, addr_family == AF_INET ?
sizeof(struct sockaddr_in) :
sizeof(struct sockaddr_in6));
if (rv) {
- siw_dbg(id->device, "id 0x%p: socket bind error: %d\n", id, rv);
+ siw_dbg(id->device, "socket bind error: %d\n", rv);
goto error;
}
cep = siw_cep_alloc(sdev);
@@ -1824,13 +1823,13 @@ static int siw_listen_address(struct iw_cm_id *id, int backlog,
rv = siw_cm_alloc_work(cep, backlog);
if (rv) {
siw_dbg(id->device,
- "id 0x%p: alloc_work error %d, backlog %d\n", id,
+ "alloc_work error %d, backlog %d\n",
rv, backlog);
goto error;
}
rv = s->ops->listen(s, backlog);
if (rv) {
- siw_dbg(id->device, "id 0x%p: listen error %d\n", id, rv);
+ siw_dbg(id->device, "listen error %d\n", rv);
goto error;
}
cep->cm_id = id;
@@ -1914,8 +1913,7 @@ static void siw_drop_listeners(struct iw_cm_id *id)
list_del(p);
- siw_dbg_cep(cep, "id 0x%p: drop cep, state %d\n", id,
- cep->state);
+ siw_dbg_cep(cep, "drop cep, state %d\n", cep->state);
siw_cep_set_inuse(cep);
@@ -1952,7 +1950,7 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)
struct net_device *dev = to_siw_dev(id->device)->netdev;
int rv = 0, listeners = 0;
- siw_dbg(id->device, "id 0x%p: backlog %d\n", id, backlog);
+ siw_dbg(id->device, "backlog %d\n", backlog);
/*
* For each attached address of the interface, create a
@@ -1964,12 +1962,16 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)
struct sockaddr_in s_laddr, *s_raddr;
const struct in_ifaddr *ifa;
+ if (!in_dev) {
+ rv = -ENODEV;
+ goto out;
+ }
memcpy(&s_laddr, &id->local_addr, sizeof(s_laddr));
s_raddr = (struct sockaddr_in *)&id->remote_addr;
siw_dbg(id->device,
- "id 0x%p: laddr %pI4:%d, raddr %pI4:%d\n",
- id, &s_laddr.sin_addr, ntohs(s_laddr.sin_port),
+ "laddr %pI4:%d, raddr %pI4:%d\n",
+ &s_laddr.sin_addr, ntohs(s_laddr.sin_port),
&s_raddr->sin_addr, ntohs(s_raddr->sin_port));
rtnl_lock();
@@ -1993,22 +1995,27 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)
struct sockaddr_in6 *s_laddr = &to_sockaddr_in6(id->local_addr),
*s_raddr = &to_sockaddr_in6(id->remote_addr);
+ if (!in6_dev) {
+ rv = -ENODEV;
+ goto out;
+ }
siw_dbg(id->device,
- "id 0x%p: laddr %pI6:%d, raddr %pI6:%d\n",
- id, &s_laddr->sin6_addr, ntohs(s_laddr->sin6_port),
+ "laddr %pI6:%d, raddr %pI6:%d\n",
+ &s_laddr->sin6_addr, ntohs(s_laddr->sin6_port),
&s_raddr->sin6_addr, ntohs(s_raddr->sin6_port));
- read_lock_bh(&in6_dev->lock);
+ rtnl_lock();
list_for_each_entry(ifp, &in6_dev->addr_list, if_list) {
- struct sockaddr_in6 bind_addr;
-
+ if (ifp->flags & (IFA_F_TENTATIVE | IFA_F_DEPRECATED))
+ continue;
if (ipv6_addr_any(&s_laddr->sin6_addr) ||
ipv6_addr_equal(&s_laddr->sin6_addr, &ifp->addr)) {
- bind_addr.sin6_family = AF_INET6;
- bind_addr.sin6_port = s_laddr->sin6_port;
- bind_addr.sin6_flowinfo = 0;
- bind_addr.sin6_addr = ifp->addr;
- bind_addr.sin6_scope_id = dev->ifindex;
+ struct sockaddr_in6 bind_addr = {
+ .sin6_family = AF_INET6,
+ .sin6_port = s_laddr->sin6_port,
+ .sin6_flowinfo = 0,
+ .sin6_addr = ifp->addr,
+ .sin6_scope_id = dev->ifindex };
rv = siw_listen_address(id, backlog,
(struct sockaddr *)&bind_addr,
@@ -2017,28 +2024,26 @@ int siw_create_listen(struct iw_cm_id *id, int backlog)
listeners++;
}
}
- read_unlock_bh(&in6_dev->lock);
-
+ rtnl_unlock();
in6_dev_put(in6_dev);
} else {
- return -EAFNOSUPPORT;
+ rv = -EAFNOSUPPORT;
}
+out:
if (listeners)
rv = 0;
else if (!rv)
rv = -EINVAL;
- siw_dbg(id->device, "id 0x%p: %s\n", id, rv ? "FAIL" : "OK");
+ siw_dbg(id->device, "%s\n", rv ? "FAIL" : "OK");
return rv;
}
int siw_destroy_listen(struct iw_cm_id *id)
{
- siw_dbg(id->device, "id 0x%p\n", id);
-
if (!id->provider_data) {
- siw_dbg(id->device, "id 0x%p: no cep(s)\n", id);
+ siw_dbg(id->device, "no cep(s)\n");
return 0;
}
siw_drop_listeners(id);
diff --git a/drivers/infiniband/sw/siw/siw_cq.c b/drivers/infiniband/sw/siw/siw_cq.c
index e381ae9..d8db3be 100644
--- a/drivers/infiniband/sw/siw/siw_cq.c
+++ b/drivers/infiniband/sw/siw/siw_cq.c
@@ -71,9 +71,10 @@ int siw_reap_cqe(struct siw_cq *cq, struct ib_wc *wc)
wc->wc_flags = IB_WC_WITH_INVALIDATE;
}
wc->qp = cqe->base_qp;
- siw_dbg_cq(cq, "idx %u, type %d, flags %2x, id 0x%p\n",
+ siw_dbg_cq(cq,
+ "idx %u, type %d, flags %2x, id 0x%pK\n",
cq->cq_get % cq->num_cqe, cqe->opcode,
- cqe->flags, (void *)cqe->id);
+ cqe->flags, (void *)(uintptr_t)cqe->id);
}
WRITE_ONCE(cqe->flags, 0);
cq->cq_get++;
diff --git a/drivers/infiniband/sw/siw/siw_mem.c b/drivers/infiniband/sw/siw/siw_mem.c
index 67171c8..87a5603 100644
--- a/drivers/infiniband/sw/siw/siw_mem.c
+++ b/drivers/infiniband/sw/siw/siw_mem.c
@@ -197,12 +197,12 @@ int siw_check_mem(struct ib_pd *pd, struct siw_mem *mem, u64 addr,
*/
if (addr < mem->va || addr + len > mem->va + mem->len) {
siw_dbg_pd(pd, "MEM interval len %d\n", len);
- siw_dbg_pd(pd, "[0x%016llx, 0x%016llx] out of bounds\n",
- (unsigned long long)addr,
- (unsigned long long)(addr + len));
- siw_dbg_pd(pd, "[0x%016llx, 0x%016llx] STag=0x%08x\n",
- (unsigned long long)mem->va,
- (unsigned long long)(mem->va + mem->len),
+ siw_dbg_pd(pd, "[0x%pK, 0x%pK] out of bounds\n",
+ (void *)(uintptr_t)addr,
+ (void *)(uintptr_t)(addr + len));
+ siw_dbg_pd(pd, "[0x%pK, 0x%pK] STag=0x%08x\n",
+ (void *)(uintptr_t)mem->va,
+ (void *)(uintptr_t)(mem->va + mem->len),
mem->stag);
return -E_BASE_BOUNDS;
@@ -330,7 +330,7 @@ int siw_invalidate_stag(struct ib_pd *pd, u32 stag)
* Optionally, provides remaining len within current element, and
* current PBL index for later resume at same element.
*/
-u64 siw_pbl_get_buffer(struct siw_pbl *pbl, u64 off, int *len, int *idx)
+dma_addr_t siw_pbl_get_buffer(struct siw_pbl *pbl, u64 off, int *len, int *idx)
{
int i = idx ? *idx : 0;
diff --git a/drivers/infiniband/sw/siw/siw_mem.h b/drivers/infiniband/sw/siw/siw_mem.h
index f43daf2..db138c8 100644
--- a/drivers/infiniband/sw/siw/siw_mem.h
+++ b/drivers/infiniband/sw/siw/siw_mem.h
@@ -9,7 +9,7 @@
struct siw_umem *siw_umem_get(u64 start, u64 len, bool writable);
void siw_umem_release(struct siw_umem *umem, bool dirty);
struct siw_pbl *siw_pbl_alloc(u32 num_buf);
-u64 siw_pbl_get_buffer(struct siw_pbl *pbl, u64 off, int *len, int *idx);
+dma_addr_t siw_pbl_get_buffer(struct siw_pbl *pbl, u64 off, int *len, int *idx);
struct siw_mem *siw_mem_id2obj(struct siw_device *sdev, int stag_index);
int siw_mem_add(struct siw_device *sdev, struct siw_mem *m);
int siw_invalidate_stag(struct ib_pd *pd, u32 stag);
diff --git a/drivers/infiniband/sw/siw/siw_qp.c b/drivers/infiniband/sw/siw/siw_qp.c
index 0990307..430314c 100644
--- a/drivers/infiniband/sw/siw/siw_qp.c
+++ b/drivers/infiniband/sw/siw/siw_qp.c
@@ -949,7 +949,7 @@ int siw_activate_tx(struct siw_qp *qp)
rv = -EINVAL;
goto out;
}
- wqe->sqe.sge[0].laddr = (u64)&wqe->sqe.sge[1];
+ wqe->sqe.sge[0].laddr = (uintptr_t)&wqe->sqe.sge[1];
wqe->sqe.sge[0].lkey = 0;
wqe->sqe.num_sge = 1;
}
diff --git a/drivers/infiniband/sw/siw/siw_qp_rx.c b/drivers/infiniband/sw/siw/siw_qp_rx.c
index f87657a..c0a8872 100644
--- a/drivers/infiniband/sw/siw/siw_qp_rx.c
+++ b/drivers/infiniband/sw/siw/siw_qp_rx.c
@@ -38,9 +38,10 @@ static int siw_rx_umem(struct siw_rx_stream *srx, struct siw_umem *umem,
p = siw_get_upage(umem, dest_addr);
if (unlikely(!p)) {
- pr_warn("siw: %s: [QP %u]: bogus addr: %p, %p\n",
+ pr_warn("siw: %s: [QP %u]: bogus addr: %pK, %pK\n",
__func__, qp_id(rx_qp(srx)),
- (void *)dest_addr, (void *)umem->fp_addr);
+ (void *)(uintptr_t)dest_addr,
+ (void *)(uintptr_t)umem->fp_addr);
/* siw internal error */
srx->skb_copied += copied;
srx->skb_new -= copied;
@@ -50,7 +51,7 @@ static int siw_rx_umem(struct siw_rx_stream *srx, struct siw_umem *umem,
pg_off = dest_addr & ~PAGE_MASK;
bytes = min(len, (int)PAGE_SIZE - pg_off);
- siw_dbg_qp(rx_qp(srx), "page %p, bytes=%u\n", p, bytes);
+ siw_dbg_qp(rx_qp(srx), "page %pK, bytes=%u\n", p, bytes);
dest = kmap_atomic(p);
rv = skb_copy_bits(srx->skb, srx->skb_offset, dest + pg_off,
@@ -104,11 +105,11 @@ static int siw_rx_kva(struct siw_rx_stream *srx, void *kva, int len)
{
int rv;
- siw_dbg_qp(rx_qp(srx), "kva: 0x%p, len: %u\n", kva, len);
+ siw_dbg_qp(rx_qp(srx), "kva: 0x%pK, len: %u\n", kva, len);
rv = skb_copy_bits(srx->skb, srx->skb_offset, kva, len);
if (unlikely(rv)) {
- pr_warn("siw: [QP %u]: %s, len %d, kva 0x%p, rv %d\n",
+ pr_warn("siw: [QP %u]: %s, len %d, kva 0x%pK, rv %d\n",
qp_id(rx_qp(srx)), __func__, len, kva, rv);
return rv;
@@ -132,7 +133,7 @@ static int siw_rx_pbl(struct siw_rx_stream *srx, int *pbl_idx,
while (len) {
int bytes;
- u64 buf_addr =
+ dma_addr_t buf_addr =
siw_pbl_get_buffer(pbl, offset, &bytes, pbl_idx);
if (!buf_addr)
break;
@@ -485,8 +486,8 @@ int siw_proc_send(struct siw_qp *qp)
mem_p = *mem;
if (mem_p->mem_obj == NULL)
rv = siw_rx_kva(srx,
- (void *)(sge->laddr + frx->sge_off),
- sge_bytes);
+ (void *)(uintptr_t)(sge->laddr + frx->sge_off),
+ sge_bytes);
else if (!mem_p->is_pbl)
rv = siw_rx_umem(srx, mem_p->umem,
sge->laddr + frx->sge_off, sge_bytes);
@@ -598,8 +599,8 @@ int siw_proc_write(struct siw_qp *qp)
if (mem->mem_obj == NULL)
rv = siw_rx_kva(srx,
- (void *)(srx->ddp_to + srx->fpdu_part_rcvd),
- bytes);
+ (void *)(uintptr_t)(srx->ddp_to + srx->fpdu_part_rcvd),
+ bytes);
else if (!mem->is_pbl)
rv = siw_rx_umem(srx, mem->umem,
srx->ddp_to + srx->fpdu_part_rcvd, bytes);
@@ -841,8 +842,9 @@ int siw_proc_rresp(struct siw_qp *qp)
bytes = min(srx->fpdu_part_rem, srx->skb_new);
if (mem_p->mem_obj == NULL)
- rv = siw_rx_kva(srx, (void *)(sge->laddr + wqe->processed),
- bytes);
+ rv = siw_rx_kva(srx,
+ (void *)(uintptr_t)(sge->laddr + wqe->processed),
+ bytes);
else if (!mem_p->is_pbl)
rv = siw_rx_umem(srx, mem_p->umem, sge->laddr + wqe->processed,
bytes);
diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c
index 43020d2..438a291 100644
--- a/drivers/infiniband/sw/siw/siw_qp_tx.c
+++ b/drivers/infiniband/sw/siw/siw_qp_tx.c
@@ -26,7 +26,7 @@ static struct page *siw_get_pblpage(struct siw_mem *mem, u64 addr, int *idx)
{
struct siw_pbl *pbl = mem->pbl;
u64 offset = addr - mem->va;
- u64 paddr = siw_pbl_get_buffer(pbl, offset, NULL, idx);
+ dma_addr_t paddr = siw_pbl_get_buffer(pbl, offset, NULL, idx);
if (paddr)
return virt_to_page(paddr);
@@ -37,7 +37,7 @@ static struct page *siw_get_pblpage(struct siw_mem *mem, u64 addr, int *idx)
/*
* Copy short payload at provided destination payload address
*/
-static int siw_try_1seg(struct siw_iwarp_tx *c_tx, u64 paddr)
+static int siw_try_1seg(struct siw_iwarp_tx *c_tx, void *paddr)
{
struct siw_wqe *wqe = &c_tx->wqe_active;
struct siw_sge *sge = &wqe->sqe.sge[0];
@@ -50,16 +50,16 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, u64 paddr)
return 0;
if (tx_flags(wqe) & SIW_WQE_INLINE) {
- memcpy((void *)paddr, &wqe->sqe.sge[1], bytes);
+ memcpy(paddr, &wqe->sqe.sge[1], bytes);
} else {
struct siw_mem *mem = wqe->mem[0];
if (!mem->mem_obj) {
/* Kernel client using kva */
- memcpy((void *)paddr, (void *)sge->laddr, bytes);
+ memcpy(paddr,
+ (const void *)(uintptr_t)sge->laddr, bytes);
} else if (c_tx->in_syscall) {
- if (copy_from_user((void *)paddr,
- (const void __user *)sge->laddr,
+ if (copy_from_user(paddr, u64_to_user_ptr(sge->laddr),
bytes))
return -EFAULT;
} else {
@@ -79,12 +79,12 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, u64 paddr)
buffer = kmap_atomic(p);
if (likely(PAGE_SIZE - off >= bytes)) {
- memcpy((void *)paddr, buffer + off, bytes);
+ memcpy(paddr, buffer + off, bytes);
kunmap_atomic(buffer);
} else {
unsigned long part = bytes - (PAGE_SIZE - off);
- memcpy((void *)paddr, buffer + off, part);
+ memcpy(paddr, buffer + off, part);
kunmap_atomic(buffer);
if (!mem->is_pbl)
@@ -98,7 +98,7 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, u64 paddr)
return -EFAULT;
buffer = kmap_atomic(p);
- memcpy((void *)(paddr + part), buffer,
+ memcpy(paddr + part, buffer,
bytes - part);
kunmap_atomic(buffer);
}
@@ -166,7 +166,7 @@ static int siw_qp_prepare_tx(struct siw_iwarp_tx *c_tx)
c_tx->ctrl_len = sizeof(struct iwarp_send);
crc = (char *)&c_tx->pkt.send_pkt.crc;
- data = siw_try_1seg(c_tx, (u64)crc);
+ data = siw_try_1seg(c_tx, crc);
break;
case SIW_OP_SEND_REMOTE_INV:
@@ -189,7 +189,7 @@ static int siw_qp_prepare_tx(struct siw_iwarp_tx *c_tx)
c_tx->ctrl_len = sizeof(struct iwarp_send_inv);
crc = (char *)&c_tx->pkt.send_pkt.crc;
- data = siw_try_1seg(c_tx, (u64)crc);
+ data = siw_try_1seg(c_tx, crc);
break;
case SIW_OP_WRITE:
@@ -201,7 +201,7 @@ static int siw_qp_prepare_tx(struct siw_iwarp_tx *c_tx)
c_tx->ctrl_len = sizeof(struct iwarp_rdma_write);
crc = (char *)&c_tx->pkt.write_pkt.crc;
- data = siw_try_1seg(c_tx, (u64)crc);
+ data = siw_try_1seg(c_tx, crc);
break;
case SIW_OP_READ_RESPONSE:
@@ -216,7 +216,7 @@ static int siw_qp_prepare_tx(struct siw_iwarp_tx *c_tx)
c_tx->ctrl_len = sizeof(struct iwarp_rdma_rresp);
crc = (char *)&c_tx->pkt.write_pkt.crc;
- data = siw_try_1seg(c_tx, (u64)crc);
+ data = siw_try_1seg(c_tx, crc);
break;
default:
@@ -398,15 +398,13 @@ static int siw_0copy_tx(struct socket *s, struct page **page,
#define MAX_TRAILER (MPA_CRC_SIZE + 4)
-static void siw_unmap_pages(struct page **pages, int hdr_len, int num_maps)
+static void siw_unmap_pages(struct page **pp, unsigned long kmap_mask)
{
- if (hdr_len) {
- ++pages;
- --num_maps;
- }
- while (num_maps-- > 0) {
- kunmap(*pages);
- pages++;
+ while (kmap_mask) {
+ if (kmap_mask & BIT(0))
+ kunmap(*pp);
+ pp++;
+ kmap_mask >>= 1;
}
}
@@ -437,6 +435,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
unsigned int data_len = c_tx->bytes_unsent, hdr_len = 0, trl_len = 0,
sge_off = c_tx->sge_off, sge_idx = c_tx->sge_idx,
pbl_idx = c_tx->pbl_idx;
+ unsigned long kmap_mask = 0L;
if (c_tx->state == SIW_SEND_HDR) {
if (c_tx->use_sendpage) {
@@ -463,8 +462,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
if (!(tx_flags(wqe) & SIW_WQE_INLINE)) {
mem = wqe->mem[sge_idx];
- if (!mem->mem_obj)
- is_kva = 1;
+ is_kva = mem->mem_obj == NULL ? 1 : 0;
} else {
is_kva = 1;
}
@@ -473,7 +471,8 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
* tx from kernel virtual address: either inline data
* or memory region with assigned kernel buffer
*/
- iov[seg].iov_base = (void *)(sge->laddr + sge_off);
+ iov[seg].iov_base =
+ (void *)(uintptr_t)(sge->laddr + sge_off);
iov[seg].iov_len = sge_len;
if (do_crc)
@@ -500,12 +499,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
p = siw_get_upage(mem->umem,
sge->laddr + sge_off);
if (unlikely(!p)) {
- if (hdr_len)
- seg--;
- if (!c_tx->use_sendpage && seg) {
- siw_unmap_pages(page_array,
- hdr_len, seg);
- }
+ siw_unmap_pages(page_array, kmap_mask);
wqe->processed -= c_tx->bytes_unsent;
rv = -EFAULT;
goto done_crc;
@@ -515,6 +509,10 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
if (!c_tx->use_sendpage) {
iov[seg].iov_base = kmap(p) + fp_off;
iov[seg].iov_len = plen;
+
+ /* Remember for later kunmap() */
+ kmap_mask |= BIT(seg);
+
if (do_crc)
crypto_shash_update(
c_tx->mpa_crc_hd,
@@ -526,13 +524,13 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
page_address(p) + fp_off,
plen);
} else {
- u64 pa = ((sge->laddr + sge_off) & PAGE_MASK);
+ u64 va = sge->laddr + sge_off;
- page_array[seg] = virt_to_page(pa);
+ page_array[seg] = virt_to_page(va & PAGE_MASK);
if (do_crc)
crypto_shash_update(
c_tx->mpa_crc_hd,
- (void *)(sge->laddr + sge_off),
+ (void *)(uintptr_t)va,
plen);
}
@@ -543,10 +541,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
if (++seg > (int)MAX_ARRAY) {
siw_dbg_qp(tx_qp(c_tx), "to many fragments\n");
- if (!is_kva && !c_tx->use_sendpage) {
- siw_unmap_pages(page_array, hdr_len,
- seg - 1);
- }
+ siw_unmap_pages(page_array, kmap_mask);
wqe->processed -= c_tx->bytes_unsent;
rv = -EMSGSIZE;
goto done_crc;
@@ -597,8 +592,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
} else {
rv = kernel_sendmsg(s, &msg, iov, seg + 1,
hdr_len + data_len + trl_len);
- if (!is_kva)
- siw_unmap_pages(page_array, hdr_len, seg);
+ siw_unmap_pages(page_array, kmap_mask);
}
if (rv < (int)hdr_len) {
/* Not even complete hdr pushed or negative rv */
@@ -829,7 +823,8 @@ static int siw_qp_sq_proc_tx(struct siw_qp *qp, struct siw_wqe *wqe)
rv = -EINVAL;
goto tx_error;
}
- wqe->sqe.sge[0].laddr = (u64)&wqe->sqe.sge[1];
+ wqe->sqe.sge[0].laddr =
+ (u64)(uintptr_t)&wqe->sqe.sge[1];
}
}
wqe->wr_status = SIW_WR_INPROGRESS;
@@ -924,7 +919,7 @@ static int siw_qp_sq_proc_tx(struct siw_qp *qp, struct siw_wqe *wqe)
static int siw_fastreg_mr(struct ib_pd *pd, struct siw_sqe *sqe)
{
- struct ib_mr *base_mr = (struct ib_mr *)sqe->base_mr;
+ struct ib_mr *base_mr = (struct ib_mr *)(uintptr_t)sqe->base_mr;
struct siw_device *sdev = to_siw_dev(pd->device);
struct siw_mem *mem = siw_mem_id2obj(sdev, sqe->rkey >> 8);
int rv = 0;
@@ -954,8 +949,7 @@ static int siw_fastreg_mr(struct ib_pd *pd, struct siw_sqe *sqe)
mem->stag = sqe->rkey;
mem->perms = sqe->access;
- siw_dbg_mem(mem, "STag now valid, MR va: 0x%016llx -> 0x%016llx\n",
- mem->va, base_mr->iova);
+ siw_dbg_mem(mem, "STag 0x%08x now valid\n", sqe->rkey);
mem->va = base_mr->iova;
mem->stag_valid = 1;
out:
diff --git a/drivers/infiniband/sw/siw/siw_verbs.c b/drivers/infiniband/sw/siw/siw_verbs.c
index e7f3a23..da52c90 100644
--- a/drivers/infiniband/sw/siw/siw_verbs.c
+++ b/drivers/infiniband/sw/siw/siw_verbs.c
@@ -424,8 +424,7 @@ struct ib_qp *siw_create_qp(struct ib_pd *pd,
*/
qp->srq = to_siw_srq(attrs->srq);
qp->attrs.rq_size = 0;
- siw_dbg(base_dev, "QP [%u]: [SRQ 0x%p] attached\n",
- qp->qp_num, qp->srq);
+ siw_dbg(base_dev, "QP [%u]: SRQ attached\n", qp->qp_num);
} else if (num_rqe) {
if (qp->kernel_verbs)
qp->recvq = vzalloc(num_rqe * sizeof(struct siw_rqe));
@@ -610,7 +609,7 @@ int siw_destroy_qp(struct ib_qp *base_qp, struct ib_udata *udata)
base_ucontext);
struct siw_qp_attrs qp_attrs;
- siw_dbg_qp(qp, "state %d, cep 0x%p\n", qp->attrs.state, qp->cep);
+ siw_dbg_qp(qp, "state %d\n", qp->attrs.state);
/*
* Mark QP as in process of destruction to prevent from
@@ -662,7 +661,7 @@ static int siw_copy_inline_sgl(const struct ib_send_wr *core_wr,
void *kbuf = &sqe->sge[1];
int num_sge = core_wr->num_sge, bytes = 0;
- sqe->sge[0].laddr = (u64)kbuf;
+ sqe->sge[0].laddr = (uintptr_t)kbuf;
sqe->sge[0].lkey = 0;
while (num_sge--) {
@@ -825,7 +824,7 @@ int siw_post_send(struct ib_qp *base_qp, const struct ib_send_wr *wr,
break;
case IB_WR_REG_MR:
- sqe->base_mr = (uint64_t)reg_wr(wr)->mr;
+ sqe->base_mr = (uintptr_t)reg_wr(wr)->mr;
sqe->rkey = reg_wr(wr)->key;
sqe->access = reg_wr(wr)->access & IWARP_ACCESS_MASK;
sqe->opcode = SIW_OP_REG_MR;
@@ -842,8 +841,9 @@ int siw_post_send(struct ib_qp *base_qp, const struct ib_send_wr *wr,
rv = -EINVAL;
break;
}
- siw_dbg_qp(qp, "opcode %d, flags 0x%x, wr_id 0x%p\n",
- sqe->opcode, sqe->flags, (void *)sqe->id);
+ siw_dbg_qp(qp, "opcode %d, flags 0x%x, wr_id 0x%pK\n",
+ sqe->opcode, sqe->flags,
+ (void *)(uintptr_t)sqe->id);
if (unlikely(rv < 0))
break;
@@ -1205,8 +1205,8 @@ struct ib_mr *siw_reg_user_mr(struct ib_pd *pd, u64 start, u64 len,
unsigned long mem_limit = rlimit(RLIMIT_MEMLOCK);
int rv;
- siw_dbg_pd(pd, "start: 0x%016llx, va: 0x%016llx, len: %llu\n",
- (unsigned long long)start, (unsigned long long)rnic_va,
+ siw_dbg_pd(pd, "start: 0x%pK, va: 0x%pK, len: %llu\n",
+ (void *)(uintptr_t)start, (void *)(uintptr_t)rnic_va,
(unsigned long long)len);
if (atomic_inc_return(&sdev->num_mr) > SIW_MAX_MR) {
@@ -1363,7 +1363,7 @@ int siw_map_mr_sg(struct ib_mr *base_mr, struct scatterlist *sl, int num_sle,
struct siw_mem *mem = mr->mem;
struct siw_pbl *pbl = mem->pbl;
struct siw_pble *pble;
- u64 pbl_size;
+ unsigned long pbl_size;
int i, rv;
if (!pbl) {
@@ -1402,16 +1402,18 @@ int siw_map_mr_sg(struct ib_mr *base_mr, struct scatterlist *sl, int num_sle,
pbl_size += sg_dma_len(slp);
}
siw_dbg_mem(mem,
- "sge[%d], size %llu, addr 0x%016llx, total %llu\n",
- i, pble->size, pble->addr, pbl_size);
+ "sge[%d], size %u, addr 0x%p, total %lu\n",
+ i, pble->size, (void *)(uintptr_t)pble->addr,
+ pbl_size);
}
rv = ib_sg_to_pages(base_mr, sl, num_sle, sg_off, siw_set_pbl_page);
if (rv > 0) {
mem->len = base_mr->length;
mem->va = base_mr->iova;
siw_dbg_mem(mem,
- "%llu bytes, start 0x%016llx, %u SLE to %u entries\n",
- mem->len, mem->va, num_sle, pbl->num_buf);
+ "%llu bytes, start 0x%pK, %u SLE to %u entries\n",
+ mem->len, (void *)(uintptr_t)mem->va, num_sle,
+ pbl->num_buf);
}
return rv;
}
@@ -1529,7 +1531,7 @@ int siw_create_srq(struct ib_srq *base_srq,
}
spin_lock_init(&srq->lock);
- siw_dbg_pd(base_srq->pd, "[SRQ 0x%p]: success\n", srq);
+ siw_dbg_pd(base_srq->pd, "[SRQ]: success\n");
return 0;
@@ -1650,8 +1652,7 @@ int siw_post_srq_recv(struct ib_srq *base_srq, const struct ib_recv_wr *wr,
if (unlikely(!srq->kernel_verbs)) {
siw_dbg_pd(base_srq->pd,
- "[SRQ 0x%p]: no kernel post_recv for mapped srq\n",
- srq);
+ "[SRQ]: no kernel post_recv for mapped srq\n");
rv = -EINVAL;
goto out;
}
@@ -1673,8 +1674,7 @@ int siw_post_srq_recv(struct ib_srq *base_srq, const struct ib_recv_wr *wr,
}
if (unlikely(wr->num_sge > srq->max_sge)) {
siw_dbg_pd(base_srq->pd,
- "[SRQ 0x%p]: too many sge's: %d\n", srq,
- wr->num_sge);
+ "[SRQ]: too many sge's: %d\n", wr->num_sge);
rv = -EINVAL;
break;
}
@@ -1693,7 +1693,7 @@ int siw_post_srq_recv(struct ib_srq *base_srq, const struct ib_recv_wr *wr,
spin_unlock_irqrestore(&srq->lock, flags);
out:
if (unlikely(rv < 0)) {
- siw_dbg_pd(base_srq->pd, "[SRQ 0x%p]: error %d\n", srq, rv);
+ siw_dbg_pd(base_srq->pd, "[SRQ]: error %d\n", rv);
*bad_wr = wr;
}
return rv;
diff --git a/drivers/input/serio/hyperv-keyboard.c b/drivers/input/serio/hyperv-keyboard.c
index 88ae7c2..e486a8a 100644
--- a/drivers/input/serio/hyperv-keyboard.c
+++ b/drivers/input/serio/hyperv-keyboard.c
@@ -237,40 +237,17 @@ static void hv_kbd_handle_received_packet(struct hv_device *hv_dev,
static void hv_kbd_on_channel_callback(void *context)
{
+ struct vmpacket_descriptor *desc;
struct hv_device *hv_dev = context;
- void *buffer;
- int bufferlen = 0x100; /* Start with sensible size */
u32 bytes_recvd;
u64 req_id;
- int error;
- buffer = kmalloc(bufferlen, GFP_ATOMIC);
- if (!buffer)
- return;
+ foreach_vmbus_pkt(desc, hv_dev->channel) {
+ bytes_recvd = desc->len8 * 8;
+ req_id = desc->trans_id;
- while (1) {
- error = vmbus_recvpacket_raw(hv_dev->channel, buffer, bufferlen,
- &bytes_recvd, &req_id);
- switch (error) {
- case 0:
- if (bytes_recvd == 0) {
- kfree(buffer);
- return;
- }
-
- hv_kbd_handle_received_packet(hv_dev, buffer,
- bytes_recvd, req_id);
- break;
-
- case -ENOBUFS:
- kfree(buffer);
- /* Handle large packet */
- bufferlen = bytes_recvd;
- buffer = kmalloc(bytes_recvd, GFP_ATOMIC);
- if (!buffer)
- return;
- break;
- }
+ hv_kbd_handle_received_packet(hv_dev, desc, bytes_recvd,
+ req_id);
}
}
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index d991d40..f68a62c 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -965,11 +965,14 @@ static void *iommu_dma_alloc_pages(struct device *dev, size_t size,
{
bool coherent = dev_is_dma_coherent(dev);
size_t alloc_size = PAGE_ALIGN(size);
+ int node = dev_to_node(dev);
struct page *page = NULL;
void *cpu_addr;
page = dma_alloc_contiguous(dev, alloc_size, gfp);
if (!page)
+ page = alloc_pages_node(node, gfp, get_order(alloc_size));
+ if (!page)
return NULL;
if (IS_ENABLED(CONFIG_DMA_REMAP) && (!coherent || PageHighMem(page))) {
diff --git a/drivers/isdn/hardware/mISDN/avmfritz.c b/drivers/isdn/hardware/mISDN/avmfritz.c
index 81f2b18..1137dd1 100644
--- a/drivers/isdn/hardware/mISDN/avmfritz.c
+++ b/drivers/isdn/hardware/mISDN/avmfritz.c
@@ -509,8 +509,7 @@ HDLC_irq_xpr(struct bchannel *bch)
if (bch->tx_skb && bch->tx_idx < bch->tx_skb->len) {
hdlc_fill_fifo(bch);
} else {
- if (bch->tx_skb)
- dev_kfree_skb(bch->tx_skb);
+ dev_kfree_skb(bch->tx_skb);
if (get_next_bframe(bch)) {
hdlc_fill_fifo(bch);
test_and_clear_bit(FLG_TX_EMPTY, &bch->Flags);
diff --git a/drivers/isdn/hardware/mISDN/hfcpci.c b/drivers/isdn/hardware/mISDN/hfcpci.c
index 4a06958..2330a7d 100644
--- a/drivers/isdn/hardware/mISDN/hfcpci.c
+++ b/drivers/isdn/hardware/mISDN/hfcpci.c
@@ -1119,8 +1119,7 @@ tx_birq(struct bchannel *bch)
if (bch->tx_skb && bch->tx_idx < bch->tx_skb->len)
hfcpci_fill_fifo(bch);
else {
- if (bch->tx_skb)
- dev_kfree_skb(bch->tx_skb);
+ dev_kfree_skb(bch->tx_skb);
if (get_next_bframe(bch))
hfcpci_fill_fifo(bch);
}
@@ -1132,8 +1131,7 @@ tx_dirq(struct dchannel *dch)
if (dch->tx_skb && dch->tx_idx < dch->tx_skb->len)
hfcpci_fill_dfifo(dch->hw);
else {
- if (dch->tx_skb)
- dev_kfree_skb(dch->tx_skb);
+ dev_kfree_skb(dch->tx_skb);
if (get_next_dframe(dch))
hfcpci_fill_dfifo(dch->hw);
}
diff --git a/drivers/isdn/hardware/mISDN/mISDNipac.c b/drivers/isdn/hardware/mISDN/mISDNipac.c
index f915399..bca8802 100644
--- a/drivers/isdn/hardware/mISDN/mISDNipac.c
+++ b/drivers/isdn/hardware/mISDN/mISDNipac.c
@@ -190,8 +190,7 @@ isac_rme_irq(struct isac_hw *isac)
#endif
}
WriteISAC(isac, ISAC_CMDR, 0x80);
- if (isac->dch.rx_skb)
- dev_kfree_skb(isac->dch.rx_skb);
+ dev_kfree_skb(isac->dch.rx_skb);
isac->dch.rx_skb = NULL;
} else {
count = ReadISAC(isac, ISAC_RBCL) & 0x1f;
@@ -210,8 +209,7 @@ isac_xpr_irq(struct isac_hw *isac)
if (isac->dch.tx_skb && isac->dch.tx_idx < isac->dch.tx_skb->len) {
isac_fill_fifo(isac);
} else {
- if (isac->dch.tx_skb)
- dev_kfree_skb(isac->dch.tx_skb);
+ dev_kfree_skb(isac->dch.tx_skb);
if (get_next_dframe(&isac->dch))
isac_fill_fifo(isac);
}
@@ -464,8 +462,7 @@ isacsx_rme_irq(struct isac_hw *isac)
isac->dch.err_crc++;
#endif
WriteISAC(isac, ISACX_CMDRD, ISACX_CMDRD_RMC);
- if (isac->dch.rx_skb)
- dev_kfree_skb(isac->dch.rx_skb);
+ dev_kfree_skb(isac->dch.rx_skb);
isac->dch.rx_skb = NULL;
} else {
count = ReadISAC(isac, ISACX_RBCLD) & 0x1f;
@@ -1012,8 +1009,7 @@ hscx_xpr(struct hscx_hw *hx)
if (hx->bch.tx_skb && hx->bch.tx_idx < hx->bch.tx_skb->len) {
hscx_fill_fifo(hx);
} else {
- if (hx->bch.tx_skb)
- dev_kfree_skb(hx->bch.tx_skb);
+ dev_kfree_skb(hx->bch.tx_skb);
if (get_next_bframe(&hx->bch)) {
hscx_fill_fifo(hx);
test_and_clear_bit(FLG_TX_EMPTY, &hx->bch.Flags);
diff --git a/drivers/isdn/hardware/mISDN/mISDNisar.c b/drivers/isdn/hardware/mISDN/mISDNisar.c
index fd5c52f..4a3e748 100644
--- a/drivers/isdn/hardware/mISDN/mISDNisar.c
+++ b/drivers/isdn/hardware/mISDN/mISDNisar.c
@@ -690,8 +690,7 @@ send_next(struct isar_ch *ch)
}
}
}
- if (ch->bch.tx_skb)
- dev_kfree_skb(ch->bch.tx_skb);
+ dev_kfree_skb(ch->bch.tx_skb);
if (get_next_bframe(&ch->bch)) {
isar_fill_fifo(ch);
test_and_clear_bit(FLG_TX_EMPTY, &ch->bch.Flags);
diff --git a/drivers/isdn/hardware/mISDN/netjet.c b/drivers/isdn/hardware/mISDN/netjet.c
index 4e30aff..61caa7e 100644
--- a/drivers/isdn/hardware/mISDN/netjet.c
+++ b/drivers/isdn/hardware/mISDN/netjet.c
@@ -605,8 +605,7 @@ bc_next_frame(struct tiger_ch *bc)
if (bc->bch.tx_skb && bc->bch.tx_idx < bc->bch.tx_skb->len) {
fill_dma(bc);
} else {
- if (bc->bch.tx_skb)
- dev_kfree_skb(bc->bch.tx_skb);
+ dev_kfree_skb(bc->bch.tx_skb);
if (get_next_bframe(&bc->bch)) {
fill_dma(bc);
test_and_clear_bit(FLG_TX_EMPTY, &bc->bch.Flags);
diff --git a/drivers/isdn/hardware/mISDN/w6692.c b/drivers/isdn/hardware/mISDN/w6692.c
index 2402608..bad55fd 100644
--- a/drivers/isdn/hardware/mISDN/w6692.c
+++ b/drivers/isdn/hardware/mISDN/w6692.c
@@ -356,8 +356,7 @@ handle_rxD(struct w6692_hw *card) {
card->dch.err_rx++;
#endif
}
- if (card->dch.rx_skb)
- dev_kfree_skb(card->dch.rx_skb);
+ dev_kfree_skb(card->dch.rx_skb);
card->dch.rx_skb = NULL;
WriteW6692(card, W_D_CMDR, W_D_CMDR_RACK | W_D_CMDR_RRST);
} else {
@@ -376,8 +375,7 @@ handle_txD(struct w6692_hw *card) {
if (card->dch.tx_skb && card->dch.tx_idx < card->dch.tx_skb->len) {
W6692_fill_Dfifo(card);
} else {
- if (card->dch.tx_skb)
- dev_kfree_skb(card->dch.tx_skb);
+ dev_kfree_skb(card->dch.tx_skb);
if (get_next_dframe(&card->dch))
W6692_fill_Dfifo(card);
}
@@ -636,8 +634,7 @@ send_next(struct w6692_ch *wch)
if (wch->bch.tx_skb && wch->bch.tx_idx < wch->bch.tx_skb->len) {
W6692_fill_Bfifo(wch);
} else {
- if (wch->bch.tx_skb)
- dev_kfree_skb(wch->bch.tx_skb);
+ dev_kfree_skb(wch->bch.tx_skb);
if (get_next_bframe(&wch->bch)) {
W6692_fill_Bfifo(wch);
test_and_clear_bit(FLG_TX_EMPTY, &wch->bch.Flags);
diff --git a/drivers/isdn/mISDN/l1oip_core.c b/drivers/isdn/mISDN/l1oip_core.c
index 447f241..b57dcb8 100644
--- a/drivers/isdn/mISDN/l1oip_core.c
+++ b/drivers/isdn/mISDN/l1oip_core.c
@@ -1254,8 +1254,7 @@ release_card(struct l1oip *hc)
mISDN_freebchannel(hc->chan[ch].bch);
kfree(hc->chan[ch].bch);
#ifdef REORDER_DEBUG
- if (hc->chan[ch].disorder_skb)
- dev_kfree_skb(hc->chan[ch].disorder_skb);
+ dev_kfree_skb(hc->chan[ch].disorder_skb);
#endif
}
}
diff --git a/drivers/isdn/mISDN/layer2.c b/drivers/isdn/mISDN/layer2.c
index 68a48151..5bf7fcb 100644
--- a/drivers/isdn/mISDN/layer2.c
+++ b/drivers/isdn/mISDN/layer2.c
@@ -900,8 +900,7 @@ l2_disconnect(struct FsmInst *fi, int event, void *arg)
send_uframe(l2, NULL, DISC | 0x10, CMD);
mISDN_FsmDelTimer(&l2->t203, 1);
restart_t200(l2, 2);
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
}
static void
@@ -1722,8 +1721,7 @@ l2_set_own_busy(struct FsmInst *fi, int event, void *arg)
enquiry_cr(l2, RNR, RSP, 0);
test_and_clear_bit(FLG_ACK_PEND, &l2->flag);
}
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
}
static void
@@ -1736,8 +1734,7 @@ l2_clear_own_busy(struct FsmInst *fi, int event, void *arg)
enquiry_cr(l2, RR, RSP, 0);
test_and_clear_bit(FLG_ACK_PEND, &l2->flag);
}
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
}
static void
diff --git a/drivers/isdn/mISDN/stack.c b/drivers/isdn/mISDN/stack.c
index fa2237e..27aa329 100644
--- a/drivers/isdn/mISDN/stack.c
+++ b/drivers/isdn/mISDN/stack.c
@@ -75,8 +75,7 @@ send_socklist(struct mISDN_sock_list *sl, struct sk_buff *skb)
cskb = NULL;
}
read_unlock(&sl->lock);
- if (cskb)
- dev_kfree_skb(cskb);
+ dev_kfree_skb(cskb);
}
static void
@@ -134,8 +133,7 @@ send_layer2(struct mISDNstack *st, struct sk_buff *skb)
}
out:
mutex_unlock(&st->lmutex);
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
}
static inline int
diff --git a/drivers/isdn/mISDN/tei.c b/drivers/isdn/mISDN/tei.c
index a4fa594..59d28cb 100644
--- a/drivers/isdn/mISDN/tei.c
+++ b/drivers/isdn/mISDN/tei.c
@@ -1328,10 +1328,8 @@ mgr_bcast(struct mISDNchannel *ch, struct sk_buff *skb)
}
out:
read_unlock_irqrestore(&mgr->lock, flags);
- if (cskb)
- dev_kfree_skb(cskb);
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(cskb);
+ dev_kfree_skb(skb);
return 0;
}
diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index b6b5acc..2a48ea3 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -1599,7 +1599,9 @@ dm_bufio_shrink_scan(struct shrinker *shrink, struct shrink_control *sc)
unsigned long freed;
c = container_of(shrink, struct dm_bufio_client, shrinker);
- if (!dm_bufio_trylock(c))
+ if (sc->gfp_mask & __GFP_FS)
+ dm_bufio_lock(c);
+ else if (!dm_bufio_trylock(c))
return SHRINK_STOP;
freed = __scan(c, sc->nr_to_scan, sc->gfp_mask);
diff --git a/drivers/md/dm-dust.c b/drivers/md/dm-dust.c
index 845f376..8288887 100644
--- a/drivers/md/dm-dust.c
+++ b/drivers/md/dm-dust.c
@@ -25,6 +25,7 @@ struct dust_device {
unsigned long long badblock_count;
spinlock_t dust_lock;
unsigned int blksz;
+ int sect_per_block_shift;
unsigned int sect_per_block;
sector_t start;
bool fail_read_on_bb:1;
@@ -79,7 +80,7 @@ static int dust_remove_block(struct dust_device *dd, unsigned long long block)
unsigned long flags;
spin_lock_irqsave(&dd->dust_lock, flags);
- bblock = dust_rb_search(&dd->badblocklist, block * dd->sect_per_block);
+ bblock = dust_rb_search(&dd->badblocklist, block);
if (bblock == NULL) {
if (!dd->quiet_mode) {
@@ -113,7 +114,7 @@ static int dust_add_block(struct dust_device *dd, unsigned long long block)
}
spin_lock_irqsave(&dd->dust_lock, flags);
- bblock->bb = block * dd->sect_per_block;
+ bblock->bb = block;
if (!dust_rb_insert(&dd->badblocklist, bblock)) {
if (!dd->quiet_mode) {
DMERR("%s: block %llu already in badblocklist",
@@ -138,7 +139,7 @@ static int dust_query_block(struct dust_device *dd, unsigned long long block)
unsigned long flags;
spin_lock_irqsave(&dd->dust_lock, flags);
- bblock = dust_rb_search(&dd->badblocklist, block * dd->sect_per_block);
+ bblock = dust_rb_search(&dd->badblocklist, block);
if (bblock != NULL)
DMINFO("%s: block %llu found in badblocklist", __func__, block);
else
@@ -165,6 +166,7 @@ static int dust_map_read(struct dust_device *dd, sector_t thisblock,
int ret = DM_MAPIO_REMAPPED;
if (fail_read_on_bb) {
+ thisblock >>= dd->sect_per_block_shift;
spin_lock_irqsave(&dd->dust_lock, flags);
ret = __dust_map_read(dd, thisblock);
spin_unlock_irqrestore(&dd->dust_lock, flags);
@@ -195,6 +197,7 @@ static int dust_map_write(struct dust_device *dd, sector_t thisblock,
unsigned long flags;
if (fail_read_on_bb) {
+ thisblock >>= dd->sect_per_block_shift;
spin_lock_irqsave(&dd->dust_lock, flags);
__dust_map_write(dd, thisblock);
spin_unlock_irqrestore(&dd->dust_lock, flags);
@@ -331,6 +334,8 @@ static int dust_ctr(struct dm_target *ti, unsigned int argc, char **argv)
dd->blksz = blksz;
dd->start = tmp;
+ dd->sect_per_block_shift = __ffs(sect_per_block);
+
/*
* Whether to fail a read on a "bad" block.
* Defaults to false; enabled later by message.
diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c
index b1b0de40..9118ab8 100644
--- a/drivers/md/dm-integrity.c
+++ b/drivers/md/dm-integrity.c
@@ -1943,7 +1943,22 @@ static void dm_integrity_map_continue(struct dm_integrity_io *dio, bool from_map
queue_work(ic->wait_wq, &dio->work);
return;
}
+ if (journal_read_pos != NOT_FOUND)
+ dio->range.n_sectors = ic->sectors_per_block;
wait_and_add_new_range(ic, &dio->range);
+ /*
+ * wait_and_add_new_range drops the spinlock, so the journal
+ * may have been changed arbitrarily. We need to recheck.
+ * To simplify the code, we restrict I/O size to just one block.
+ */
+ if (journal_read_pos != NOT_FOUND) {
+ sector_t next_sector;
+ unsigned new_pos = find_journal_node(ic, dio->range.logical_sector, &next_sector);
+ if (unlikely(new_pos != journal_read_pos)) {
+ remove_range_unlocked(ic, &dio->range);
+ goto retry;
+ }
+ }
}
spin_unlock_irq(&ic->endio_wait.lock);
diff --git a/drivers/md/dm-kcopyd.c b/drivers/md/dm-kcopyd.c
index df2011d..1bbe4a3 100644
--- a/drivers/md/dm-kcopyd.c
+++ b/drivers/md/dm-kcopyd.c
@@ -566,8 +566,10 @@ static int run_io_job(struct kcopyd_job *job)
* no point in continuing.
*/
if (test_bit(DM_KCOPYD_WRITE_SEQ, &job->flags) &&
- job->master_job->write_err)
+ job->master_job->write_err) {
+ job->write_err = job->master_job->write_err;
return -EIO;
+ }
io_job_start(job->kc->throttle);
@@ -619,6 +621,7 @@ static int process_jobs(struct list_head *jobs, struct dm_kcopyd_client *kc,
else
job->read_err = 1;
push(&kc->complete_jobs, job);
+ wake(kc);
break;
}
diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c
index 8a60a4a..1f933dd 100644
--- a/drivers/md/dm-raid.c
+++ b/drivers/md/dm-raid.c
@@ -3194,7 +3194,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)
*/
r = rs_prepare_reshape(rs);
if (r)
- return r;
+ goto bad;
/* Reshaping ain't recovery, so disable recovery */
rs_setup_recovery(rs, MaxSector);
diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index 7b6c3ee..8820931 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -1342,7 +1342,7 @@ void dm_table_event(struct dm_table *t)
}
EXPORT_SYMBOL(dm_table_event);
-sector_t dm_table_get_size(struct dm_table *t)
+inline sector_t dm_table_get_size(struct dm_table *t)
{
return t->num_targets ? (t->highs[t->num_targets - 1] + 1) : 0;
}
@@ -1367,6 +1367,9 @@ struct dm_target *dm_table_find_target(struct dm_table *t, sector_t sector)
unsigned int l, n = 0, k = 0;
sector_t *node;
+ if (unlikely(sector >= dm_table_get_size(t)))
+ return &t->targets[t->num_targets];
+
for (l = 0; l < t->depth; l++) {
n = get_child(n, k);
node = get_node(t, l, n);
diff --git a/drivers/md/dm-zoned-metadata.c b/drivers/md/dm-zoned-metadata.c
index 8545dce..595a731 100644
--- a/drivers/md/dm-zoned-metadata.c
+++ b/drivers/md/dm-zoned-metadata.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2017 Western Digital Corporation or its affiliates.
*
@@ -34,7 +35,7 @@
* (1) Super block (1 block)
* (2) Chunk mapping table (nr_map_blocks)
* (3) Bitmap blocks (nr_bitmap_blocks)
- * All metadata blocks are stored in conventional zones, starting from the
+ * All metadata blocks are stored in conventional zones, starting from
* the first conventional zone found on disk.
*/
struct dmz_super {
@@ -233,7 +234,7 @@ void dmz_unlock_map(struct dmz_metadata *zmd)
* Lock/unlock metadata access. This is a "read" lock on a semaphore
* that prevents metadata flush from running while metadata are being
* modified. The actual metadata write mutual exclusion is achieved with
- * the map lock and zone styate management (active and reclaim state are
+ * the map lock and zone state management (active and reclaim state are
* mutually exclusive).
*/
void dmz_lock_metadata(struct dmz_metadata *zmd)
@@ -402,15 +403,18 @@ static struct dmz_mblock *dmz_get_mblock_slow(struct dmz_metadata *zmd,
sector_t block = zmd->sb[zmd->mblk_primary].block + mblk_no;
struct bio *bio;
+ if (dmz_bdev_is_dying(zmd->dev))
+ return ERR_PTR(-EIO);
+
/* Get a new block and a BIO to read it */
mblk = dmz_alloc_mblock(zmd, mblk_no);
if (!mblk)
- return NULL;
+ return ERR_PTR(-ENOMEM);
bio = bio_alloc(GFP_NOIO, 1);
if (!bio) {
dmz_free_mblock(zmd, mblk);
- return NULL;
+ return ERR_PTR(-ENOMEM);
}
spin_lock(&zmd->mblk_lock);
@@ -541,8 +545,8 @@ static struct dmz_mblock *dmz_get_mblock(struct dmz_metadata *zmd,
if (!mblk) {
/* Cache miss: read the block from disk */
mblk = dmz_get_mblock_slow(zmd, mblk_no);
- if (!mblk)
- return ERR_PTR(-ENOMEM);
+ if (IS_ERR(mblk))
+ return mblk;
}
/* Wait for on-going read I/O and check for error */
@@ -570,16 +574,19 @@ static void dmz_dirty_mblock(struct dmz_metadata *zmd, struct dmz_mblock *mblk)
/*
* Issue a metadata block write BIO.
*/
-static void dmz_write_mblock(struct dmz_metadata *zmd, struct dmz_mblock *mblk,
- unsigned int set)
+static int dmz_write_mblock(struct dmz_metadata *zmd, struct dmz_mblock *mblk,
+ unsigned int set)
{
sector_t block = zmd->sb[set].block + mblk->no;
struct bio *bio;
+ if (dmz_bdev_is_dying(zmd->dev))
+ return -EIO;
+
bio = bio_alloc(GFP_NOIO, 1);
if (!bio) {
set_bit(DMZ_META_ERROR, &mblk->state);
- return;
+ return -ENOMEM;
}
set_bit(DMZ_META_WRITING, &mblk->state);
@@ -591,6 +598,8 @@ static void dmz_write_mblock(struct dmz_metadata *zmd, struct dmz_mblock *mblk,
bio_set_op_attrs(bio, REQ_OP_WRITE, REQ_META | REQ_PRIO);
bio_add_page(bio, mblk->page, DMZ_BLOCK_SIZE, 0);
submit_bio(bio);
+
+ return 0;
}
/*
@@ -602,6 +611,9 @@ static int dmz_rdwr_block(struct dmz_metadata *zmd, int op, sector_t block,
struct bio *bio;
int ret;
+ if (dmz_bdev_is_dying(zmd->dev))
+ return -EIO;
+
bio = bio_alloc(GFP_NOIO, 1);
if (!bio)
return -ENOMEM;
@@ -659,22 +671,29 @@ static int dmz_write_dirty_mblocks(struct dmz_metadata *zmd,
{
struct dmz_mblock *mblk;
struct blk_plug plug;
- int ret = 0;
+ int ret = 0, nr_mblks_submitted = 0;
/* Issue writes */
blk_start_plug(&plug);
- list_for_each_entry(mblk, write_list, link)
- dmz_write_mblock(zmd, mblk, set);
+ list_for_each_entry(mblk, write_list, link) {
+ ret = dmz_write_mblock(zmd, mblk, set);
+ if (ret)
+ break;
+ nr_mblks_submitted++;
+ }
blk_finish_plug(&plug);
/* Wait for completion */
list_for_each_entry(mblk, write_list, link) {
+ if (!nr_mblks_submitted)
+ break;
wait_on_bit_io(&mblk->state, DMZ_META_WRITING,
TASK_UNINTERRUPTIBLE);
if (test_bit(DMZ_META_ERROR, &mblk->state)) {
clear_bit(DMZ_META_ERROR, &mblk->state);
ret = -EIO;
}
+ nr_mblks_submitted--;
}
/* Flush drive cache (this will also sync data) */
@@ -736,6 +755,11 @@ int dmz_flush_metadata(struct dmz_metadata *zmd)
*/
dmz_lock_flush(zmd);
+ if (dmz_bdev_is_dying(zmd->dev)) {
+ ret = -EIO;
+ goto out;
+ }
+
/* Get dirty blocks */
spin_lock(&zmd->mblk_lock);
list_splice_init(&zmd->mblk_dirty_list, &write_list);
@@ -1542,7 +1566,7 @@ static struct dm_zone *dmz_get_rnd_zone_for_reclaim(struct dmz_metadata *zmd)
struct dm_zone *zone;
if (list_empty(&zmd->map_rnd_list))
- return NULL;
+ return ERR_PTR(-EBUSY);
list_for_each_entry(zone, &zmd->map_rnd_list, link) {
if (dmz_is_buf(zone))
@@ -1553,7 +1577,7 @@ static struct dm_zone *dmz_get_rnd_zone_for_reclaim(struct dmz_metadata *zmd)
return dzone;
}
- return NULL;
+ return ERR_PTR(-EBUSY);
}
/*
@@ -1564,7 +1588,7 @@ static struct dm_zone *dmz_get_seq_zone_for_reclaim(struct dmz_metadata *zmd)
struct dm_zone *zone;
if (list_empty(&zmd->map_seq_list))
- return NULL;
+ return ERR_PTR(-EBUSY);
list_for_each_entry(zone, &zmd->map_seq_list, link) {
if (!zone->bzone)
@@ -1573,7 +1597,7 @@ static struct dm_zone *dmz_get_seq_zone_for_reclaim(struct dmz_metadata *zmd)
return zone;
}
- return NULL;
+ return ERR_PTR(-EBUSY);
}
/*
@@ -1628,9 +1652,13 @@ struct dm_zone *dmz_get_chunk_mapping(struct dmz_metadata *zmd, unsigned int chu
if (op != REQ_OP_WRITE)
goto out;
- /* Alloate a random zone */
+ /* Allocate a random zone */
dzone = dmz_alloc_zone(zmd, DMZ_ALLOC_RND);
if (!dzone) {
+ if (dmz_bdev_is_dying(zmd->dev)) {
+ dzone = ERR_PTR(-EIO);
+ goto out;
+ }
dmz_wait_for_free_zones(zmd);
goto again;
}
@@ -1725,9 +1753,13 @@ struct dm_zone *dmz_get_chunk_buffer(struct dmz_metadata *zmd,
if (bzone)
goto out;
- /* Alloate a random zone */
+ /* Allocate a random zone */
bzone = dmz_alloc_zone(zmd, DMZ_ALLOC_RND);
if (!bzone) {
+ if (dmz_bdev_is_dying(zmd->dev)) {
+ bzone = ERR_PTR(-EIO);
+ goto out;
+ }
dmz_wait_for_free_zones(zmd);
goto again;
}
diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index edf4b95..d240d7c 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2017 Western Digital Corporation or its affiliates.
*
@@ -37,7 +38,7 @@ enum {
/*
* Number of seconds of target BIO inactivity to consider the target idle.
*/
-#define DMZ_IDLE_PERIOD (10UL * HZ)
+#define DMZ_IDLE_PERIOD (10UL * HZ)
/*
* Percentage of unmapped (free) random zones below which reclaim starts
@@ -134,6 +135,9 @@ static int dmz_reclaim_copy(struct dmz_reclaim *zrc,
set_bit(DM_KCOPYD_WRITE_SEQ, &flags);
while (block < end_block) {
+ if (dev->flags & DMZ_BDEV_DYING)
+ return -EIO;
+
/* Get a valid region from the source zone */
ret = dmz_first_valid_block(zmd, src_zone, &block);
if (ret <= 0)
@@ -215,7 +219,7 @@ static int dmz_reclaim_buf(struct dmz_reclaim *zrc, struct dm_zone *dzone)
dmz_unlock_flush(zmd);
- return 0;
+ return ret;
}
/*
@@ -259,7 +263,7 @@ static int dmz_reclaim_seq_data(struct dmz_reclaim *zrc, struct dm_zone *dzone)
dmz_unlock_flush(zmd);
- return 0;
+ return ret;
}
/*
@@ -312,7 +316,7 @@ static int dmz_reclaim_rnd_data(struct dmz_reclaim *zrc, struct dm_zone *dzone)
dmz_unlock_flush(zmd);
- return 0;
+ return ret;
}
/*
@@ -334,7 +338,7 @@ static void dmz_reclaim_empty(struct dmz_reclaim *zrc, struct dm_zone *dzone)
/*
* Find a candidate zone for reclaim and process it.
*/
-static void dmz_reclaim(struct dmz_reclaim *zrc)
+static int dmz_do_reclaim(struct dmz_reclaim *zrc)
{
struct dmz_metadata *zmd = zrc->metadata;
struct dm_zone *dzone;
@@ -344,8 +348,8 @@ static void dmz_reclaim(struct dmz_reclaim *zrc)
/* Get a data zone */
dzone = dmz_get_zone_for_reclaim(zmd);
- if (!dzone)
- return;
+ if (IS_ERR(dzone))
+ return PTR_ERR(dzone);
start = jiffies;
@@ -391,13 +395,20 @@ static void dmz_reclaim(struct dmz_reclaim *zrc)
out:
if (ret) {
dmz_unlock_zone_reclaim(dzone);
- return;
+ return ret;
}
- (void) dmz_flush_metadata(zrc->metadata);
+ ret = dmz_flush_metadata(zrc->metadata);
+ if (ret) {
+ dmz_dev_debug(zrc->dev,
+ "Metadata flush for zone %u failed, err %d\n",
+ dmz_id(zmd, rzone), ret);
+ return ret;
+ }
dmz_dev_debug(zrc->dev, "Reclaimed zone %u in %u ms",
dmz_id(zmd, rzone), jiffies_to_msecs(jiffies - start));
+ return 0;
}
/*
@@ -427,7 +438,7 @@ static bool dmz_should_reclaim(struct dmz_reclaim *zrc)
return false;
/*
- * If the percentage of unmappped random zones is low,
+ * If the percentage of unmapped random zones is low,
* reclaim even if the target is busy.
*/
return p_unmap_rnd <= DMZ_RECLAIM_LOW_UNMAP_RND;
@@ -442,6 +453,10 @@ static void dmz_reclaim_work(struct work_struct *work)
struct dmz_metadata *zmd = zrc->metadata;
unsigned int nr_rnd, nr_unmap_rnd;
unsigned int p_unmap_rnd;
+ int ret;
+
+ if (dmz_bdev_is_dying(zrc->dev))
+ return;
if (!dmz_should_reclaim(zrc)) {
mod_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
@@ -471,7 +486,17 @@ static void dmz_reclaim_work(struct work_struct *work)
(dmz_target_idle(zrc) ? "Idle" : "Busy"),
p_unmap_rnd, nr_unmap_rnd, nr_rnd);
- dmz_reclaim(zrc);
+ ret = dmz_do_reclaim(zrc);
+ if (ret) {
+ dmz_dev_debug(zrc->dev, "Reclaim error %d\n", ret);
+ if (ret == -EIO)
+ /*
+ * LLD might be performing some error handling sequence
+ * at the underlying device. To not interfere, do not
+ * attempt to schedule the next reclaim run immediately.
+ */
+ return;
+ }
dmz_schedule_reclaim(zrc);
}
diff --git a/drivers/md/dm-zoned-target.c b/drivers/md/dm-zoned-target.c
index 51d029b..31478fe 100644
--- a/drivers/md/dm-zoned-target.c
+++ b/drivers/md/dm-zoned-target.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2017 Western Digital Corporation or its affiliates.
*
@@ -133,6 +134,8 @@ static int dmz_submit_bio(struct dmz_target *dmz, struct dm_zone *zone,
refcount_inc(&bioctx->ref);
generic_make_request(clone);
+ if (clone->bi_status == BLK_STS_IOERR)
+ return -EIO;
if (bio_op(bio) == REQ_OP_WRITE && dmz_is_seq(zone))
zone->wp_block += nr_blocks;
@@ -277,8 +280,8 @@ static int dmz_handle_buffered_write(struct dmz_target *dmz,
/* Get the buffer zone. One will be allocated if needed */
bzone = dmz_get_chunk_buffer(zmd, zone);
- if (!bzone)
- return -ENOSPC;
+ if (IS_ERR(bzone))
+ return PTR_ERR(bzone);
if (dmz_is_readonly(bzone))
return -EROFS;
@@ -389,6 +392,11 @@ static void dmz_handle_bio(struct dmz_target *dmz, struct dm_chunk_work *cw,
dmz_lock_metadata(zmd);
+ if (dmz->dev->flags & DMZ_BDEV_DYING) {
+ ret = -EIO;
+ goto out;
+ }
+
/*
* Get the data zone mapping the chunk. There may be no
* mapping for read and discard. If a mapping is obtained,
@@ -493,6 +501,8 @@ static void dmz_flush_work(struct work_struct *work)
/* Flush dirty metadata blocks */
ret = dmz_flush_metadata(dmz->metadata);
+ if (ret)
+ dmz_dev_debug(dmz->dev, "Metadata flush failed, rc=%d\n", ret);
/* Process queued flush requests */
while (1) {
@@ -513,22 +523,24 @@ static void dmz_flush_work(struct work_struct *work)
* Get a chunk work and start it to process a new BIO.
* If the BIO chunk has no work yet, create one.
*/
-static void dmz_queue_chunk_work(struct dmz_target *dmz, struct bio *bio)
+static int dmz_queue_chunk_work(struct dmz_target *dmz, struct bio *bio)
{
unsigned int chunk = dmz_bio_chunk(dmz->dev, bio);
struct dm_chunk_work *cw;
+ int ret = 0;
mutex_lock(&dmz->chunk_lock);
/* Get the BIO chunk work. If one is not active yet, create one */
cw = radix_tree_lookup(&dmz->chunk_rxtree, chunk);
if (!cw) {
- int ret;
/* Create a new chunk work */
cw = kmalloc(sizeof(struct dm_chunk_work), GFP_NOIO);
- if (!cw)
+ if (unlikely(!cw)) {
+ ret = -ENOMEM;
goto out;
+ }
INIT_WORK(&cw->work, dmz_chunk_work);
refcount_set(&cw->refcount, 0);
@@ -539,7 +551,6 @@ static void dmz_queue_chunk_work(struct dmz_target *dmz, struct bio *bio)
ret = radix_tree_insert(&dmz->chunk_rxtree, chunk, cw);
if (unlikely(ret)) {
kfree(cw);
- cw = NULL;
goto out;
}
}
@@ -547,10 +558,38 @@ static void dmz_queue_chunk_work(struct dmz_target *dmz, struct bio *bio)
bio_list_add(&cw->bio_list, bio);
dmz_get_chunk_work(cw);
+ dmz_reclaim_bio_acc(dmz->reclaim);
if (queue_work(dmz->chunk_wq, &cw->work))
dmz_get_chunk_work(cw);
out:
mutex_unlock(&dmz->chunk_lock);
+ return ret;
+}
+
+/*
+ * Check the backing device availability. If it's on the way out,
+ * start failing I/O. Reclaim and metadata components also call this
+ * function to cleanly abort operation in the event of such failure.
+ */
+bool dmz_bdev_is_dying(struct dmz_dev *dmz_dev)
+{
+ struct gendisk *disk;
+
+ if (!(dmz_dev->flags & DMZ_BDEV_DYING)) {
+ disk = dmz_dev->bdev->bd_disk;
+ if (blk_queue_dying(bdev_get_queue(dmz_dev->bdev))) {
+ dmz_dev_warn(dmz_dev, "Backing device queue dying");
+ dmz_dev->flags |= DMZ_BDEV_DYING;
+ } else if (disk->fops->check_events) {
+ if (disk->fops->check_events(disk, 0) &
+ DISK_EVENT_MEDIA_CHANGE) {
+ dmz_dev_warn(dmz_dev, "Backing device offline");
+ dmz_dev->flags |= DMZ_BDEV_DYING;
+ }
+ }
+ }
+
+ return dmz_dev->flags & DMZ_BDEV_DYING;
}
/*
@@ -564,6 +603,10 @@ static int dmz_map(struct dm_target *ti, struct bio *bio)
sector_t sector = bio->bi_iter.bi_sector;
unsigned int nr_sectors = bio_sectors(bio);
sector_t chunk_sector;
+ int ret;
+
+ if (dmz_bdev_is_dying(dmz->dev))
+ return DM_MAPIO_KILL;
dmz_dev_debug(dev, "BIO op %d sector %llu + %u => chunk %llu, block %llu, %u blocks",
bio_op(bio), (unsigned long long)sector, nr_sectors,
@@ -601,8 +644,14 @@ static int dmz_map(struct dm_target *ti, struct bio *bio)
dm_accept_partial_bio(bio, dev->zone_nr_sectors - chunk_sector);
/* Now ready to handle this BIO */
- dmz_reclaim_bio_acc(dmz->reclaim);
- dmz_queue_chunk_work(dmz, bio);
+ ret = dmz_queue_chunk_work(dmz, bio);
+ if (ret) {
+ dmz_dev_debug(dmz->dev,
+ "BIO op %d, can't process chunk %llu, err %i\n",
+ bio_op(bio), (u64)dmz_bio_chunk(dmz->dev, bio),
+ ret);
+ return DM_MAPIO_REQUEUE;
+ }
return DM_MAPIO_SUBMITTED;
}
@@ -855,6 +904,9 @@ static int dmz_prepare_ioctl(struct dm_target *ti, struct block_device **bdev)
{
struct dmz_target *dmz = ti->private;
+ if (dmz_bdev_is_dying(dmz->dev))
+ return -ENODEV;
+
*bdev = dmz->dev->bdev;
return 0;
diff --git a/drivers/md/dm-zoned.h b/drivers/md/dm-zoned.h
index ed8de49..d8e70b0 100644
--- a/drivers/md/dm-zoned.h
+++ b/drivers/md/dm-zoned.h
@@ -1,3 +1,4 @@
+/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2017 Western Digital Corporation or its affiliates.
*
@@ -56,6 +57,8 @@ struct dmz_dev {
unsigned int nr_zones;
+ unsigned int flags;
+
sector_t zone_nr_sectors;
unsigned int zone_nr_sectors_shift;
@@ -67,6 +70,9 @@ struct dmz_dev {
(dev)->zone_nr_sectors_shift)
#define dmz_chunk_block(dev, b) ((b) & ((dev)->zone_nr_blocks - 1))
+/* Device flags. */
+#define DMZ_BDEV_DYING (1 << 0)
+
/*
* Zone descriptor.
*/
@@ -245,4 +251,9 @@ void dmz_resume_reclaim(struct dmz_reclaim *zrc);
void dmz_reclaim_bio_acc(struct dmz_reclaim *zrc);
void dmz_schedule_reclaim(struct dmz_reclaim *zrc);
+/*
+ * Functions defined in dm-zoned-target.c
+ */
+bool dmz_bdev_is_dying(struct dmz_dev *dmz_dev);
+
#endif /* DM_ZONED_H */
diff --git a/drivers/md/persistent-data/dm-btree.c b/drivers/md/persistent-data/dm-btree.c
index 58b3197..8aae062 100644
--- a/drivers/md/persistent-data/dm-btree.c
+++ b/drivers/md/persistent-data/dm-btree.c
@@ -628,39 +628,40 @@ static int btree_split_beneath(struct shadow_spine *s, uint64_t key)
new_parent = shadow_current(s);
+ pn = dm_block_data(new_parent);
+ size = le32_to_cpu(pn->header.flags) & INTERNAL_NODE ?
+ sizeof(__le64) : s->info->value_type.size;
+
+ /* create & init the left block */
r = new_block(s->info, &left);
if (r < 0)
return r;
+ ln = dm_block_data(left);
+ nr_left = le32_to_cpu(pn->header.nr_entries) / 2;
+
+ ln->header.flags = pn->header.flags;
+ ln->header.nr_entries = cpu_to_le32(nr_left);
+ ln->header.max_entries = pn->header.max_entries;
+ ln->header.value_size = pn->header.value_size;
+ memcpy(ln->keys, pn->keys, nr_left * sizeof(pn->keys[0]));
+ memcpy(value_ptr(ln, 0), value_ptr(pn, 0), nr_left * size);
+
+ /* create & init the right block */
r = new_block(s->info, &right);
if (r < 0) {
unlock_block(s->info, left);
return r;
}
- pn = dm_block_data(new_parent);
- ln = dm_block_data(left);
rn = dm_block_data(right);
-
- nr_left = le32_to_cpu(pn->header.nr_entries) / 2;
nr_right = le32_to_cpu(pn->header.nr_entries) - nr_left;
- ln->header.flags = pn->header.flags;
- ln->header.nr_entries = cpu_to_le32(nr_left);
- ln->header.max_entries = pn->header.max_entries;
- ln->header.value_size = pn->header.value_size;
-
rn->header.flags = pn->header.flags;
rn->header.nr_entries = cpu_to_le32(nr_right);
rn->header.max_entries = pn->header.max_entries;
rn->header.value_size = pn->header.value_size;
-
- memcpy(ln->keys, pn->keys, nr_left * sizeof(pn->keys[0]));
memcpy(rn->keys, pn->keys + nr_left, nr_right * sizeof(pn->keys[0]));
-
- size = le32_to_cpu(pn->header.flags) & INTERNAL_NODE ?
- sizeof(__le64) : s->info->value_type.size;
- memcpy(value_ptr(ln, 0), value_ptr(pn, 0), nr_left * size);
memcpy(value_ptr(rn, 0), value_ptr(pn, nr_left),
nr_right * size);
diff --git a/drivers/md/persistent-data/dm-space-map-metadata.c b/drivers/md/persistent-data/dm-space-map-metadata.c
index aec4492..2532858 100644
--- a/drivers/md/persistent-data/dm-space-map-metadata.c
+++ b/drivers/md/persistent-data/dm-space-map-metadata.c
@@ -249,7 +249,7 @@ static int out(struct sm_metadata *smm)
}
if (smm->recursion_count == 1)
- apply_bops(smm);
+ r = apply_bops(smm);
smm->recursion_count--;
diff --git a/drivers/mfd/rk808.c b/drivers/mfd/rk808.c
index 601cefb..050478c 100644
--- a/drivers/mfd/rk808.c
+++ b/drivers/mfd/rk808.c
@@ -729,7 +729,7 @@ static int rk808_remove(struct i2c_client *client)
return 0;
}
-static int rk8xx_suspend(struct device *dev)
+static int __maybe_unused rk8xx_suspend(struct device *dev)
{
struct rk808 *rk808 = i2c_get_clientdata(rk808_i2c_client);
int ret = 0;
@@ -749,7 +749,7 @@ static int rk8xx_suspend(struct device *dev)
return ret;
}
-static int rk8xx_resume(struct device *dev)
+static int __maybe_unused rk8xx_resume(struct device *dev)
{
struct rk808 *rk808 = i2c_get_clientdata(rk808_i2c_client);
int ret = 0;
@@ -768,7 +768,7 @@ static int rk8xx_resume(struct device *dev)
return ret;
}
-SIMPLE_DEV_PM_OPS(rk8xx_pm_ops, rk8xx_suspend, rk8xx_resume);
+static SIMPLE_DEV_PM_OPS(rk8xx_pm_ops, rk8xx_suspend, rk8xx_resume);
static struct i2c_driver rk808_i2c_driver = {
.driver = {
diff --git a/drivers/misc/lkdtm/bugs.c b/drivers/misc/lkdtm/bugs.c
index 1606658..24245cc 100644
--- a/drivers/misc/lkdtm/bugs.c
+++ b/drivers/misc/lkdtm/bugs.c
@@ -22,7 +22,7 @@ struct lkdtm_list {
* recurse past the end of THREAD_SIZE by default.
*/
#if defined(CONFIG_FRAME_WARN) && (CONFIG_FRAME_WARN > 0)
-#define REC_STACK_SIZE (CONFIG_FRAME_WARN / 2)
+#define REC_STACK_SIZE (_AC(CONFIG_FRAME_WARN, UL) / 2)
#else
#define REC_STACK_SIZE (THREAD_SIZE / 8)
#endif
@@ -91,7 +91,7 @@ void lkdtm_LOOP(void)
void lkdtm_EXHAUST_STACK(void)
{
- pr_info("Calling function with %d frame size to depth %d ...\n",
+ pr_info("Calling function with %lu frame size to depth %d ...\n",
REC_STACK_SIZE, recur_count);
recursive_loop(recur_count);
pr_info("FAIL: survived without exhausting stack?!\n");
diff --git a/drivers/misc/mei/hw-me-regs.h b/drivers/misc/mei/hw-me-regs.h
index 6c01737..77f7dff 100644
--- a/drivers/misc/mei/hw-me-regs.h
+++ b/drivers/misc/mei/hw-me-regs.h
@@ -81,6 +81,8 @@
#define MEI_DEV_ID_ICP_LP 0x34E0 /* Ice Lake Point LP */
+#define MEI_DEV_ID_TGP_LP 0xA0E0 /* Tiger Lake Point LP */
+
#define MEI_DEV_ID_MCC 0x4B70 /* Mule Creek Canyon (EHL) */
#define MEI_DEV_ID_MCC_4 0x4B75 /* Mule Creek Canyon 4 (EHL) */
diff --git a/drivers/misc/mei/pci-me.c b/drivers/misc/mei/pci-me.c
index 57cb68f..541538e 100644
--- a/drivers/misc/mei/pci-me.c
+++ b/drivers/misc/mei/pci-me.c
@@ -98,6 +98,8 @@ static const struct pci_device_id mei_me_pci_tbl[] = {
{MEI_PCI_DEVICE(MEI_DEV_ID_ICP_LP, MEI_ME_PCH12_CFG)},
+ {MEI_PCI_DEVICE(MEI_DEV_ID_TGP_LP, MEI_ME_PCH12_CFG)},
+
{MEI_PCI_DEVICE(MEI_DEV_ID_MCC, MEI_ME_PCH12_CFG)},
{MEI_PCI_DEVICE(MEI_DEV_ID_MCC_4, MEI_ME_PCH8_CFG)},
diff --git a/drivers/misc/vmw_balloon.c b/drivers/misc/vmw_balloon.c
index 8840299..5e6be15 100644
--- a/drivers/misc/vmw_balloon.c
+++ b/drivers/misc/vmw_balloon.c
@@ -691,7 +691,6 @@ static int vmballoon_alloc_page_list(struct vmballoon *b,
}
if (page) {
- vmballoon_mark_page_offline(page, ctl->page_size);
/* Success. Add the page to the list and continue. */
list_add(&page->lru, &ctl->pages);
continue;
@@ -930,7 +929,6 @@ static void vmballoon_release_page_list(struct list_head *page_list,
list_for_each_entry_safe(page, tmp, page_list, lru) {
list_del(&page->lru);
- vmballoon_mark_page_online(page, page_size);
__free_pages(page, vmballoon_page_order(page_size));
}
@@ -1005,6 +1003,7 @@ static void vmballoon_enqueue_page_list(struct vmballoon *b,
enum vmballoon_page_size_type page_size)
{
unsigned long flags;
+ struct page *page;
if (page_size == VMW_BALLOON_4K_PAGE) {
balloon_page_list_enqueue(&b->b_dev_info, pages);
@@ -1014,6 +1013,11 @@ static void vmballoon_enqueue_page_list(struct vmballoon *b,
* for the balloon compaction mechanism.
*/
spin_lock_irqsave(&b->b_dev_info.pages_lock, flags);
+
+ list_for_each_entry(page, pages, lru) {
+ vmballoon_mark_page_offline(page, VMW_BALLOON_2M_PAGE);
+ }
+
list_splice_init(pages, &b->huge_pages);
__count_vm_events(BALLOON_INFLATE, *n_pages *
vmballoon_page_in_frames(VMW_BALLOON_2M_PAGE));
@@ -1056,6 +1060,8 @@ static void vmballoon_dequeue_page_list(struct vmballoon *b,
/* 2MB pages */
spin_lock_irqsave(&b->b_dev_info.pages_lock, flags);
list_for_each_entry_safe(page, tmp, &b->huge_pages, lru) {
+ vmballoon_mark_page_online(page, VMW_BALLOON_2M_PAGE);
+
list_move(&page->lru, pages);
if (++i == n_req_pages)
break;
diff --git a/drivers/misc/vmw_vmci/vmci_doorbell.c b/drivers/misc/vmw_vmci/vmci_doorbell.c
index bad89b6..345addd 100644
--- a/drivers/misc/vmw_vmci/vmci_doorbell.c
+++ b/drivers/misc/vmw_vmci/vmci_doorbell.c
@@ -310,7 +310,8 @@ int vmci_dbell_host_context_notify(u32 src_cid, struct vmci_handle handle)
entry = container_of(resource, struct dbell_entry, resource);
if (entry->run_delayed) {
- schedule_work(&entry->work);
+ if (!schedule_work(&entry->work))
+ vmci_resource_put(resource);
} else {
entry->notify_cb(entry->client_data);
vmci_resource_put(resource);
@@ -361,7 +362,8 @@ static void dbell_fire_entries(u32 notify_idx)
atomic_read(&dbell->active) == 1) {
if (dbell->run_delayed) {
vmci_resource_get(&dbell->resource);
- schedule_work(&dbell->work);
+ if (!schedule_work(&dbell->work))
+ vmci_resource_put(&dbell->resource);
} else {
dbell->notify_cb(dbell->client_data);
}
diff --git a/drivers/mmc/core/sd.c b/drivers/mmc/core/sd.c
index d681e8a..fe914ff 100644
--- a/drivers/mmc/core/sd.c
+++ b/drivers/mmc/core/sd.c
@@ -1292,6 +1292,12 @@ int mmc_attach_sd(struct mmc_host *host)
goto err;
}
+ /*
+ * Some SD cards claims an out of spec VDD voltage range. Let's treat
+ * these bits as being in-valid and especially also bit7.
+ */
+ ocr &= ~0x7FFF;
+
rocr = mmc_select_voltage(host, ocr);
/*
diff --git a/drivers/mmc/host/sdhci-cadence.c b/drivers/mmc/host/sdhci-cadence.c
index 163d1cf..44139fc 100644
--- a/drivers/mmc/host/sdhci-cadence.c
+++ b/drivers/mmc/host/sdhci-cadence.c
@@ -369,6 +369,7 @@ static int sdhci_cdns_probe(struct platform_device *pdev)
host->mmc_host_ops.execute_tuning = sdhci_cdns_execute_tuning;
host->mmc_host_ops.hs400_enhanced_strobe =
sdhci_cdns_hs400_enhanced_strobe;
+ sdhci_enable_v4_mode(host);
sdhci_get_of_property(pdev);
diff --git a/drivers/mmc/host/sdhci-of-at91.c b/drivers/mmc/host/sdhci-of-at91.c
index d4e7e8b..e7d1920 100644
--- a/drivers/mmc/host/sdhci-of-at91.c
+++ b/drivers/mmc/host/sdhci-of-at91.c
@@ -357,6 +357,9 @@ static int sdhci_at91_probe(struct platform_device *pdev)
pm_runtime_set_autosuspend_delay(&pdev->dev, 50);
pm_runtime_use_autosuspend(&pdev->dev);
+ /* HS200 is broken at this moment */
+ host->quirks2 = SDHCI_QUIRK2_BROKEN_HS200;
+
ret = sdhci_add_host(host);
if (ret)
goto pm_runtime_disable;
diff --git a/drivers/mmc/host/sdhci-sprd.c b/drivers/mmc/host/sdhci-sprd.c
index 83a4767..d07b979 100644
--- a/drivers/mmc/host/sdhci-sprd.c
+++ b/drivers/mmc/host/sdhci-sprd.c
@@ -217,10 +217,11 @@ static inline void _sdhci_sprd_set_clock(struct sdhci_host *host,
struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host);
u32 div, val, mask;
- div = sdhci_sprd_calc_div(sprd_host->base_rate, clk);
+ sdhci_writew(host, 0, SDHCI_CLOCK_CONTROL);
- clk |= ((div & 0x300) >> 2) | ((div & 0xFF) << 8);
- sdhci_enable_clk(host, clk);
+ div = sdhci_sprd_calc_div(sprd_host->base_rate, clk);
+ div = ((div & 0x300) >> 2) | ((div & 0xFF) << 8);
+ sdhci_enable_clk(host, div);
/* enable auto gate sdhc_enable_auto_gate */
val = sdhci_readl(host, SDHCI_SPRD_REG_32_BUSY_POSI);
@@ -373,6 +374,11 @@ static unsigned int sdhci_sprd_get_max_timeout_count(struct sdhci_host *host)
return 1 << 31;
}
+static unsigned int sdhci_sprd_get_ro(struct sdhci_host *host)
+{
+ return 0;
+}
+
static struct sdhci_ops sdhci_sprd_ops = {
.read_l = sdhci_sprd_readl,
.write_l = sdhci_sprd_writel,
@@ -385,6 +391,7 @@ static struct sdhci_ops sdhci_sprd_ops = {
.set_uhs_signaling = sdhci_sprd_set_uhs_signaling,
.hw_reset = sdhci_sprd_hw_reset,
.get_max_timeout_count = sdhci_sprd_get_max_timeout_count,
+ .get_ro = sdhci_sprd_get_ro,
};
static void sdhci_sprd_request(struct mmc_host *mmc, struct mmc_request *mrq)
@@ -501,9 +508,12 @@ static void sdhci_sprd_phy_param_parse(struct sdhci_sprd_host *sprd_host,
}
static const struct sdhci_pltfm_data sdhci_sprd_pdata = {
- .quirks = SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK,
+ .quirks = SDHCI_QUIRK_BROKEN_CARD_DETECTION |
+ SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK |
+ SDHCI_QUIRK_MISSING_CAPS,
.quirks2 = SDHCI_QUIRK2_BROKEN_HS200 |
- SDHCI_QUIRK2_USE_32BIT_BLK_CNT,
+ SDHCI_QUIRK2_USE_32BIT_BLK_CNT |
+ SDHCI_QUIRK2_PRESET_VALUE_BROKEN,
.ops = &sdhci_sprd_ops,
};
@@ -605,6 +615,16 @@ static int sdhci_sprd_probe(struct platform_device *pdev)
sdhci_enable_v4_mode(host);
+ /*
+ * Supply the existing CAPS, but clear the UHS-I modes. This
+ * will allow these modes to be specified only by device
+ * tree properties through mmc_of_parse().
+ */
+ host->caps = sdhci_readl(host, SDHCI_CAPABILITIES);
+ host->caps1 = sdhci_readl(host, SDHCI_CAPABILITIES_1);
+ host->caps1 &= ~(SDHCI_SUPPORT_SDR50 | SDHCI_SUPPORT_SDR104 |
+ SDHCI_SUPPORT_DDR50);
+
ret = sdhci_setup_host(host);
if (ret)
goto pm_runtime_disable;
diff --git a/drivers/mmc/host/sdhci-tegra.c b/drivers/mmc/host/sdhci-tegra.c
index f4d4761..02d8f52 100644
--- a/drivers/mmc/host/sdhci-tegra.c
+++ b/drivers/mmc/host/sdhci-tegra.c
@@ -258,6 +258,16 @@ static void tegra210_sdhci_writew(struct sdhci_host *host, u16 val, int reg)
}
}
+static unsigned int tegra_sdhci_get_ro(struct sdhci_host *host)
+{
+ /*
+ * Write-enable shall be assumed if GPIO is missing in a board's
+ * device-tree because SDHCI's WRITE_PROTECT bit doesn't work on
+ * Tegra.
+ */
+ return mmc_gpio_get_ro(host->mmc);
+}
+
static bool tegra_sdhci_is_pad_and_regulator_valid(struct sdhci_host *host)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
@@ -1224,6 +1234,7 @@ static const struct cqhci_host_ops sdhci_tegra_cqhci_ops = {
};
static const struct sdhci_ops tegra_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
.read_w = tegra_sdhci_readw,
.write_l = tegra_sdhci_writel,
.set_clock = tegra_sdhci_set_clock,
@@ -1279,6 +1290,7 @@ static const struct sdhci_tegra_soc_data soc_data_tegra30 = {
};
static const struct sdhci_ops tegra114_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
.read_w = tegra_sdhci_readw,
.write_w = tegra_sdhci_writew,
.write_l = tegra_sdhci_writel,
@@ -1332,6 +1344,7 @@ static const struct sdhci_tegra_soc_data soc_data_tegra124 = {
};
static const struct sdhci_ops tegra210_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
.read_w = tegra_sdhci_readw,
.write_w = tegra210_sdhci_writew,
.write_l = tegra_sdhci_writel,
@@ -1366,6 +1379,7 @@ static const struct sdhci_tegra_soc_data soc_data_tegra210 = {
};
static const struct sdhci_ops tegra186_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
.read_w = tegra_sdhci_readw,
.write_l = tegra_sdhci_writel,
.set_clock = tegra_sdhci_set_clock,
diff --git a/drivers/mtd/hyperbus/Kconfig b/drivers/mtd/hyperbus/Kconfig
index b4e3caf..a4d8968 100644
--- a/drivers/mtd/hyperbus/Kconfig
+++ b/drivers/mtd/hyperbus/Kconfig
@@ -1,5 +1,6 @@
menuconfig MTD_HYPERBUS
tristate "HyperBus support"
+ depends on HAS_IOMEM
select MTD_CFI
select MTD_MAP_BANK_WIDTH_2
select MTD_CFI_AMDSTD
diff --git a/drivers/mtd/maps/sa1100-flash.c b/drivers/mtd/maps/sa1100-flash.c
index 895510d..47602af 100644
--- a/drivers/mtd/maps/sa1100-flash.c
+++ b/drivers/mtd/maps/sa1100-flash.c
@@ -81,6 +81,7 @@ static int sa1100_probe_subdev(struct sa_subdev_info *subdev, struct resource *r
default:
printk(KERN_WARNING "SA1100 flash: unknown base address "
"0x%08lx, assuming CS0\n", phys);
+ /* Fall through */
case SA1100_CS0_PHYS:
subdev->map.bankwidth = (MSC0 & MSC_RBW) ? 2 : 4;
diff --git a/drivers/net/arcnet/capmode.c b/drivers/net/arcnet/capmode.c
index b780be6..c09b567 100644
--- a/drivers/net/arcnet/capmode.c
+++ b/drivers/net/arcnet/capmode.c
@@ -44,7 +44,7 @@ static void rx(struct net_device *dev, int bufnum,
{
struct arcnet_local *lp = netdev_priv(dev);
struct sk_buff *skb;
- struct archdr *pkt = pkthdr;
+ struct archdr *pkt;
char *pktbuf, *pkthdrbuf;
int ofs;
diff --git a/drivers/net/can/dev.c b/drivers/net/can/dev.c
index 483d270..ac86be5 100644
--- a/drivers/net/can/dev.c
+++ b/drivers/net/can/dev.c
@@ -1,6 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Copyright (C) 2005 Marc Kleine-Budde, Pengutronix
+/* Copyright (C) 2005 Marc Kleine-Budde, Pengutronix
* Copyright (C) 2006 Andrey Volkov, Varma Electronics
* Copyright (C) 2008-2009 Wolfgang Grandegger <wg@grandegger.com>
*/
@@ -12,6 +11,7 @@
#include <linux/if_arp.h>
#include <linux/workqueue.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/dev.h>
#include <linux/can/skb.h>
#include <linux/can/netlink.h>
@@ -62,8 +62,7 @@ EXPORT_SYMBOL_GPL(can_len2dlc);
#define CAN_CALC_MAX_ERROR 50 /* in one-tenth of a percent */
#define CAN_CALC_SYNC_SEG 1
-/*
- * Bit-timing calculation derived from:
+/* Bit-timing calculation derived from:
*
* Code based on LinCAN sources and H8S2638 project
* Copyright 2004-2006 Pavel Pisa - DCE FELK CVUT cz
@@ -75,10 +74,11 @@ EXPORT_SYMBOL_GPL(can_len2dlc);
* registers of the CAN controller. You can find more information
* in the header file linux/can/netlink.h.
*/
-static int can_update_sample_point(const struct can_bittiming_const *btc,
- unsigned int sample_point_nominal, unsigned int tseg,
- unsigned int *tseg1_ptr, unsigned int *tseg2_ptr,
- unsigned int *sample_point_error_ptr)
+static int
+can_update_sample_point(const struct can_bittiming_const *btc,
+ unsigned int sample_point_nominal, unsigned int tseg,
+ unsigned int *tseg1_ptr, unsigned int *tseg2_ptr,
+ unsigned int *sample_point_error_ptr)
{
unsigned int sample_point_error, best_sample_point_error = UINT_MAX;
unsigned int sample_point, best_sample_point = 0;
@@ -86,7 +86,9 @@ static int can_update_sample_point(const struct can_bittiming_const *btc,
int i;
for (i = 0; i <= 1; i++) {
- tseg2 = tseg + CAN_CALC_SYNC_SEG - (sample_point_nominal * (tseg + CAN_CALC_SYNC_SEG)) / 1000 - i;
+ tseg2 = tseg + CAN_CALC_SYNC_SEG -
+ (sample_point_nominal * (tseg + CAN_CALC_SYNC_SEG)) /
+ 1000 - i;
tseg2 = clamp(tseg2, btc->tseg2_min, btc->tseg2_max);
tseg1 = tseg - tseg2;
if (tseg1 > btc->tseg1_max) {
@@ -94,10 +96,12 @@ static int can_update_sample_point(const struct can_bittiming_const *btc,
tseg2 = tseg - tseg1;
}
- sample_point = 1000 * (tseg + CAN_CALC_SYNC_SEG - tseg2) / (tseg + CAN_CALC_SYNC_SEG);
+ sample_point = 1000 * (tseg + CAN_CALC_SYNC_SEG - tseg2) /
+ (tseg + CAN_CALC_SYNC_SEG);
sample_point_error = abs(sample_point_nominal - sample_point);
- if ((sample_point <= sample_point_nominal) && (sample_point_error < best_sample_point_error)) {
+ if (sample_point <= sample_point_nominal &&
+ sample_point_error < best_sample_point_error) {
best_sample_point = sample_point;
best_sample_point_error = sample_point_error;
*tseg1_ptr = tseg1;
@@ -148,7 +152,7 @@ static int can_calc_bittiming(struct net_device *dev, struct can_bittiming *bt,
/* choose brp step which is possible in system */
brp = (brp / btc->brp_inc) * btc->brp_inc;
- if ((brp < btc->brp_min) || (brp > btc->brp_max))
+ if (brp < btc->brp_min || brp > btc->brp_max)
continue;
bitrate = priv->clock.freq / (brp * tsegall);
@@ -162,7 +166,8 @@ static int can_calc_bittiming(struct net_device *dev, struct can_bittiming *bt,
if (bitrate_error < best_bitrate_error)
best_sample_point_error = UINT_MAX;
- can_update_sample_point(btc, sample_point_nominal, tseg / 2, &tseg1, &tseg2, &sample_point_error);
+ can_update_sample_point(btc, sample_point_nominal, tseg / 2,
+ &tseg1, &tseg2, &sample_point_error);
if (sample_point_error > best_sample_point_error)
continue;
@@ -191,8 +196,9 @@ static int can_calc_bittiming(struct net_device *dev, struct can_bittiming *bt,
}
/* real sample point */
- bt->sample_point = can_update_sample_point(btc, sample_point_nominal, best_tseg,
- &tseg1, &tseg2, NULL);
+ bt->sample_point = can_update_sample_point(btc, sample_point_nominal,
+ best_tseg, &tseg1, &tseg2,
+ NULL);
v64 = (u64)best_brp * 1000 * 1000 * 1000;
do_div(v64, priv->clock.freq);
@@ -216,7 +222,8 @@ static int can_calc_bittiming(struct net_device *dev, struct can_bittiming *bt,
bt->brp = best_brp;
/* real bitrate */
- bt->bitrate = priv->clock.freq / (bt->brp * (CAN_CALC_SYNC_SEG + tseg1 + tseg2));
+ bt->bitrate = priv->clock.freq /
+ (bt->brp * (CAN_CALC_SYNC_SEG + tseg1 + tseg2));
return 0;
}
@@ -229,8 +236,7 @@ static int can_calc_bittiming(struct net_device *dev, struct can_bittiming *bt,
}
#endif /* CONFIG_CAN_CALC_BITTIMING */
-/*
- * Checks the validity of the specified bit-timing parameters prop_seg,
+/* Checks the validity of the specified bit-timing parameters prop_seg,
* phase_seg1, phase_seg2 and sjw and tries to determine the bitrate
* prescaler value brp. You can find more information in the header
* file linux/can/netlink.h.
@@ -270,9 +276,10 @@ static int can_fixup_bittiming(struct net_device *dev, struct can_bittiming *bt,
}
/* Checks the validity of predefined bitrate settings */
-static int can_validate_bitrate(struct net_device *dev, struct can_bittiming *bt,
- const u32 *bitrate_const,
- const unsigned int bitrate_const_cnt)
+static int
+can_validate_bitrate(struct net_device *dev, struct can_bittiming *bt,
+ const u32 *bitrate_const,
+ const unsigned int bitrate_const_cnt)
{
struct can_priv *priv = netdev_priv(dev);
unsigned int i;
@@ -295,8 +302,7 @@ static int can_get_bittiming(struct net_device *dev, struct can_bittiming *bt,
{
int err;
- /*
- * Depending on the given can_bittiming parameter structure the CAN
+ /* Depending on the given can_bittiming parameter structure the CAN
* timing parameters are calculated based on the provided bitrate OR
* alternatively the CAN timing parameters (tq, prop_seg, etc.) are
* provided directly which are then checked and fixed up.
@@ -397,8 +403,7 @@ void can_change_state(struct net_device *dev, struct can_frame *cf,
}
EXPORT_SYMBOL_GPL(can_change_state);
-/*
- * Local echo of CAN messages
+/* Local echo of CAN messages
*
* CAN network devices *should* support a local echo functionality
* (see Documentation/networking/can.rst). To test the handling of CAN
@@ -423,8 +428,7 @@ static void can_flush_echo_skb(struct net_device *dev)
}
}
-/*
- * Put the skb on the stack to be looped backed locally lateron
+/* Put the skb on the stack to be looped backed locally lateron
*
* The function is typically called in the start_xmit function
* of the device driver. The driver must protect access to
@@ -446,7 +450,6 @@ void can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
}
if (!priv->echo_skb[idx]) {
-
skb = can_create_echo_skb(skb);
if (!skb)
return;
@@ -466,7 +469,8 @@ void can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
}
EXPORT_SYMBOL_GPL(can_put_echo_skb);
-struct sk_buff *__can_get_echo_skb(struct net_device *dev, unsigned int idx, u8 *len_ptr)
+struct sk_buff *
+__can_get_echo_skb(struct net_device *dev, unsigned int idx, u8 *len_ptr)
{
struct can_priv *priv = netdev_priv(dev);
@@ -493,8 +497,7 @@ struct sk_buff *__can_get_echo_skb(struct net_device *dev, unsigned int idx, u8
return NULL;
}
-/*
- * Get the skb from the stack and loop it back locally
+/* Get the skb from the stack and loop it back locally
*
* The function is typically called when the TX done interrupt
* is handled in the device driver. The driver must protect
@@ -515,11 +518,10 @@ unsigned int can_get_echo_skb(struct net_device *dev, unsigned int idx)
}
EXPORT_SYMBOL_GPL(can_get_echo_skb);
-/*
- * Remove the skb from the stack and free it.
- *
- * The function is typically called when TX failed.
- */
+/* Remove the skb from the stack and free it.
+ *
+ * The function is typically called when TX failed.
+ */
void can_free_echo_skb(struct net_device *dev, unsigned int idx)
{
struct can_priv *priv = netdev_priv(dev);
@@ -533,9 +535,7 @@ void can_free_echo_skb(struct net_device *dev, unsigned int idx)
}
EXPORT_SYMBOL_GPL(can_free_echo_skb);
-/*
- * CAN device restart for bus-off recovery
- */
+/* CAN device restart for bus-off recovery */
static void can_restart(struct net_device *dev)
{
struct can_priv *priv = netdev_priv(dev);
@@ -546,15 +546,14 @@ static void can_restart(struct net_device *dev)
BUG_ON(netif_carrier_ok(dev));
- /*
- * No synchronization needed because the device is bus-off and
+ /* No synchronization needed because the device is bus-off and
* no messages can come in or go out.
*/
can_flush_echo_skb(dev);
/* send restart message upstream */
skb = alloc_can_err_skb(dev, &cf);
- if (skb == NULL) {
+ if (!skb) {
err = -ENOMEM;
goto restart;
}
@@ -580,7 +579,8 @@ static void can_restart(struct net_device *dev)
static void can_restart_work(struct work_struct *work)
{
struct delayed_work *dwork = to_delayed_work(work);
- struct can_priv *priv = container_of(dwork, struct can_priv, restart_work);
+ struct can_priv *priv = container_of(dwork, struct can_priv,
+ restart_work);
can_restart(priv->dev);
}
@@ -589,8 +589,7 @@ int can_restart_now(struct net_device *dev)
{
struct can_priv *priv = netdev_priv(dev);
- /*
- * A manual restart is only permitted if automatic restart is
+ /* A manual restart is only permitted if automatic restart is
* disabled and the device is in the bus-off state
*/
if (priv->restart_ms)
@@ -604,8 +603,7 @@ int can_restart_now(struct net_device *dev)
return 0;
}
-/*
- * CAN bus-off
+/* CAN bus-off
*
* This functions should be called when the device goes bus-off to
* tell the netif layer that no more packets can be sent or received.
@@ -708,9 +706,7 @@ struct sk_buff *alloc_can_err_skb(struct net_device *dev, struct can_frame **cf)
}
EXPORT_SYMBOL_GPL(alloc_can_err_skb);
-/*
- * Allocate and setup space for the CAN network device
- */
+/* Allocate and setup space for the CAN network device */
struct net_device *alloc_candev_mqs(int sizeof_priv, unsigned int echo_skb_max,
unsigned int txqs, unsigned int rxqs)
{
@@ -718,11 +714,24 @@ struct net_device *alloc_candev_mqs(int sizeof_priv, unsigned int echo_skb_max,
struct can_priv *priv;
int size;
+ /* We put the driver's priv, the CAN mid layer priv and the
+ * echo skb into the netdevice's priv. The memory layout for
+ * the netdev_priv is like this:
+ *
+ * +-------------------------+
+ * | driver's priv |
+ * +-------------------------+
+ * | struct can_ml_priv |
+ * +-------------------------+
+ * | array of struct sk_buff |
+ * +-------------------------+
+ */
+
+ size = ALIGN(sizeof_priv, NETDEV_ALIGN) + sizeof(struct can_ml_priv);
+
if (echo_skb_max)
- size = ALIGN(sizeof_priv, sizeof(struct sk_buff *)) +
+ size = ALIGN(size, sizeof(struct sk_buff *)) +
echo_skb_max * sizeof(struct sk_buff *);
- else
- size = sizeof_priv;
dev = alloc_netdev_mqs(size, "can%d", NET_NAME_UNKNOWN, can_setup,
txqs, rxqs);
@@ -732,10 +741,12 @@ struct net_device *alloc_candev_mqs(int sizeof_priv, unsigned int echo_skb_max,
priv = netdev_priv(dev);
priv->dev = dev;
+ dev->ml_priv = (void *)priv + ALIGN(sizeof_priv, NETDEV_ALIGN);
+
if (echo_skb_max) {
priv->echo_skb_max = echo_skb_max;
priv->echo_skb = (void *)priv +
- ALIGN(sizeof_priv, sizeof(struct sk_buff *));
+ (size - echo_skb_max * sizeof(struct sk_buff *));
}
priv->state = CAN_STATE_STOPPED;
@@ -746,18 +757,14 @@ struct net_device *alloc_candev_mqs(int sizeof_priv, unsigned int echo_skb_max,
}
EXPORT_SYMBOL_GPL(alloc_candev_mqs);
-/*
- * Free space of the CAN network device
- */
+/* Free space of the CAN network device */
void free_candev(struct net_device *dev)
{
free_netdev(dev);
}
EXPORT_SYMBOL_GPL(free_candev);
-/*
- * changing MTU and control mode for CAN/CANFD devices
- */
+/* changing MTU and control mode for CAN/CANFD devices */
int can_change_mtu(struct net_device *dev, int new_mtu)
{
struct can_priv *priv = netdev_priv(dev);
@@ -794,8 +801,7 @@ int can_change_mtu(struct net_device *dev, int new_mtu)
}
EXPORT_SYMBOL_GPL(can_change_mtu);
-/*
- * Common open function when the device gets opened.
+/* Common open function when the device gets opened.
*
* This function should be called in the open function of the device
* driver.
@@ -812,7 +818,7 @@ int open_candev(struct net_device *dev)
/* For CAN FD the data bitrate has to be >= the arbitration bitrate */
if ((priv->ctrlmode & CAN_CTRLMODE_FD) &&
(!priv->data_bittiming.bitrate ||
- (priv->data_bittiming.bitrate < priv->bittiming.bitrate))) {
+ priv->data_bittiming.bitrate < priv->bittiming.bitrate)) {
netdev_err(dev, "incorrect/missing data bit-timing\n");
return -EINVAL;
}
@@ -848,8 +854,7 @@ void of_can_transceiver(struct net_device *dev)
EXPORT_SYMBOL_GPL(of_can_transceiver);
#endif
-/*
- * Common close function for cleanup before the device gets closed.
+/* Common close function for cleanup before the device gets closed.
*
* This function should be called in the close function of the device
* driver.
@@ -863,9 +868,7 @@ void close_candev(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(close_candev);
-/*
- * CAN netlink interface
- */
+/* CAN netlink interface */
static const struct nla_policy can_policy[IFLA_CAN_MAX + 1] = {
[IFLA_CAN_STATE] = { .type = NLA_U32 },
[IFLA_CAN_CTRLMODE] = { .len = sizeof(struct can_ctrlmode) },
@@ -1209,7 +1212,6 @@ static int can_newlink(struct net *src_net, struct net_device *dev,
static void can_dellink(struct net_device *dev, struct list_head *head)
{
- return;
}
static struct rtnl_link_ops can_link_ops __read_mostly = {
@@ -1227,9 +1229,7 @@ static struct rtnl_link_ops can_link_ops __read_mostly = {
.fill_xstats = can_fill_xstats,
};
-/*
- * Register the CAN network device
- */
+/* Register the CAN network device */
int register_candev(struct net_device *dev)
{
struct can_priv *priv = netdev_priv(dev);
@@ -1255,22 +1255,19 @@ int register_candev(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(register_candev);
-/*
- * Unregister the CAN network device
- */
+/* Unregister the CAN network device */
void unregister_candev(struct net_device *dev)
{
unregister_netdev(dev);
}
EXPORT_SYMBOL_GPL(unregister_candev);
-/*
- * Test if a network device is a candev based device
+/* Test if a network device is a candev based device
* and return the can_priv* if so.
*/
struct can_priv *safe_candev_priv(struct net_device *dev)
{
- if ((dev->type != ARPHRD_CAN) || (dev->rtnl_link_ops != &can_link_ops))
+ if (dev->type != ARPHRD_CAN || dev->rtnl_link_ops != &can_link_ops)
return NULL;
return netdev_priv(dev);
@@ -1285,7 +1282,7 @@ static __init int can_dev_init(void)
err = rtnl_link_register(&can_link_ops);
if (!err)
- printk(KERN_INFO MOD_DESC "\n");
+ pr_info(MOD_DESC "\n");
return err;
}
diff --git a/drivers/net/can/kvaser_pciefd.c b/drivers/net/can/kvaser_pciefd.c
index f9815fd..6f76691 100644
--- a/drivers/net/can/kvaser_pciefd.c
+++ b/drivers/net/can/kvaser_pciefd.c
@@ -65,6 +65,7 @@ MODULE_DESCRIPTION("CAN driver for Kvaser CAN/PCIe devices");
#define KVASER_PCIEFD_SYSID_BASE 0x1f020
#define KVASER_PCIEFD_SYSID_VERSION_REG (KVASER_PCIEFD_SYSID_BASE + 0x8)
#define KVASER_PCIEFD_SYSID_CANFREQ_REG (KVASER_PCIEFD_SYSID_BASE + 0xc)
+#define KVASER_PCIEFD_SYSID_BUSFREQ_REG (KVASER_PCIEFD_SYSID_BASE + 0x10)
#define KVASER_PCIEFD_SYSID_BUILD_REG (KVASER_PCIEFD_SYSID_BASE + 0x14)
/* Shared receive buffer registers */
#define KVASER_PCIEFD_SRB_BASE 0x1f200
@@ -268,6 +269,7 @@ struct kvaser_pciefd {
struct kvaser_pciefd_can *can[KVASER_PCIEFD_MAX_CAN_CHANNELS];
void *dma_data[KVASER_PCIEFD_DMA_COUNT];
u8 nr_channels;
+ u32 bus_freq;
u32 freq;
u32 freq_to_ticks_div;
};
@@ -666,7 +668,7 @@ static void kvaser_pciefd_pwm_start(struct kvaser_pciefd_can *can)
spin_lock_irqsave(&can->lock, irq);
/* Set frequency to 500 KHz*/
- top = can->can.clock.freq / (2 * 500000) - 1;
+ top = can->kv_pcie->bus_freq / (2 * 500000) - 1;
pwm_ctrl = top & 0xff;
pwm_ctrl |= (top & 0xff) << KVASER_PCIEFD_KCAN_PWM_TOP_SHIFT;
@@ -1119,6 +1121,8 @@ static int kvaser_pciefd_setup_board(struct kvaser_pciefd *pcie)
return -ENODEV;
}
+ pcie->bus_freq = ioread32(pcie->reg_base +
+ KVASER_PCIEFD_SYSID_BUSFREQ_REG);
pcie->freq = ioread32(pcie->reg_base + KVASER_PCIEFD_SYSID_CANFREQ_REG);
pcie->freq_to_ticks_div = pcie->freq / 1000000;
if (pcie->freq_to_ticks_div == 0)
diff --git a/drivers/net/can/m_can/tcan4x5x.c b/drivers/net/can/m_can/tcan4x5x.c
index a697996d..3db6192 100644
--- a/drivers/net/can/m_can/tcan4x5x.c
+++ b/drivers/net/can/m_can/tcan4x5x.c
@@ -117,7 +117,6 @@ struct tcan4x5x_priv {
struct m_can_classdev *mcan_dev;
struct gpio_desc *reset_gpio;
- struct gpio_desc *interrupt_gpio;
struct gpio_desc *device_wake_gpio;
struct gpio_desc *device_state_gpio;
struct regulator *power;
@@ -236,8 +235,6 @@ static u32 tcan4x5x_read_reg(struct m_can_classdev *cdev, int reg)
struct tcan4x5x_priv *priv = cdev->device_data;
u32 val;
- tcan4x5x_check_wake(priv);
-
regmap_read(priv->regmap, priv->reg_offset + reg, &val);
return val;
@@ -248,8 +245,6 @@ static u32 tcan4x5x_read_fifo(struct m_can_classdev *cdev, int addr_offset)
struct tcan4x5x_priv *priv = cdev->device_data;
u32 val;
- tcan4x5x_check_wake(priv);
-
regmap_read(priv->regmap, priv->mram_start + addr_offset, &val);
return val;
@@ -259,8 +254,6 @@ static int tcan4x5x_write_reg(struct m_can_classdev *cdev, int reg, int val)
{
struct tcan4x5x_priv *priv = cdev->device_data;
- tcan4x5x_check_wake(priv);
-
return regmap_write(priv->regmap, priv->reg_offset + reg, val);
}
@@ -269,8 +262,6 @@ static int tcan4x5x_write_fifo(struct m_can_classdev *cdev,
{
struct tcan4x5x_priv *priv = cdev->device_data;
- tcan4x5x_check_wake(priv);
-
return regmap_write(priv->regmap, priv->mram_start + addr_offset, val);
}
@@ -290,18 +281,13 @@ static int tcan4x5x_write_tcan_reg(struct m_can_classdev *cdev,
{
struct tcan4x5x_priv *priv = cdev->device_data;
- tcan4x5x_check_wake(priv);
-
return regmap_write(priv->regmap, reg, val);
}
static int tcan4x5x_clear_interrupts(struct m_can_classdev *cdev)
{
- struct tcan4x5x_priv *tcan4x5x = cdev->device_data;
int ret;
- tcan4x5x_check_wake(tcan4x5x);
-
ret = tcan4x5x_write_tcan_reg(cdev, TCAN4X5X_STATUS,
TCAN4X5X_CLEAR_ALL_INT);
if (ret)
@@ -356,13 +342,6 @@ static int tcan4x5x_parse_config(struct m_can_classdev *cdev)
{
struct tcan4x5x_priv *tcan4x5x = cdev->device_data;
- tcan4x5x->interrupt_gpio = devm_gpiod_get(cdev->dev, "data-ready",
- GPIOD_IN);
- if (IS_ERR(tcan4x5x->interrupt_gpio)) {
- dev_err(cdev->dev, "data-ready gpio not defined\n");
- return -EINVAL;
- }
-
tcan4x5x->device_wake_gpio = devm_gpiod_get(cdev->dev, "device-wake",
GPIOD_OUT_HIGH);
if (IS_ERR(tcan4x5x->device_wake_gpio)) {
@@ -381,8 +360,6 @@ static int tcan4x5x_parse_config(struct m_can_classdev *cdev)
if (IS_ERR(tcan4x5x->device_state_gpio))
tcan4x5x->device_state_gpio = NULL;
- cdev->net->irq = gpiod_to_irq(tcan4x5x->interrupt_gpio);
-
tcan4x5x->power = devm_regulator_get_optional(cdev->dev,
"vsup");
if (PTR_ERR(tcan4x5x->power) == -EPROBE_DEFER)
@@ -447,6 +424,7 @@ static int tcan4x5x_can_probe(struct spi_device *spi)
mcan_class->is_peripheral = true;
mcan_class->bit_timing = &tcan4x5x_bittiming_const;
mcan_class->data_timing = &tcan4x5x_data_bittiming_const;
+ mcan_class->net->irq = spi->irq;
spi_set_drvdata(spi, priv);
diff --git a/drivers/net/can/slcan.c b/drivers/net/can/slcan.c
index aa97dbc..bb60322 100644
--- a/drivers/net/can/slcan.c
+++ b/drivers/net/can/slcan.c
@@ -55,6 +55,7 @@
#include <linux/workqueue.h>
#include <linux/can.h>
#include <linux/can/skb.h>
+#include <linux/can/can-ml.h>
MODULE_ALIAS_LDISC(N_SLCAN);
MODULE_DESCRIPTION("serial line CAN interface");
@@ -514,6 +515,7 @@ static struct slcan *slc_alloc(void)
char name[IFNAMSIZ];
struct net_device *dev = NULL;
struct slcan *sl;
+ int size;
for (i = 0; i < maxdev; i++) {
dev = slcan_devs[i];
@@ -527,12 +529,14 @@ static struct slcan *slc_alloc(void)
return NULL;
sprintf(name, "slcan%d", i);
- dev = alloc_netdev(sizeof(*sl), name, NET_NAME_UNKNOWN, slc_setup);
+ size = ALIGN(sizeof(*sl), NETDEV_ALIGN) + sizeof(struct can_ml_priv);
+ dev = alloc_netdev(size, name, NET_NAME_UNKNOWN, slc_setup);
if (!dev)
return NULL;
dev->base_addr = i;
sl = netdev_priv(dev);
+ dev->ml_priv = (void *)sl + ALIGN(sizeof(*sl), NETDEV_ALIGN);
/* Initialize channel control data */
sl->magic = SLCAN_MAGIC;
diff --git a/drivers/net/can/spi/hi311x.c b/drivers/net/can/spi/hi311x.c
index 28badace..73d48c3 100644
--- a/drivers/net/can/spi/hi311x.c
+++ b/drivers/net/can/spi/hi311x.c
@@ -177,8 +177,7 @@ static void hi3110_clean(struct net_device *net)
if (priv->tx_skb || priv->tx_len)
net->stats.tx_errors++;
- if (priv->tx_skb)
- dev_kfree_skb(priv->tx_skb);
+ dev_kfree_skb(priv->tx_skb);
if (priv->tx_len)
can_free_echo_skb(priv->net, 0);
priv->tx_skb = NULL;
diff --git a/drivers/net/can/spi/mcp251x.c b/drivers/net/can/spi/mcp251x.c
index 05547dd..bee9f7b 100644
--- a/drivers/net/can/spi/mcp251x.c
+++ b/drivers/net/can/spi/mcp251x.c
@@ -17,26 +17,6 @@
* - Sascha Hauer, Marc Kleine-Budde, Pengutronix
* - Simon Kallweit, intefo AG
* Copyright 2007
- *
- * Your platform definition file should specify something like:
- *
- * static struct mcp251x_platform_data mcp251x_info = {
- * .oscillator_frequency = 8000000,
- * };
- *
- * static struct spi_board_info spi_board_info[] = {
- * {
- * .modalias = "mcp2510",
- * // "mcp2515" or "mcp25625" depending on your controller
- * .platform_data = &mcp251x_info,
- * .irq = IRQ_EINT13,
- * .max_speed_hz = 2*1000*1000,
- * .chip_select = 2,
- * },
- * };
- *
- * Please see mcp251x.h for a description of the fields in
- * struct mcp251x_platform_data.
*/
#include <linux/can/core.h>
@@ -53,8 +33,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/netdevice.h>
-#include <linux/of.h>
-#include <linux/of_device.h>
+#include <linux/property.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/spi/spi.h>
@@ -264,8 +243,7 @@ static void mcp251x_clean(struct net_device *net)
if (priv->tx_skb || priv->tx_len)
net->stats.tx_errors++;
- if (priv->tx_skb)
- dev_kfree_skb(priv->tx_skb);
+ dev_kfree_skb(priv->tx_skb);
if (priv->tx_len)
can_free_echo_skb(priv->net, 0);
priv->tx_skb = NULL;
@@ -915,7 +893,7 @@ static int mcp251x_open(struct net_device *net)
priv->tx_skb = NULL;
priv->tx_len = 0;
- if (!spi->dev.of_node)
+ if (!dev_fwnode(&spi->dev))
flags = IRQF_TRIGGER_FALLING;
ret = request_threaded_irq(spi->irq, NULL, mcp251x_can_ist,
@@ -1007,23 +985,20 @@ MODULE_DEVICE_TABLE(spi, mcp251x_id_table);
static int mcp251x_can_probe(struct spi_device *spi)
{
- const struct of_device_id *of_id = of_match_device(mcp251x_of_match,
- &spi->dev);
+ const void *match = device_get_match_data(&spi->dev);
struct mcp251x_platform_data *pdata = dev_get_platdata(&spi->dev);
struct net_device *net;
struct mcp251x_priv *priv;
struct clk *clk;
int freq, ret;
- clk = devm_clk_get(&spi->dev, NULL);
- if (IS_ERR(clk)) {
- if (pdata)
- freq = pdata->oscillator_frequency;
- else
- return PTR_ERR(clk);
- } else {
- freq = clk_get_rate(clk);
- }
+ clk = devm_clk_get_optional(&spi->dev, NULL);
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
+
+ freq = clk_get_rate(clk);
+ if (freq == 0 && pdata)
+ freq = pdata->oscillator_frequency;
/* Sanity check */
if (freq < 1000000 || freq > 25000000)
@@ -1034,11 +1009,9 @@ static int mcp251x_can_probe(struct spi_device *spi)
if (!net)
return -ENOMEM;
- if (!IS_ERR(clk)) {
- ret = clk_prepare_enable(clk);
- if (ret)
- goto out_free;
- }
+ ret = clk_prepare_enable(clk);
+ if (ret)
+ goto out_free;
net->netdev_ops = &mcp251x_netdev_ops;
net->flags |= IFF_ECHO;
@@ -1049,8 +1022,8 @@ static int mcp251x_can_probe(struct spi_device *spi)
priv->can.clock.freq = freq / 2;
priv->can.ctrlmode_supported = CAN_CTRLMODE_3_SAMPLES |
CAN_CTRLMODE_LOOPBACK | CAN_CTRLMODE_LISTENONLY;
- if (of_id)
- priv->model = (enum mcp251x_model)of_id->data;
+ if (match)
+ priv->model = (enum mcp251x_model)match;
else
priv->model = spi_get_device_id(spi)->driver_data;
priv->net = net;
@@ -1123,8 +1096,7 @@ static int mcp251x_can_probe(struct spi_device *spi)
mcp251x_power_enable(priv->power, 0);
out_clk:
- if (!IS_ERR(clk))
- clk_disable_unprepare(clk);
+ clk_disable_unprepare(clk);
out_free:
free_candev(net);
@@ -1142,8 +1114,7 @@ static int mcp251x_can_remove(struct spi_device *spi)
mcp251x_power_enable(priv->power, 0);
- if (!IS_ERR(priv->clk))
- clk_disable_unprepare(priv->clk);
+ clk_disable_unprepare(priv->clk);
free_candev(net);
@@ -1171,10 +1142,8 @@ static int __maybe_unused mcp251x_can_suspend(struct device *dev)
priv->after_suspend = AFTER_SUSPEND_DOWN;
}
- if (!IS_ERR_OR_NULL(priv->power)) {
- regulator_disable(priv->power);
- priv->after_suspend |= AFTER_SUSPEND_POWER;
- }
+ mcp251x_power_enable(priv->power, 0);
+ priv->after_suspend |= AFTER_SUSPEND_POWER;
return 0;
}
diff --git a/drivers/net/can/vcan.c b/drivers/net/can/vcan.c
index daf2713..39ca14b 100644
--- a/drivers/net/can/vcan.c
+++ b/drivers/net/can/vcan.c
@@ -46,6 +46,7 @@
#include <linux/if_arp.h>
#include <linux/if_ether.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/dev.h>
#include <linux/can/skb.h>
#include <linux/slab.h>
@@ -152,6 +153,7 @@ static void vcan_setup(struct net_device *dev)
dev->addr_len = 0;
dev->tx_queue_len = 0;
dev->flags = IFF_NOARP;
+ dev->ml_priv = netdev_priv(dev);
/* set flags according to driver capabilities */
if (echo)
@@ -162,8 +164,9 @@ static void vcan_setup(struct net_device *dev)
}
static struct rtnl_link_ops vcan_link_ops __read_mostly = {
- .kind = DRV_NAME,
- .setup = vcan_setup,
+ .kind = DRV_NAME,
+ .priv_size = sizeof(struct can_ml_priv),
+ .setup = vcan_setup,
};
static __init int vcan_init_module(void)
diff --git a/drivers/net/can/vxcan.c b/drivers/net/can/vxcan.c
index b210629..d6ba942 100644
--- a/drivers/net/can/vxcan.c
+++ b/drivers/net/can/vxcan.c
@@ -18,6 +18,7 @@
#include <linux/can/dev.h>
#include <linux/can/skb.h>
#include <linux/can/vxcan.h>
+#include <linux/can/can-ml.h>
#include <linux/slab.h>
#include <net/rtnetlink.h>
@@ -146,6 +147,7 @@ static void vxcan_setup(struct net_device *dev)
dev->flags = (IFF_NOARP|IFF_ECHO);
dev->netdev_ops = &vxcan_netdev_ops;
dev->needs_free_netdev = true;
+ dev->ml_priv = netdev_priv(dev) + ALIGN(sizeof(struct vxcan_priv), NETDEV_ALIGN);
}
/* forward declaration for rtnl_create_link() */
@@ -281,7 +283,7 @@ static struct net *vxcan_get_link_net(const struct net_device *dev)
static struct rtnl_link_ops vxcan_link_ops = {
.kind = DRV_NAME,
- .priv_size = sizeof(struct vxcan_priv),
+ .priv_size = ALIGN(sizeof(struct vxcan_priv), NETDEV_ALIGN) + sizeof(struct can_ml_priv),
.setup = vxcan_setup,
.newlink = vxcan_newlink,
.dellink = vxcan_dellink,
diff --git a/drivers/net/dsa/bcm_sf2.c b/drivers/net/dsa/bcm_sf2.c
index 4f83934..26509fa 100644
--- a/drivers/net/dsa/bcm_sf2.c
+++ b/drivers/net/dsa/bcm_sf2.c
@@ -481,6 +481,7 @@ static void bcm_sf2_sw_validate(struct dsa_switch *ds, int port,
unsigned long *supported,
struct phylink_link_state *state)
{
+ struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds);
__ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
if (!phy_interface_mode_is_rgmii(state->interface) &&
@@ -490,8 +491,10 @@ static void bcm_sf2_sw_validate(struct dsa_switch *ds, int port,
state->interface != PHY_INTERFACE_MODE_INTERNAL &&
state->interface != PHY_INTERFACE_MODE_MOCA) {
bitmap_zero(supported, __ETHTOOL_LINK_MODE_MASK_NBITS);
- dev_err(ds->dev,
- "Unsupported interface: %d\n", state->interface);
+ if (port != core_readl(priv, CORE_IMP0_PRT_ID))
+ dev_err(ds->dev,
+ "Unsupported interface: %d for port %d\n",
+ state->interface, port);
return;
}
@@ -529,6 +532,9 @@ static void bcm_sf2_sw_mac_config(struct dsa_switch *ds, int port,
u32 id_mode_dis = 0, port_mode;
u32 reg, offset;
+ if (port == core_readl(priv, CORE_IMP0_PRT_ID))
+ return;
+
if (priv->type == BCM7445_DEVICE_ID)
offset = CORE_STS_OVERRIDE_GMIIP_PORT(port);
else
diff --git a/drivers/net/dsa/microchip/ksz9477_spi.c b/drivers/net/dsa/microchip/ksz9477_spi.c
index a226b38..2e402e4 100644
--- a/drivers/net/dsa/microchip/ksz9477_spi.c
+++ b/drivers/net/dsa/microchip/ksz9477_spi.c
@@ -80,6 +80,7 @@ static const struct of_device_id ksz9477_dt_ids[] = {
{ .compatible = "microchip,ksz9897" },
{ .compatible = "microchip,ksz9893" },
{ .compatible = "microchip,ksz9563" },
+ { .compatible = "microchip,ksz8563" },
{},
};
MODULE_DEVICE_TABLE(of, ksz9477_dt_ids);
diff --git a/drivers/net/dsa/microchip/ksz_common.h b/drivers/net/dsa/microchip/ksz_common.h
index c44a8d2..13d027b 100644
--- a/drivers/net/dsa/microchip/ksz_common.h
+++ b/drivers/net/dsa/microchip/ksz_common.h
@@ -299,6 +299,7 @@ static inline void ksz_pwrite32(struct ksz_device *dev, int port, int offset,
#define KSZ_REGMAP_ENTRY(width, swp, regbits, regpad, regalign) \
{ \
+ .name = #width, \
.val_bits = (width), \
.reg_stride = (width) / 8, \
.reg_bits = (regbits) + (regalign), \
diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index c48e294..1d8d36d 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -13,7 +13,7 @@
#include <linux/of_mdio.h>
#include <linux/of_net.h>
#include <linux/of_platform.h>
-#include <linux/phy.h>
+#include <linux/phylink.h>
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
@@ -633,61 +633,75 @@ mt7530_get_sset_count(struct dsa_switch *ds, int port, int sset)
return ARRAY_SIZE(mt7530_mib);
}
-static void mt7530_adjust_link(struct dsa_switch *ds, int port,
- struct phy_device *phydev)
+static void mt7530_setup_port5(struct dsa_switch *ds, phy_interface_t interface)
{
struct mt7530_priv *priv = ds->priv;
+ u8 tx_delay = 0;
+ int val;
- if (phy_is_pseudo_fixed_link(phydev)) {
- dev_dbg(priv->dev, "phy-mode for master device = %x\n",
- phydev->interface);
+ mutex_lock(&priv->reg_mutex);
- /* Setup TX circuit incluing relevant PAD and driving */
- mt7530_pad_clk_setup(ds, phydev->interface);
+ val = mt7530_read(priv, MT7530_MHWTRAP);
- if (priv->id == ID_MT7530) {
- /* Setup RX circuit, relevant PAD and driving on the
- * host which must be placed after the setup on the
- * device side is all finished.
- */
- mt7623_pad_clk_setup(ds);
- }
- } else {
- u16 lcl_adv = 0, rmt_adv = 0;
- u8 flowctrl;
- u32 mcr = PMCR_USERP_LINK | PMCR_FORCE_MODE;
+ val |= MHWTRAP_MANUAL | MHWTRAP_P5_MAC_SEL | MHWTRAP_P5_DIS;
+ val &= ~MHWTRAP_P5_RGMII_MODE & ~MHWTRAP_PHY0_SEL;
- switch (phydev->speed) {
- case SPEED_1000:
- mcr |= PMCR_FORCE_SPEED_1000;
- break;
- case SPEED_100:
- mcr |= PMCR_FORCE_SPEED_100;
- break;
- }
+ switch (priv->p5_intf_sel) {
+ case P5_INTF_SEL_PHY_P0:
+ /* MT7530_P5_MODE_GPHY_P0: 2nd GMAC -> P5 -> P0 */
+ val |= MHWTRAP_PHY0_SEL;
+ /* fall through */
+ case P5_INTF_SEL_PHY_P4:
+ /* MT7530_P5_MODE_GPHY_P4: 2nd GMAC -> P5 -> P4 */
+ val &= ~MHWTRAP_P5_MAC_SEL & ~MHWTRAP_P5_DIS;
- if (phydev->link)
- mcr |= PMCR_FORCE_LNK;
-
- if (phydev->duplex) {
- mcr |= PMCR_FORCE_FDX;
-
- if (phydev->pause)
- rmt_adv = LPA_PAUSE_CAP;
- if (phydev->asym_pause)
- rmt_adv |= LPA_PAUSE_ASYM;
-
- lcl_adv = linkmode_adv_to_lcl_adv_t(
- phydev->advertising);
- flowctrl = mii_resolve_flowctrl_fdx(lcl_adv, rmt_adv);
-
- if (flowctrl & FLOW_CTRL_TX)
- mcr |= PMCR_TX_FC_EN;
- if (flowctrl & FLOW_CTRL_RX)
- mcr |= PMCR_RX_FC_EN;
- }
- mt7530_write(priv, MT7530_PMCR_P(port), mcr);
+ /* Setup the MAC by default for the cpu port */
+ mt7530_write(priv, MT7530_PMCR_P(5), 0x56300);
+ break;
+ case P5_INTF_SEL_GMAC5:
+ /* MT7530_P5_MODE_GMAC: P5 -> External phy or 2nd GMAC */
+ val &= ~MHWTRAP_P5_DIS;
+ break;
+ case P5_DISABLED:
+ interface = PHY_INTERFACE_MODE_NA;
+ break;
+ default:
+ dev_err(ds->dev, "Unsupported p5_intf_sel %d\n",
+ priv->p5_intf_sel);
+ goto unlock_exit;
}
+
+ /* Setup RGMII settings */
+ if (phy_interface_mode_is_rgmii(interface)) {
+ val |= MHWTRAP_P5_RGMII_MODE;
+
+ /* P5 RGMII RX Clock Control: delay setting for 1000M */
+ mt7530_write(priv, MT7530_P5RGMIIRXCR, CSR_RGMII_EDGE_ALIGN);
+
+ /* Don't set delay in DSA mode */
+ if (!dsa_is_dsa_port(priv->ds, 5) &&
+ (interface == PHY_INTERFACE_MODE_RGMII_TXID ||
+ interface == PHY_INTERFACE_MODE_RGMII_ID))
+ tx_delay = 4; /* n * 0.5 ns */
+
+ /* P5 RGMII TX Clock Control: delay x */
+ mt7530_write(priv, MT7530_P5RGMIITXCR,
+ CSR_RGMII_TXC_CFG(0x10 + tx_delay));
+
+ /* reduce P5 RGMII Tx driving, 8mA */
+ mt7530_write(priv, MT7530_IO_DRV_CR,
+ P5_IO_CLK_DRV(1) | P5_IO_DATA_DRV(1));
+ }
+
+ mt7530_write(priv, MT7530_MHWTRAP, val);
+
+ dev_dbg(ds->dev, "Setup P5, HWTRAP=0x%x, intf_sel=%s, phy-mode=%s\n",
+ val, p5_intf_modes(priv->p5_intf_sel), phy_modes(interface));
+
+ priv->p5_interface = interface;
+
+unlock_exit:
+ mutex_unlock(&priv->reg_mutex);
}
static int
@@ -698,9 +712,6 @@ mt7530_cpu_port_enable(struct mt7530_priv *priv,
mt7530_write(priv, MT7530_PVC_P(port),
PORT_SPEC_TAG);
- /* Setup the MAC by default for the cpu port */
- mt7530_write(priv, MT7530_PMCR_P(port), PMCR_CPUP_LINK);
-
/* Disable auto learning on the cpu port */
mt7530_set(priv, MT7530_PSC_P(port), SA_DIS);
@@ -731,9 +742,6 @@ mt7530_port_enable(struct dsa_switch *ds, int port,
mutex_lock(&priv->reg_mutex);
- /* Setup the MAC for the user port */
- mt7530_write(priv, MT7530_PMCR_P(port), PMCR_USERP_LINK);
-
/* Allow the user port gets connected to the cpu port and also
* restore the port matrix if the port is the member of a certain
* bridge.
@@ -742,7 +750,7 @@ mt7530_port_enable(struct dsa_switch *ds, int port,
priv->ports[port].enable = true;
mt7530_rmw(priv, MT7530_PCR_P(port), PCR_MATRIX_MASK,
priv->ports[port].pm);
- mt7530_port_set_status(priv, port, 1);
+ mt7530_port_set_status(priv, port, 0);
mutex_unlock(&priv->reg_mutex);
@@ -1232,10 +1240,13 @@ static int
mt7530_setup(struct dsa_switch *ds)
{
struct mt7530_priv *priv = ds->priv;
- int ret, i;
- u32 id, val;
- struct device_node *dn;
+ struct device_node *phy_node;
+ struct device_node *mac_np;
struct mt7530_dummy_poll p;
+ phy_interface_t interface;
+ struct device_node *dn;
+ u32 id, val;
+ int ret, i;
/* The parent node of master netdev which holds the common system
* controller also is the container for two GMACs nodes representing
@@ -1305,6 +1316,8 @@ mt7530_setup(struct dsa_switch *ds)
val |= MHWTRAP_MANUAL;
mt7530_write(priv, MT7530_MHWTRAP, val);
+ priv->p6_interface = PHY_INTERFACE_MODE_NA;
+
/* Enable and reset MIB counters */
mt7530_mib_reset(ds);
@@ -1321,6 +1334,40 @@ mt7530_setup(struct dsa_switch *ds)
mt7530_port_disable(ds, i);
}
+ /* Setup port 5 */
+ priv->p5_intf_sel = P5_DISABLED;
+ interface = PHY_INTERFACE_MODE_NA;
+
+ if (!dsa_is_unused_port(ds, 5)) {
+ priv->p5_intf_sel = P5_INTF_SEL_GMAC5;
+ interface = of_get_phy_mode(ds->ports[5].dn);
+ } else {
+ /* Scan the ethernet nodes. look for GMAC1, lookup used phy */
+ for_each_child_of_node(dn, mac_np) {
+ if (!of_device_is_compatible(mac_np,
+ "mediatek,eth-mac"))
+ continue;
+
+ ret = of_property_read_u32(mac_np, "reg", &id);
+ if (ret < 0 || id != 1)
+ continue;
+
+ phy_node = of_parse_phandle(mac_np, "phy-handle", 0);
+ if (phy_node->parent == priv->dev->of_node->parent) {
+ interface = of_get_phy_mode(mac_np);
+ id = of_mdio_parse_addr(ds->dev, phy_node);
+ if (id == 0)
+ priv->p5_intf_sel = P5_INTF_SEL_PHY_P0;
+ if (id == 4)
+ priv->p5_intf_sel = P5_INTF_SEL_PHY_P4;
+ }
+ of_node_put(phy_node);
+ break;
+ }
+ }
+
+ mt7530_setup_port5(ds, interface);
+
/* Flush the FDB table */
ret = mt7530_fdb_cmd(priv, MT7530_FDB_FLUSH, NULL);
if (ret < 0)
@@ -1329,6 +1376,216 @@ mt7530_setup(struct dsa_switch *ds)
return 0;
}
+static void mt7530_phylink_mac_config(struct dsa_switch *ds, int port,
+ unsigned int mode,
+ const struct phylink_link_state *state)
+{
+ struct mt7530_priv *priv = ds->priv;
+ u32 mcr_cur, mcr_new;
+
+ switch (port) {
+ case 0: /* Internal phy */
+ case 1:
+ case 2:
+ case 3:
+ case 4:
+ if (state->interface != PHY_INTERFACE_MODE_GMII)
+ return;
+ break;
+ case 5: /* 2nd cpu port with phy of port 0 or 4 / external phy */
+ if (priv->p5_interface == state->interface)
+ break;
+ if (!phy_interface_mode_is_rgmii(state->interface) &&
+ state->interface != PHY_INTERFACE_MODE_MII &&
+ state->interface != PHY_INTERFACE_MODE_GMII)
+ return;
+
+ mt7530_setup_port5(ds, state->interface);
+ break;
+ case 6: /* 1st cpu port */
+ if (priv->p6_interface == state->interface)
+ break;
+
+ if (state->interface != PHY_INTERFACE_MODE_RGMII &&
+ state->interface != PHY_INTERFACE_MODE_TRGMII)
+ return;
+
+ /* Setup TX circuit incluing relevant PAD and driving */
+ mt7530_pad_clk_setup(ds, state->interface);
+
+ if (priv->id == ID_MT7530) {
+ /* Setup RX circuit, relevant PAD and driving on the
+ * host which must be placed after the setup on the
+ * device side is all finished.
+ */
+ mt7623_pad_clk_setup(ds);
+ }
+
+ priv->p6_interface = state->interface;
+ break;
+ default:
+ dev_err(ds->dev, "%s: unsupported port: %i\n", __func__, port);
+ return;
+ }
+
+ if (phylink_autoneg_inband(mode)) {
+ dev_err(ds->dev, "%s: in-band negotiation unsupported\n",
+ __func__);
+ return;
+ }
+
+ mcr_cur = mt7530_read(priv, MT7530_PMCR_P(port));
+ mcr_new = mcr_cur;
+ mcr_new &= ~(PMCR_FORCE_SPEED_1000 | PMCR_FORCE_SPEED_100 |
+ PMCR_FORCE_FDX | PMCR_TX_FC_EN | PMCR_RX_FC_EN);
+ mcr_new |= PMCR_IFG_XMIT(1) | PMCR_MAC_MODE | PMCR_BACKOFF_EN |
+ PMCR_BACKPR_EN | PMCR_FORCE_MODE | PMCR_FORCE_LNK;
+
+ /* Are we connected to external phy */
+ if (port == 5 && dsa_is_user_port(ds, 5))
+ mcr_new |= PMCR_EXT_PHY;
+
+ switch (state->speed) {
+ case SPEED_1000:
+ mcr_new |= PMCR_FORCE_SPEED_1000;
+ break;
+ case SPEED_100:
+ mcr_new |= PMCR_FORCE_SPEED_100;
+ break;
+ }
+ if (state->duplex == DUPLEX_FULL) {
+ mcr_new |= PMCR_FORCE_FDX;
+ if (state->pause & MLO_PAUSE_TX)
+ mcr_new |= PMCR_TX_FC_EN;
+ if (state->pause & MLO_PAUSE_RX)
+ mcr_new |= PMCR_RX_FC_EN;
+ }
+
+ if (mcr_new != mcr_cur)
+ mt7530_write(priv, MT7530_PMCR_P(port), mcr_new);
+}
+
+static void mt7530_phylink_mac_link_down(struct dsa_switch *ds, int port,
+ unsigned int mode,
+ phy_interface_t interface)
+{
+ struct mt7530_priv *priv = ds->priv;
+
+ mt7530_port_set_status(priv, port, 0);
+}
+
+static void mt7530_phylink_mac_link_up(struct dsa_switch *ds, int port,
+ unsigned int mode,
+ phy_interface_t interface,
+ struct phy_device *phydev)
+{
+ struct mt7530_priv *priv = ds->priv;
+
+ mt7530_port_set_status(priv, port, 1);
+}
+
+static void mt7530_phylink_validate(struct dsa_switch *ds, int port,
+ unsigned long *supported,
+ struct phylink_link_state *state)
+{
+ __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
+
+ switch (port) {
+ case 0: /* Internal phy */
+ case 1:
+ case 2:
+ case 3:
+ case 4:
+ if (state->interface != PHY_INTERFACE_MODE_NA &&
+ state->interface != PHY_INTERFACE_MODE_GMII)
+ goto unsupported;
+ break;
+ case 5: /* 2nd cpu port with phy of port 0 or 4 / external phy */
+ if (state->interface != PHY_INTERFACE_MODE_NA &&
+ !phy_interface_mode_is_rgmii(state->interface) &&
+ state->interface != PHY_INTERFACE_MODE_MII &&
+ state->interface != PHY_INTERFACE_MODE_GMII)
+ goto unsupported;
+ break;
+ case 6: /* 1st cpu port */
+ if (state->interface != PHY_INTERFACE_MODE_NA &&
+ state->interface != PHY_INTERFACE_MODE_RGMII &&
+ state->interface != PHY_INTERFACE_MODE_TRGMII)
+ goto unsupported;
+ break;
+ default:
+ dev_err(ds->dev, "%s: unsupported port: %i\n", __func__, port);
+unsupported:
+ linkmode_zero(supported);
+ return;
+ }
+
+ phylink_set_port_modes(mask);
+ phylink_set(mask, Autoneg);
+
+ if (state->interface == PHY_INTERFACE_MODE_TRGMII) {
+ phylink_set(mask, 1000baseT_Full);
+ } else {
+ phylink_set(mask, 10baseT_Half);
+ phylink_set(mask, 10baseT_Full);
+ phylink_set(mask, 100baseT_Half);
+ phylink_set(mask, 100baseT_Full);
+
+ if (state->interface != PHY_INTERFACE_MODE_MII) {
+ phylink_set(mask, 1000baseT_Half);
+ phylink_set(mask, 1000baseT_Full);
+ if (port == 5)
+ phylink_set(mask, 1000baseX_Full);
+ }
+ }
+
+ phylink_set(mask, Pause);
+ phylink_set(mask, Asym_Pause);
+
+ linkmode_and(supported, supported, mask);
+ linkmode_and(state->advertising, state->advertising, mask);
+}
+
+static int
+mt7530_phylink_mac_link_state(struct dsa_switch *ds, int port,
+ struct phylink_link_state *state)
+{
+ struct mt7530_priv *priv = ds->priv;
+ u32 pmsr;
+
+ if (port < 0 || port >= MT7530_NUM_PORTS)
+ return -EINVAL;
+
+ pmsr = mt7530_read(priv, MT7530_PMSR_P(port));
+
+ state->link = (pmsr & PMSR_LINK);
+ state->an_complete = state->link;
+ state->duplex = !!(pmsr & PMSR_DPX);
+
+ switch (pmsr & PMSR_SPEED_MASK) {
+ case PMSR_SPEED_10:
+ state->speed = SPEED_10;
+ break;
+ case PMSR_SPEED_100:
+ state->speed = SPEED_100;
+ break;
+ case PMSR_SPEED_1000:
+ state->speed = SPEED_1000;
+ break;
+ default:
+ state->speed = SPEED_UNKNOWN;
+ break;
+ }
+
+ state->pause &= ~(MLO_PAUSE_RX | MLO_PAUSE_TX);
+ if (pmsr & PMSR_RX_FC)
+ state->pause |= MLO_PAUSE_RX;
+ if (pmsr & PMSR_TX_FC)
+ state->pause |= MLO_PAUSE_TX;
+
+ return 1;
+}
+
static const struct dsa_switch_ops mt7530_switch_ops = {
.get_tag_protocol = mtk_get_tag_protocol,
.setup = mt7530_setup,
@@ -1337,7 +1594,6 @@ static const struct dsa_switch_ops mt7530_switch_ops = {
.phy_write = mt7530_phy_write,
.get_ethtool_stats = mt7530_get_ethtool_stats,
.get_sset_count = mt7530_get_sset_count,
- .adjust_link = mt7530_adjust_link,
.port_enable = mt7530_port_enable,
.port_disable = mt7530_port_disable,
.port_stp_state_set = mt7530_stp_state_set,
@@ -1350,6 +1606,11 @@ static const struct dsa_switch_ops mt7530_switch_ops = {
.port_vlan_prepare = mt7530_port_vlan_prepare,
.port_vlan_add = mt7530_port_vlan_add,
.port_vlan_del = mt7530_port_vlan_del,
+ .phylink_validate = mt7530_phylink_validate,
+ .phylink_mac_link_state = mt7530_phylink_mac_link_state,
+ .phylink_mac_config = mt7530_phylink_mac_config,
+ .phylink_mac_link_down = mt7530_phylink_mac_link_down,
+ .phylink_mac_link_up = mt7530_phylink_mac_link_up,
};
static const struct of_device_id mt7530_of_match[] = {
diff --git a/drivers/net/dsa/mt7530.h b/drivers/net/dsa/mt7530.h
index bfac90f..ccb9da8 100644
--- a/drivers/net/dsa/mt7530.h
+++ b/drivers/net/dsa/mt7530.h
@@ -186,6 +186,7 @@ enum mt7530_vlan_port_attr {
/* Register for port MAC control register */
#define MT7530_PMCR_P(x) (0x3000 + ((x) * 0x100))
#define PMCR_IFG_XMIT(x) (((x) & 0x3) << 18)
+#define PMCR_EXT_PHY BIT(17)
#define PMCR_MAC_MODE BIT(16)
#define PMCR_FORCE_MODE BIT(15)
#define PMCR_TX_EN BIT(14)
@@ -198,26 +199,20 @@ enum mt7530_vlan_port_attr {
#define PMCR_FORCE_SPEED_100 BIT(2)
#define PMCR_FORCE_FDX BIT(1)
#define PMCR_FORCE_LNK BIT(0)
-#define PMCR_COMMON_LINK (PMCR_IFG_XMIT(1) | PMCR_MAC_MODE | \
- PMCR_BACKOFF_EN | PMCR_BACKPR_EN | \
- PMCR_TX_EN | PMCR_RX_EN | \
- PMCR_TX_FC_EN | PMCR_RX_FC_EN)
-#define PMCR_CPUP_LINK (PMCR_COMMON_LINK | PMCR_FORCE_MODE | \
- PMCR_FORCE_SPEED_1000 | \
- PMCR_FORCE_FDX | \
- PMCR_FORCE_LNK)
-#define PMCR_USERP_LINK PMCR_COMMON_LINK
-#define PMCR_FIXED_LINK (PMCR_IFG_XMIT(1) | PMCR_MAC_MODE | \
- PMCR_FORCE_MODE | PMCR_TX_EN | \
- PMCR_RX_EN | PMCR_BACKPR_EN | \
- PMCR_BACKOFF_EN | \
- PMCR_FORCE_SPEED_1000 | \
- PMCR_FORCE_FDX | \
- PMCR_FORCE_LNK)
-#define PMCR_FIXED_LINK_FC (PMCR_FIXED_LINK | \
- PMCR_TX_FC_EN | PMCR_RX_FC_EN)
+#define PMCR_SPEED_MASK (PMCR_FORCE_SPEED_100 | \
+ PMCR_FORCE_SPEED_1000)
#define MT7530_PMSR_P(x) (0x3008 + (x) * 0x100)
+#define PMSR_EEE1G BIT(7)
+#define PMSR_EEE100M BIT(6)
+#define PMSR_RX_FC BIT(5)
+#define PMSR_TX_FC BIT(4)
+#define PMSR_SPEED_1000 BIT(3)
+#define PMSR_SPEED_100 BIT(2)
+#define PMSR_SPEED_10 0x00
+#define PMSR_SPEED_MASK (PMSR_SPEED_100 | PMSR_SPEED_1000)
+#define PMSR_DPX BIT(1)
+#define PMSR_LINK BIT(0)
/* Register for MIB */
#define MT7530_PORT_MIB_COUNTER(x) (0x4000 + (x) * 0x100)
@@ -251,6 +246,7 @@ enum mt7530_vlan_port_attr {
/* Register for hw trap modification */
#define MT7530_MHWTRAP 0x7804
+#define MHWTRAP_PHY0_SEL BIT(20)
#define MHWTRAP_MANUAL BIT(16)
#define MHWTRAP_P5_MAC_SEL BIT(13)
#define MHWTRAP_P6_DIS BIT(8)
@@ -408,6 +404,30 @@ struct mt7530_port {
u16 pvid;
};
+/* Port 5 interface select definitions */
+enum p5_interface_select {
+ P5_DISABLED = 0,
+ P5_INTF_SEL_PHY_P0,
+ P5_INTF_SEL_PHY_P4,
+ P5_INTF_SEL_GMAC5,
+};
+
+static const char *p5_intf_modes(unsigned int p5_interface)
+{
+ switch (p5_interface) {
+ case P5_DISABLED:
+ return "DISABLED";
+ case P5_INTF_SEL_PHY_P0:
+ return "PHY P0";
+ case P5_INTF_SEL_PHY_P4:
+ return "PHY P4";
+ case P5_INTF_SEL_GMAC5:
+ return "GMAC5";
+ default:
+ return "unknown";
+ }
+}
+
/* struct mt7530_priv - This is the main data structure for holding the state
* of the driver
* @dev: The device pointer
@@ -423,6 +443,8 @@ struct mt7530_port {
* @ports: Holding the state among ports
* @reg_mutex: The lock for protecting among process accessing
* registers
+ * @p6_interface Holding the current port 6 interface
+ * @p5_intf_sel: Holding the current port 5 interface select
*/
struct mt7530_priv {
struct device *dev;
@@ -435,6 +457,9 @@ struct mt7530_priv {
struct gpio_desc *reset;
unsigned int id;
bool mcm;
+ phy_interface_t p6_interface;
+ phy_interface_t p5_interface;
+ unsigned int p5_intf_sel;
struct mt7530_port ports[MT7530_NUM_PORTS];
/* protect among processes for registers access*/
diff --git a/drivers/net/dsa/mv88e6xxx/Makefile b/drivers/net/dsa/mv88e6xxx/Makefile
index e85755d..aa645ff 100644
--- a/drivers/net/dsa/mv88e6xxx/Makefile
+++ b/drivers/net/dsa/mv88e6xxx/Makefile
@@ -10,6 +10,7 @@
mv88e6xxx-$(CONFIG_NET_DSA_MV88E6XXX_PTP) += hwtstamp.o
mv88e6xxx-objs += phy.o
mv88e6xxx-objs += port.o
+mv88e6xxx-objs += port_hidden.o
mv88e6xxx-$(CONFIG_NET_DSA_MV88E6XXX_PTP) += ptp.o
mv88e6xxx-objs += serdes.o
mv88e6xxx-objs += smi.o
diff --git a/drivers/net/dsa/mv88e6xxx/chip.c b/drivers/net/dsa/mv88e6xxx/chip.c
index d0bf98c..6787d56 100644
--- a/drivers/net/dsa/mv88e6xxx/chip.c
+++ b/drivers/net/dsa/mv88e6xxx/chip.c
@@ -1497,7 +1497,7 @@ static int mv88e6xxx_port_db_load_purge(struct mv88e6xxx_chip *chip, int port,
fid = vlan.fid;
}
- entry.state = MV88E6XXX_G1_ATU_DATA_STATE_UNUSED;
+ entry.state = 0;
ether_addr_copy(entry.mac, addr);
eth_addr_dec(entry.mac);
@@ -1506,17 +1506,16 @@ static int mv88e6xxx_port_db_load_purge(struct mv88e6xxx_chip *chip, int port,
return err;
/* Initialize a fresh ATU entry if it isn't found */
- if (entry.state == MV88E6XXX_G1_ATU_DATA_STATE_UNUSED ||
- !ether_addr_equal(entry.mac, addr)) {
+ if (!entry.state || !ether_addr_equal(entry.mac, addr)) {
memset(&entry, 0, sizeof(entry));
ether_addr_copy(entry.mac, addr);
}
/* Purge the ATU entry only if no port is using it anymore */
- if (state == MV88E6XXX_G1_ATU_DATA_STATE_UNUSED) {
+ if (!state) {
entry.portvec &= ~BIT(port);
if (!entry.portvec)
- entry.state = MV88E6XXX_G1_ATU_DATA_STATE_UNUSED;
+ entry.state = 0;
} else {
entry.portvec |= BIT(port);
entry.state = state;
@@ -1525,6 +1524,216 @@ static int mv88e6xxx_port_db_load_purge(struct mv88e6xxx_chip *chip, int port,
return mv88e6xxx_g1_atu_loadpurge(chip, fid, &entry);
}
+static int mv88e6xxx_policy_apply(struct mv88e6xxx_chip *chip, int port,
+ const struct mv88e6xxx_policy *policy)
+{
+ enum mv88e6xxx_policy_mapping mapping = policy->mapping;
+ enum mv88e6xxx_policy_action action = policy->action;
+ const u8 *addr = policy->addr;
+ u16 vid = policy->vid;
+ u8 state;
+ int err;
+ int id;
+
+ if (!chip->info->ops->port_set_policy)
+ return -EOPNOTSUPP;
+
+ switch (mapping) {
+ case MV88E6XXX_POLICY_MAPPING_DA:
+ case MV88E6XXX_POLICY_MAPPING_SA:
+ if (action == MV88E6XXX_POLICY_ACTION_NORMAL)
+ state = 0; /* Dissociate the port and address */
+ else if (action == MV88E6XXX_POLICY_ACTION_DISCARD &&
+ is_multicast_ether_addr(addr))
+ state = MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_POLICY;
+ else if (action == MV88E6XXX_POLICY_ACTION_DISCARD &&
+ is_unicast_ether_addr(addr))
+ state = MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_POLICY;
+ else
+ return -EOPNOTSUPP;
+
+ err = mv88e6xxx_port_db_load_purge(chip, port, addr, vid,
+ state);
+ if (err)
+ return err;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ /* Skip the port's policy clearing if the mapping is still in use */
+ if (action == MV88E6XXX_POLICY_ACTION_NORMAL)
+ idr_for_each_entry(&chip->policies, policy, id)
+ if (policy->port == port &&
+ policy->mapping == mapping &&
+ policy->action != action)
+ return 0;
+
+ return chip->info->ops->port_set_policy(chip, port, mapping, action);
+}
+
+static int mv88e6xxx_policy_insert(struct mv88e6xxx_chip *chip, int port,
+ struct ethtool_rx_flow_spec *fs)
+{
+ struct ethhdr *mac_entry = &fs->h_u.ether_spec;
+ struct ethhdr *mac_mask = &fs->m_u.ether_spec;
+ enum mv88e6xxx_policy_mapping mapping;
+ enum mv88e6xxx_policy_action action;
+ struct mv88e6xxx_policy *policy;
+ u16 vid = 0;
+ u8 *addr;
+ int err;
+ int id;
+
+ if (fs->location != RX_CLS_LOC_ANY)
+ return -EINVAL;
+
+ if (fs->ring_cookie == RX_CLS_FLOW_DISC)
+ action = MV88E6XXX_POLICY_ACTION_DISCARD;
+ else
+ return -EOPNOTSUPP;
+
+ switch (fs->flow_type & ~FLOW_EXT) {
+ case ETHER_FLOW:
+ if (!is_zero_ether_addr(mac_mask->h_dest) &&
+ is_zero_ether_addr(mac_mask->h_source)) {
+ mapping = MV88E6XXX_POLICY_MAPPING_DA;
+ addr = mac_entry->h_dest;
+ } else if (is_zero_ether_addr(mac_mask->h_dest) &&
+ !is_zero_ether_addr(mac_mask->h_source)) {
+ mapping = MV88E6XXX_POLICY_MAPPING_SA;
+ addr = mac_entry->h_source;
+ } else {
+ /* Cannot support DA and SA mapping in the same rule */
+ return -EOPNOTSUPP;
+ }
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ if ((fs->flow_type & FLOW_EXT) && fs->m_ext.vlan_tci) {
+ if (fs->m_ext.vlan_tci != 0xffff)
+ return -EOPNOTSUPP;
+ vid = be16_to_cpu(fs->h_ext.vlan_tci) & VLAN_VID_MASK;
+ }
+
+ idr_for_each_entry(&chip->policies, policy, id) {
+ if (policy->port == port && policy->mapping == mapping &&
+ policy->action == action && policy->vid == vid &&
+ ether_addr_equal(policy->addr, addr))
+ return -EEXIST;
+ }
+
+ policy = devm_kzalloc(chip->dev, sizeof(*policy), GFP_KERNEL);
+ if (!policy)
+ return -ENOMEM;
+
+ fs->location = 0;
+ err = idr_alloc_u32(&chip->policies, policy, &fs->location, 0xffffffff,
+ GFP_KERNEL);
+ if (err) {
+ devm_kfree(chip->dev, policy);
+ return err;
+ }
+
+ memcpy(&policy->fs, fs, sizeof(*fs));
+ ether_addr_copy(policy->addr, addr);
+ policy->mapping = mapping;
+ policy->action = action;
+ policy->port = port;
+ policy->vid = vid;
+
+ err = mv88e6xxx_policy_apply(chip, port, policy);
+ if (err) {
+ idr_remove(&chip->policies, fs->location);
+ devm_kfree(chip->dev, policy);
+ return err;
+ }
+
+ return 0;
+}
+
+static int mv88e6xxx_get_rxnfc(struct dsa_switch *ds, int port,
+ struct ethtool_rxnfc *rxnfc, u32 *rule_locs)
+{
+ struct ethtool_rx_flow_spec *fs = &rxnfc->fs;
+ struct mv88e6xxx_chip *chip = ds->priv;
+ struct mv88e6xxx_policy *policy;
+ int err;
+ int id;
+
+ mv88e6xxx_reg_lock(chip);
+
+ switch (rxnfc->cmd) {
+ case ETHTOOL_GRXCLSRLCNT:
+ rxnfc->data = 0;
+ rxnfc->data |= RX_CLS_LOC_SPECIAL;
+ rxnfc->rule_cnt = 0;
+ idr_for_each_entry(&chip->policies, policy, id)
+ if (policy->port == port)
+ rxnfc->rule_cnt++;
+ err = 0;
+ break;
+ case ETHTOOL_GRXCLSRULE:
+ err = -ENOENT;
+ policy = idr_find(&chip->policies, fs->location);
+ if (policy) {
+ memcpy(fs, &policy->fs, sizeof(*fs));
+ err = 0;
+ }
+ break;
+ case ETHTOOL_GRXCLSRLALL:
+ rxnfc->data = 0;
+ rxnfc->rule_cnt = 0;
+ idr_for_each_entry(&chip->policies, policy, id)
+ if (policy->port == port)
+ rule_locs[rxnfc->rule_cnt++] = id;
+ err = 0;
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ break;
+ }
+
+ mv88e6xxx_reg_unlock(chip);
+
+ return err;
+}
+
+static int mv88e6xxx_set_rxnfc(struct dsa_switch *ds, int port,
+ struct ethtool_rxnfc *rxnfc)
+{
+ struct ethtool_rx_flow_spec *fs = &rxnfc->fs;
+ struct mv88e6xxx_chip *chip = ds->priv;
+ struct mv88e6xxx_policy *policy;
+ int err;
+
+ mv88e6xxx_reg_lock(chip);
+
+ switch (rxnfc->cmd) {
+ case ETHTOOL_SRXCLSRLINS:
+ err = mv88e6xxx_policy_insert(chip, port, fs);
+ break;
+ case ETHTOOL_SRXCLSRLDEL:
+ err = -ENOENT;
+ policy = idr_remove(&chip->policies, fs->location);
+ if (policy) {
+ policy->action = MV88E6XXX_POLICY_ACTION_NORMAL;
+ err = mv88e6xxx_policy_apply(chip, port, policy);
+ devm_kfree(chip->dev, policy);
+ }
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ break;
+ }
+
+ mv88e6xxx_reg_unlock(chip);
+
+ return err;
+}
+
static int mv88e6xxx_port_add_broadcast(struct mv88e6xxx_chip *chip, int port,
u16 vid)
{
@@ -1732,8 +1941,7 @@ static int mv88e6xxx_port_fdb_del(struct dsa_switch *ds, int port,
int err;
mv88e6xxx_reg_lock(chip);
- err = mv88e6xxx_port_db_load_purge(chip, port, addr, vid,
- MV88E6XXX_G1_ATU_DATA_STATE_UNUSED);
+ err = mv88e6xxx_port_db_load_purge(chip, port, addr, vid, 0);
mv88e6xxx_reg_unlock(chip);
return err;
@@ -1747,7 +1955,7 @@ static int mv88e6xxx_port_db_dump_fid(struct mv88e6xxx_chip *chip,
bool is_static;
int err;
- addr.state = MV88E6XXX_G1_ATU_DATA_STATE_UNUSED;
+ addr.state = 0;
eth_broadcast_addr(addr.mac);
do {
@@ -1755,7 +1963,7 @@ static int mv88e6xxx_port_db_dump_fid(struct mv88e6xxx_chip *chip,
if (err)
return err;
- if (addr.state == MV88E6XXX_G1_ATU_DATA_STATE_UNUSED)
+ if (!addr.state)
break;
if (addr.trunk || (addr.portvec & BIT(port)) == 0)
@@ -2054,26 +2262,93 @@ static int mv88e6xxx_setup_egress_floods(struct mv88e6xxx_chip *chip, int port)
return 0;
}
+static irqreturn_t mv88e6xxx_serdes_irq_thread_fn(int irq, void *dev_id)
+{
+ struct mv88e6xxx_port *mvp = dev_id;
+ struct mv88e6xxx_chip *chip = mvp->chip;
+ irqreturn_t ret = IRQ_NONE;
+ int port = mvp->port;
+ u8 lane;
+
+ mv88e6xxx_reg_lock(chip);
+ lane = mv88e6xxx_serdes_get_lane(chip, port);
+ if (lane)
+ ret = mv88e6xxx_serdes_irq_status(chip, port, lane);
+ mv88e6xxx_reg_unlock(chip);
+
+ return ret;
+}
+
+static int mv88e6xxx_serdes_irq_request(struct mv88e6xxx_chip *chip, int port,
+ u8 lane)
+{
+ struct mv88e6xxx_port *dev_id = &chip->ports[port];
+ unsigned int irq;
+ int err;
+
+ /* Nothing to request if this SERDES port has no IRQ */
+ irq = mv88e6xxx_serdes_irq_mapping(chip, port);
+ if (!irq)
+ return 0;
+
+ /* Requesting the IRQ will trigger IRQ callbacks, so release the lock */
+ mv88e6xxx_reg_unlock(chip);
+ err = request_threaded_irq(irq, NULL, mv88e6xxx_serdes_irq_thread_fn,
+ IRQF_ONESHOT, "mv88e6xxx-serdes", dev_id);
+ mv88e6xxx_reg_lock(chip);
+ if (err)
+ return err;
+
+ dev_id->serdes_irq = irq;
+
+ return mv88e6xxx_serdes_irq_enable(chip, port, lane);
+}
+
+static int mv88e6xxx_serdes_irq_free(struct mv88e6xxx_chip *chip, int port,
+ u8 lane)
+{
+ struct mv88e6xxx_port *dev_id = &chip->ports[port];
+ unsigned int irq = dev_id->serdes_irq;
+ int err;
+
+ /* Nothing to free if no IRQ has been requested */
+ if (!irq)
+ return 0;
+
+ err = mv88e6xxx_serdes_irq_disable(chip, port, lane);
+
+ /* Freeing the IRQ will trigger IRQ callbacks, so release the lock */
+ mv88e6xxx_reg_unlock(chip);
+ free_irq(irq, dev_id);
+ mv88e6xxx_reg_lock(chip);
+
+ dev_id->serdes_irq = 0;
+
+ return err;
+}
+
static int mv88e6xxx_serdes_power(struct mv88e6xxx_chip *chip, int port,
bool on)
{
+ u8 lane;
int err;
- if (!chip->info->ops->serdes_power)
+ lane = mv88e6xxx_serdes_get_lane(chip, port);
+ if (!lane)
return 0;
if (on) {
- err = chip->info->ops->serdes_power(chip, port, true);
+ err = mv88e6xxx_serdes_power_up(chip, port, lane);
if (err)
return err;
- if (chip->info->ops->serdes_irq_setup)
- err = chip->info->ops->serdes_irq_setup(chip, port);
+ err = mv88e6xxx_serdes_irq_request(chip, port, lane);
} else {
- if (chip->info->ops->serdes_irq_free)
- chip->info->ops->serdes_irq_free(chip, port);
+ err = mv88e6xxx_serdes_irq_free(chip, port, lane);
+ if (err)
+ return err;
- err = chip->info->ops->serdes_power(chip, port, false);
+ err = mv88e6xxx_serdes_power_down(chip, port, lane);
}
return err;
@@ -2317,60 +2592,6 @@ static int mv88e6xxx_stats_setup(struct mv88e6xxx_chip *chip)
return mv88e6xxx_g1_stats_clear(chip);
}
-/* The mv88e6390 has some hidden registers used for debug and
- * development. The errata also makes use of them.
- */
-static int mv88e6390_hidden_write(struct mv88e6xxx_chip *chip, int port,
- int reg, u16 val)
-{
- u16 ctrl;
- int err;
-
- err = mv88e6xxx_port_write(chip, PORT_RESERVED_1A_DATA_PORT,
- PORT_RESERVED_1A, val);
- if (err)
- return err;
-
- ctrl = PORT_RESERVED_1A_BUSY | PORT_RESERVED_1A_WRITE |
- PORT_RESERVED_1A_BLOCK | port << PORT_RESERVED_1A_PORT_SHIFT |
- reg;
-
- return mv88e6xxx_port_write(chip, PORT_RESERVED_1A_CTRL_PORT,
- PORT_RESERVED_1A, ctrl);
-}
-
-static int mv88e6390_hidden_wait(struct mv88e6xxx_chip *chip)
-{
- int bit = __bf_shf(PORT_RESERVED_1A_BUSY);
-
- return mv88e6xxx_wait_bit(chip, PORT_RESERVED_1A_CTRL_PORT,
- PORT_RESERVED_1A, bit, 0);
-}
-
-
-static int mv88e6390_hidden_read(struct mv88e6xxx_chip *chip, int port,
- int reg, u16 *val)
-{
- u16 ctrl;
- int err;
-
- ctrl = PORT_RESERVED_1A_BUSY | PORT_RESERVED_1A_READ |
- PORT_RESERVED_1A_BLOCK | port << PORT_RESERVED_1A_PORT_SHIFT |
- reg;
-
- err = mv88e6xxx_port_write(chip, PORT_RESERVED_1A_CTRL_PORT,
- PORT_RESERVED_1A, ctrl);
- if (err)
- return err;
-
- err = mv88e6390_hidden_wait(chip);
- if (err)
- return err;
-
- return mv88e6xxx_port_read(chip, PORT_RESERVED_1A_DATA_PORT,
- PORT_RESERVED_1A, val);
-}
-
/* Check if the errata has already been applied. */
static bool mv88e6390_setup_errata_applied(struct mv88e6xxx_chip *chip)
{
@@ -2379,7 +2600,7 @@ static bool mv88e6390_setup_errata_applied(struct mv88e6xxx_chip *chip)
u16 val;
for (port = 0; port < mv88e6xxx_num_ports(chip); port++) {
- err = mv88e6390_hidden_read(chip, port, 0, &val);
+ err = mv88e6xxx_port_hidden_read(chip, 0xf, port, 0, &val);
if (err) {
dev_err(chip->dev,
"Error reading hidden register: %d\n", err);
@@ -2412,7 +2633,7 @@ static int mv88e6390_setup_errata(struct mv88e6xxx_chip *chip)
}
for (port = 0; port < mv88e6xxx_num_ports(chip); port++) {
- err = mv88e6390_hidden_write(chip, port, 0, 0x01c0);
+ err = mv88e6xxx_port_hidden_write(chip, 0xf, port, 0, 0x01c0);
if (err)
return err;
}
@@ -2967,6 +3188,7 @@ static const struct mv88e6xxx_ops mv88e6141_ops = {
.port_disable_pri_override = mv88e6xxx_port_disable_pri_override,
.port_link_state = mv88e6352_port_link_state,
.port_get_cmode = mv88e6352_port_get_cmode,
+ .port_set_cmode = mv88e6341_port_set_cmode,
.port_setup_message_port = mv88e6xxx_setup_message_port,
.stats_snapshot = mv88e6390_g1_stats_snapshot,
.stats_set_histogram = mv88e6095_g1_stats_set_histogram,
@@ -2981,7 +3203,11 @@ static const struct mv88e6xxx_ops mv88e6141_ops = {
.reset = mv88e6352_g1_reset,
.vtu_getnext = mv88e6352_g1_vtu_getnext,
.vtu_loadpurge = mv88e6352_g1_vtu_loadpurge,
- .serdes_power = mv88e6341_serdes_power,
+ .serdes_power = mv88e6390_serdes_power,
+ .serdes_get_lane = mv88e6341_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.phylink_validate = mv88e6341_phylink_validate,
};
@@ -3116,6 +3342,7 @@ static const struct mv88e6xxx_ops mv88e6172_ops = {
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed = mv88e6352_port_set_speed,
.port_tag_remap = mv88e6095_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3141,6 +3368,7 @@ static const struct mv88e6xxx_ops mv88e6172_ops = {
.rmu_disable = mv88e6352_g1_rmu_disable,
.vtu_getnext = mv88e6352_g1_vtu_getnext,
.vtu_loadpurge = mv88e6352_g1_vtu_loadpurge,
+ .serdes_get_lane = mv88e6352_serdes_get_lane,
.serdes_power = mv88e6352_serdes_power,
.gpio_ops = &mv88e6352_gpio_ops,
.phylink_validate = mv88e6352_phylink_validate,
@@ -3201,6 +3429,7 @@ static const struct mv88e6xxx_ops mv88e6176_ops = {
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed = mv88e6352_port_set_speed,
.port_tag_remap = mv88e6095_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3226,9 +3455,11 @@ static const struct mv88e6xxx_ops mv88e6176_ops = {
.rmu_disable = mv88e6352_g1_rmu_disable,
.vtu_getnext = mv88e6352_g1_vtu_getnext,
.vtu_loadpurge = mv88e6352_g1_vtu_loadpurge,
+ .serdes_get_lane = mv88e6352_serdes_get_lane,
.serdes_power = mv88e6352_serdes_power,
- .serdes_irq_setup = mv88e6352_serdes_irq_setup,
- .serdes_irq_free = mv88e6352_serdes_irq_free,
+ .serdes_irq_mapping = mv88e6352_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6352_serdes_irq_enable,
+ .serdes_irq_status = mv88e6352_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.phylink_validate = mv88e6352_phylink_validate,
};
@@ -3284,6 +3515,7 @@ static const struct mv88e6xxx_ops mv88e6190_ops = {
.port_set_speed = mv88e6390_port_set_speed,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.port_tag_remap = mv88e6390_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3309,8 +3541,10 @@ static const struct mv88e6xxx_ops mv88e6190_ops = {
.vtu_getnext = mv88e6390_g1_vtu_getnext,
.vtu_loadpurge = mv88e6390_g1_vtu_loadpurge,
.serdes_power = mv88e6390_serdes_power,
- .serdes_irq_setup = mv88e6390_serdes_irq_setup,
- .serdes_irq_free = mv88e6390_serdes_irq_free,
+ .serdes_get_lane = mv88e6390_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.phylink_validate = mv88e6390_phylink_validate,
};
@@ -3330,6 +3564,7 @@ static const struct mv88e6xxx_ops mv88e6190x_ops = {
.port_set_speed = mv88e6390x_port_set_speed,
.port_max_speed_mode = mv88e6390x_port_max_speed_mode,
.port_tag_remap = mv88e6390_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3354,9 +3589,11 @@ static const struct mv88e6xxx_ops mv88e6190x_ops = {
.rmu_disable = mv88e6390_g1_rmu_disable,
.vtu_getnext = mv88e6390_g1_vtu_getnext,
.vtu_loadpurge = mv88e6390_g1_vtu_loadpurge,
- .serdes_power = mv88e6390x_serdes_power,
- .serdes_irq_setup = mv88e6390x_serdes_irq_setup,
- .serdes_irq_free = mv88e6390x_serdes_irq_free,
+ .serdes_power = mv88e6390_serdes_power,
+ .serdes_get_lane = mv88e6390x_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.phylink_validate = mv88e6390x_phylink_validate,
};
@@ -3401,8 +3638,10 @@ static const struct mv88e6xxx_ops mv88e6191_ops = {
.vtu_getnext = mv88e6390_g1_vtu_getnext,
.vtu_loadpurge = mv88e6390_g1_vtu_loadpurge,
.serdes_power = mv88e6390_serdes_power,
- .serdes_irq_setup = mv88e6390_serdes_irq_setup,
- .serdes_irq_free = mv88e6390_serdes_irq_free,
+ .serdes_get_lane = mv88e6390_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.avb_ops = &mv88e6390_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
.phylink_validate = mv88e6390_phylink_validate,
@@ -3423,6 +3662,7 @@ static const struct mv88e6xxx_ops mv88e6240_ops = {
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed = mv88e6352_port_set_speed,
.port_tag_remap = mv88e6095_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3448,9 +3688,11 @@ static const struct mv88e6xxx_ops mv88e6240_ops = {
.rmu_disable = mv88e6352_g1_rmu_disable,
.vtu_getnext = mv88e6352_g1_vtu_getnext,
.vtu_loadpurge = mv88e6352_g1_vtu_loadpurge,
+ .serdes_get_lane = mv88e6352_serdes_get_lane,
.serdes_power = mv88e6352_serdes_power,
- .serdes_irq_setup = mv88e6352_serdes_irq_setup,
- .serdes_irq_free = mv88e6352_serdes_irq_free,
+ .serdes_irq_mapping = mv88e6352_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6352_serdes_irq_enable,
+ .serdes_irq_status = mv88e6352_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.avb_ops = &mv88e6352_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
@@ -3512,6 +3754,7 @@ static const struct mv88e6xxx_ops mv88e6290_ops = {
.port_set_speed = mv88e6390_port_set_speed,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.port_tag_remap = mv88e6390_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3537,8 +3780,10 @@ static const struct mv88e6xxx_ops mv88e6290_ops = {
.vtu_getnext = mv88e6390_g1_vtu_getnext,
.vtu_loadpurge = mv88e6390_g1_vtu_loadpurge,
.serdes_power = mv88e6390_serdes_power,
- .serdes_irq_setup = mv88e6390_serdes_irq_setup,
- .serdes_irq_free = mv88e6390_serdes_irq_free,
+ .serdes_get_lane = mv88e6390_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.avb_ops = &mv88e6390_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
@@ -3657,6 +3902,7 @@ static const struct mv88e6xxx_ops mv88e6341_ops = {
.port_disable_pri_override = mv88e6xxx_port_disable_pri_override,
.port_link_state = mv88e6352_port_link_state,
.port_get_cmode = mv88e6352_port_get_cmode,
+ .port_set_cmode = mv88e6341_port_set_cmode,
.port_setup_message_port = mv88e6xxx_setup_message_port,
.stats_snapshot = mv88e6390_g1_stats_snapshot,
.stats_set_histogram = mv88e6095_g1_stats_set_histogram,
@@ -3671,7 +3917,11 @@ static const struct mv88e6xxx_ops mv88e6341_ops = {
.reset = mv88e6352_g1_reset,
.vtu_getnext = mv88e6352_g1_vtu_getnext,
.vtu_loadpurge = mv88e6352_g1_vtu_loadpurge,
- .serdes_power = mv88e6341_serdes_power,
+ .serdes_power = mv88e6390_serdes_power,
+ .serdes_get_lane = mv88e6341_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.avb_ops = &mv88e6390_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
@@ -3775,6 +4025,7 @@ static const struct mv88e6xxx_ops mv88e6352_ops = {
.port_set_rgmii_delay = mv88e6352_port_set_rgmii_delay,
.port_set_speed = mv88e6352_port_set_speed,
.port_tag_remap = mv88e6095_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3800,9 +4051,11 @@ static const struct mv88e6xxx_ops mv88e6352_ops = {
.rmu_disable = mv88e6352_g1_rmu_disable,
.vtu_getnext = mv88e6352_g1_vtu_getnext,
.vtu_loadpurge = mv88e6352_g1_vtu_loadpurge,
+ .serdes_get_lane = mv88e6352_serdes_get_lane,
.serdes_power = mv88e6352_serdes_power,
- .serdes_irq_setup = mv88e6352_serdes_irq_setup,
- .serdes_irq_free = mv88e6352_serdes_irq_free,
+ .serdes_irq_mapping = mv88e6352_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6352_serdes_irq_enable,
+ .serdes_irq_status = mv88e6352_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.avb_ops = &mv88e6352_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
@@ -3827,6 +4080,7 @@ static const struct mv88e6xxx_ops mv88e6390_ops = {
.port_set_speed = mv88e6390_port_set_speed,
.port_max_speed_mode = mv88e6390_port_max_speed_mode,
.port_tag_remap = mv88e6390_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3854,8 +4108,10 @@ static const struct mv88e6xxx_ops mv88e6390_ops = {
.vtu_getnext = mv88e6390_g1_vtu_getnext,
.vtu_loadpurge = mv88e6390_g1_vtu_loadpurge,
.serdes_power = mv88e6390_serdes_power,
- .serdes_irq_setup = mv88e6390_serdes_irq_setup,
- .serdes_irq_free = mv88e6390_serdes_irq_free,
+ .serdes_get_lane = mv88e6390_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.avb_ops = &mv88e6390_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
@@ -3877,6 +4133,7 @@ static const struct mv88e6xxx_ops mv88e6390x_ops = {
.port_set_speed = mv88e6390x_port_set_speed,
.port_max_speed_mode = mv88e6390x_port_max_speed_mode,
.port_tag_remap = mv88e6390_port_tag_remap,
+ .port_set_policy = mv88e6352_port_set_policy,
.port_set_frame_mode = mv88e6351_port_set_frame_mode,
.port_set_egress_floods = mv88e6352_port_set_egress_floods,
.port_set_ether_type = mv88e6351_port_set_ether_type,
@@ -3903,9 +4160,11 @@ static const struct mv88e6xxx_ops mv88e6390x_ops = {
.rmu_disable = mv88e6390_g1_rmu_disable,
.vtu_getnext = mv88e6390_g1_vtu_getnext,
.vtu_loadpurge = mv88e6390_g1_vtu_loadpurge,
- .serdes_power = mv88e6390x_serdes_power,
- .serdes_irq_setup = mv88e6390x_serdes_irq_setup,
- .serdes_irq_free = mv88e6390x_serdes_irq_free,
+ .serdes_power = mv88e6390_serdes_power,
+ .serdes_get_lane = mv88e6390x_serdes_get_lane,
+ .serdes_irq_mapping = mv88e6390_serdes_irq_mapping,
+ .serdes_irq_enable = mv88e6390_serdes_irq_enable,
+ .serdes_irq_status = mv88e6390_serdes_irq_status,
.gpio_ops = &mv88e6352_gpio_ops,
.avb_ops = &mv88e6390_avb_ops,
.ptp_ops = &mv88e6352_ptp_ops,
@@ -4606,6 +4865,7 @@ static struct mv88e6xxx_chip *mv88e6xxx_alloc_chip(struct device *dev)
mutex_init(&chip->reg_lock);
INIT_LIST_HEAD(&chip->mdios);
+ idr_init(&chip->policies);
return chip;
}
@@ -4648,8 +4908,7 @@ static int mv88e6xxx_port_mdb_del(struct dsa_switch *ds, int port,
int err;
mv88e6xxx_reg_lock(chip);
- err = mv88e6xxx_port_db_load_purge(chip, port, mdb->addr, mdb->vid,
- MV88E6XXX_G1_ATU_DATA_STATE_UNUSED);
+ err = mv88e6xxx_port_db_load_purge(chip, port, mdb->addr, mdb->vid, 0);
mv88e6xxx_reg_unlock(chip);
return err;
@@ -4691,6 +4950,8 @@ static const struct dsa_switch_ops mv88e6xxx_switch_ops = {
.set_eeprom = mv88e6xxx_set_eeprom,
.get_regs_len = mv88e6xxx_get_regs_len,
.get_regs = mv88e6xxx_get_regs,
+ .get_rxnfc = mv88e6xxx_get_rxnfc,
+ .set_rxnfc = mv88e6xxx_set_rxnfc,
.set_ageing_time = mv88e6xxx_set_ageing_time,
.port_bridge_join = mv88e6xxx_port_bridge_join,
.port_bridge_leave = mv88e6xxx_port_bridge_leave,
diff --git a/drivers/net/dsa/mv88e6xxx/chip.h b/drivers/net/dsa/mv88e6xxx/chip.h
index a406be2..e9b1a1a 100644
--- a/drivers/net/dsa/mv88e6xxx/chip.h
+++ b/drivers/net/dsa/mv88e6xxx/chip.h
@@ -8,6 +8,7 @@
#ifndef _MV88E6XXX_CHIP_H
#define _MV88E6XXX_CHIP_H
+#include <linux/idr.h>
#include <linux/if_vlan.h>
#include <linux/irq.h>
#include <linux/gpio/consumer.h>
@@ -189,6 +190,33 @@ struct mv88e6xxx_port_hwtstamp {
struct hwtstamp_config tstamp_config;
};
+enum mv88e6xxx_policy_mapping {
+ MV88E6XXX_POLICY_MAPPING_DA,
+ MV88E6XXX_POLICY_MAPPING_SA,
+ MV88E6XXX_POLICY_MAPPING_VTU,
+ MV88E6XXX_POLICY_MAPPING_ETYPE,
+ MV88E6XXX_POLICY_MAPPING_PPPOE,
+ MV88E6XXX_POLICY_MAPPING_VBAS,
+ MV88E6XXX_POLICY_MAPPING_OPT82,
+ MV88E6XXX_POLICY_MAPPING_UDP,
+};
+
+enum mv88e6xxx_policy_action {
+ MV88E6XXX_POLICY_ACTION_NORMAL,
+ MV88E6XXX_POLICY_ACTION_MIRROR,
+ MV88E6XXX_POLICY_ACTION_TRAP,
+ MV88E6XXX_POLICY_ACTION_DISCARD,
+};
+
+struct mv88e6xxx_policy {
+ enum mv88e6xxx_policy_mapping mapping;
+ enum mv88e6xxx_policy_action action;
+ struct ethtool_rx_flow_spec fs;
+ u8 addr[ETH_ALEN];
+ int port;
+ u16 vid;
+};
+
struct mv88e6xxx_port {
struct mv88e6xxx_chip *chip;
int port;
@@ -199,7 +227,7 @@ struct mv88e6xxx_port {
u64 vtu_member_violation;
u64 vtu_miss_violation;
u8 cmode;
- int serdes_irq;
+ unsigned int serdes_irq;
};
struct mv88e6xxx_chip {
@@ -247,6 +275,9 @@ struct mv88e6xxx_chip {
/* List of mdio busses */
struct list_head mdios;
+ /* Policy Control List IDs and rules */
+ struct idr policies;
+
/* There can be two interrupt controllers, which are chained
* off a GPIO as interrupt source
*/
@@ -381,6 +412,10 @@ struct mv88e6xxx_ops {
int (*port_tag_remap)(struct mv88e6xxx_chip *chip, int port);
+ int (*port_set_policy)(struct mv88e6xxx_chip *chip, int port,
+ enum mv88e6xxx_policy_mapping mapping,
+ enum mv88e6xxx_policy_action action);
+
int (*port_set_frame_mode)(struct mv88e6xxx_chip *chip, int port,
enum mv88e6xxx_frame_mode mode);
int (*port_set_egress_floods)(struct mv88e6xxx_chip *chip, int port,
@@ -441,11 +476,19 @@ struct mv88e6xxx_ops {
int (*mgmt_rsvd2cpu)(struct mv88e6xxx_chip *chip);
/* Power on/off a SERDES interface */
- int (*serdes_power)(struct mv88e6xxx_chip *chip, int port, bool on);
+ int (*serdes_power)(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool up);
+
+ /* SERDES lane mapping */
+ u8 (*serdes_get_lane)(struct mv88e6xxx_chip *chip, int port);
/* SERDES interrupt handling */
- int (*serdes_irq_setup)(struct mv88e6xxx_chip *chip, int port);
- void (*serdes_irq_free)(struct mv88e6xxx_chip *chip, int port);
+ unsigned int (*serdes_irq_mapping)(struct mv88e6xxx_chip *chip,
+ int port);
+ int (*serdes_irq_enable)(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable);
+ irqreturn_t (*serdes_irq_status)(struct mv88e6xxx_chip *chip, int port,
+ u8 lane);
/* Statistics from the SERDES interface */
int (*serdes_get_sset_count)(struct mv88e6xxx_chip *chip, int port);
diff --git a/drivers/net/dsa/mv88e6xxx/global1.h b/drivers/net/dsa/mv88e6xxx/global1.h
index 78b9ae2..0870fcc 100644
--- a/drivers/net/dsa/mv88e6xxx/global1.h
+++ b/drivers/net/dsa/mv88e6xxx/global1.h
@@ -128,19 +128,36 @@
#define MV88E6XXX_G1_ATU_OP_FULL_VIOLATION BIT(4)
/* Offset 0x0C: ATU Data Register */
-#define MV88E6XXX_G1_ATU_DATA 0x0c
-#define MV88E6XXX_G1_ATU_DATA_TRUNK 0x8000
-#define MV88E6XXX_G1_ATU_DATA_TRUNK_ID_MASK 0x00f0
-#define MV88E6XXX_G1_ATU_DATA_PORT_VECTOR_MASK 0x3ff0
-#define MV88E6XXX_G1_ATU_DATA_STATE_MASK 0x000f
-#define MV88E6XXX_G1_ATU_DATA_STATE_UNUSED 0x0000
-#define MV88E6XXX_G1_ATU_DATA_STATE_UC_MGMT 0x000d
-#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC 0x000e
-#define MV88E6XXX_G1_ATU_DATA_STATE_UC_PRIO_OVER 0x000f
-#define MV88E6XXX_G1_ATU_DATA_STATE_MC_NONE_RATE 0x0005
-#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC 0x0007
-#define MV88E6XXX_G1_ATU_DATA_STATE_MC_MGMT 0x000e
-#define MV88E6XXX_G1_ATU_DATA_STATE_MC_PRIO_OVER 0x000f
+#define MV88E6XXX_G1_ATU_DATA 0x0c
+#define MV88E6XXX_G1_ATU_DATA_TRUNK 0x8000
+#define MV88E6XXX_G1_ATU_DATA_TRUNK_ID_MASK 0x00f0
+#define MV88E6XXX_G1_ATU_DATA_PORT_VECTOR_MASK 0x3ff0
+#define MV88E6XXX_G1_ATU_DATA_STATE_MASK 0x000f
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_UNUSED 0x0000
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_1_OLDEST 0x0001
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_2 0x0002
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_3 0x0003
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_4 0x0004
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_5 0x0005
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_6 0x0006
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_AGE_7_NEWEST 0x0007
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_POLICY 0x0008
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_POLICY_PO 0x0009
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_AVB_NRL 0x000a
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_AVB_NRL_PO 0x000b
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_DA_MGMT 0x000c
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_DA_MGMT_PO 0x000d
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC 0x000e
+#define MV88E6XXX_G1_ATU_DATA_STATE_UC_STATIC_PO 0x000f
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_UNUSED 0x0000
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_POLICY 0x0004
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_AVB_NRL 0x0005
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_DA_MGMT 0x0006
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC 0x0007
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_POLICY_PO 0x000c
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_AVB_NRL_PO 0x000d
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_DA_MGMT_PO 0x000e
+#define MV88E6XXX_G1_ATU_DATA_STATE_MC_STATIC_PO 0x000f
/* Offset 0x0D: ATU MAC Address Register Bytes 0 & 1
* Offset 0x0E: ATU MAC Address Register Bytes 2 & 3
diff --git a/drivers/net/dsa/mv88e6xxx/global1_atu.c b/drivers/net/dsa/mv88e6xxx/global1_atu.c
index 18b8651..792a96e 100644
--- a/drivers/net/dsa/mv88e6xxx/global1_atu.c
+++ b/drivers/net/dsa/mv88e6xxx/global1_atu.c
@@ -135,7 +135,7 @@ static int mv88e6xxx_g1_atu_data_read(struct mv88e6xxx_chip *chip,
return err;
entry->state = val & 0xf;
- if (entry->state != MV88E6XXX_G1_ATU_DATA_STATE_UNUSED) {
+ if (entry->state) {
entry->trunk = !!(val & MV88E6XXX_G1_ATU_DATA_TRUNK);
entry->portvec = (val >> 4) & mv88e6xxx_port_mask(chip);
}
@@ -148,7 +148,7 @@ static int mv88e6xxx_g1_atu_data_write(struct mv88e6xxx_chip *chip,
{
u16 data = entry->state & 0xf;
- if (entry->state != MV88E6XXX_G1_ATU_DATA_STATE_UNUSED) {
+ if (entry->state) {
if (entry->trunk)
data |= MV88E6XXX_G1_ATU_DATA_TRUNK;
@@ -209,7 +209,7 @@ int mv88e6xxx_g1_atu_getnext(struct mv88e6xxx_chip *chip, u16 fid,
return err;
/* Write the MAC address to iterate from only once */
- if (entry->state == MV88E6XXX_G1_ATU_DATA_STATE_UNUSED) {
+ if (!entry->state) {
err = mv88e6xxx_g1_atu_mac_write(chip, entry);
if (err)
return err;
diff --git a/drivers/net/dsa/mv88e6xxx/port.c b/drivers/net/dsa/mv88e6xxx/port.c
index c95cdb7..15ef816 100644
--- a/drivers/net/dsa/mv88e6xxx/port.c
+++ b/drivers/net/dsa/mv88e6xxx/port.c
@@ -392,17 +392,14 @@ phy_interface_t mv88e6390x_port_max_speed_mode(int port)
return PHY_INTERFACE_MODE_NA;
}
-int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
- phy_interface_t mode)
+static int mv88e6xxx_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
+ phy_interface_t mode)
{
- int lane;
+ u8 lane;
u16 cmode;
u16 reg;
int err;
- if (port != 9 && port != 10)
- return -EOPNOTSUPP;
-
/* Default to a slow mode, so freeing up SERDES interfaces for
* other ports which might use them for SFPs.
*/
@@ -411,7 +408,7 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
switch (mode) {
case PHY_INTERFACE_MODE_1000BASEX:
- cmode = MV88E6XXX_PORT_STS_CMODE_1000BASE_X;
+ cmode = MV88E6XXX_PORT_STS_CMODE_1000BASEX;
break;
case PHY_INTERFACE_MODE_SGMII:
cmode = MV88E6XXX_PORT_STS_CMODE_SGMII;
@@ -434,18 +431,15 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
if (cmode == chip->ports[port].cmode)
return 0;
- lane = mv88e6390x_serdes_get_lane(chip, port);
- if (lane < 0 && lane != -ENODEV)
- return lane;
-
- if (lane >= 0) {
+ lane = mv88e6xxx_serdes_get_lane(chip, port);
+ if (lane) {
if (chip->ports[port].serdes_irq) {
- err = mv88e6390_serdes_irq_disable(chip, port, lane);
+ err = mv88e6xxx_serdes_irq_disable(chip, port, lane);
if (err)
return err;
}
- err = mv88e6390x_serdes_power(chip, port, false);
+ err = mv88e6xxx_serdes_power_down(chip, port, lane);
if (err)
return err;
}
@@ -466,16 +460,16 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
chip->ports[port].cmode = cmode;
- lane = mv88e6390x_serdes_get_lane(chip, port);
- if (lane < 0)
- return lane;
+ lane = mv88e6xxx_serdes_get_lane(chip, port);
+ if (!lane)
+ return -ENODEV;
- err = mv88e6390x_serdes_power(chip, port, true);
+ err = mv88e6xxx_serdes_power_up(chip, port, lane);
if (err)
return err;
if (chip->ports[port].serdes_irq) {
- err = mv88e6390_serdes_irq_enable(chip, port, lane);
+ err = mv88e6xxx_serdes_irq_enable(chip, port, lane);
if (err)
return err;
}
@@ -484,9 +478,21 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
return 0;
}
+int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
+ phy_interface_t mode)
+{
+ if (port != 9 && port != 10)
+ return -EOPNOTSUPP;
+
+ return mv88e6xxx_port_set_cmode(chip, port, mode);
+}
+
int mv88e6390_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
phy_interface_t mode)
{
+ if (port != 9 && port != 10)
+ return -EOPNOTSUPP;
+
switch (mode) {
case PHY_INTERFACE_MODE_NA:
return 0;
@@ -498,7 +504,58 @@ int mv88e6390_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
break;
}
- return mv88e6390x_port_set_cmode(chip, port, mode);
+ return mv88e6xxx_port_set_cmode(chip, port, mode);
+}
+
+static int mv88e6341_port_set_cmode_writable(struct mv88e6xxx_chip *chip,
+ int port)
+{
+ int err, addr;
+ u16 reg, bits;
+
+ if (port != 5)
+ return -EOPNOTSUPP;
+
+ addr = chip->info->port_base_addr + port;
+
+ err = mv88e6xxx_port_hidden_read(chip, 0x7, addr, 0, ®);
+ if (err)
+ return err;
+
+ bits = MV88E6341_PORT_RESERVED_1A_FORCE_CMODE |
+ MV88E6341_PORT_RESERVED_1A_SGMII_AN;
+
+ if ((reg & bits) == bits)
+ return 0;
+
+ reg |= bits;
+ return mv88e6xxx_port_hidden_write(chip, 0x7, addr, 0, reg);
+}
+
+int mv88e6341_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
+ phy_interface_t mode)
+{
+ int err;
+
+ if (port != 5)
+ return -EOPNOTSUPP;
+
+ switch (mode) {
+ case PHY_INTERFACE_MODE_NA:
+ return 0;
+ case PHY_INTERFACE_MODE_XGMII:
+ case PHY_INTERFACE_MODE_XAUI:
+ case PHY_INTERFACE_MODE_RXAUI:
+ return -EINVAL;
+ default:
+ break;
+ }
+
+ err = mv88e6341_port_set_cmode_writable(chip, port);
+ if (err)
+ return err;
+
+ return mv88e6xxx_port_set_cmode(chip, port, mode);
}
int mv88e6185_port_get_cmode(struct mv88e6xxx_chip *chip, int port, u8 *cmode)
@@ -618,7 +675,7 @@ int mv88e6352_port_link_state(struct mv88e6xxx_chip *chip, int port,
else
state->interface = PHY_INTERFACE_MODE_RGMII;
break;
- case MV88E6XXX_PORT_STS_CMODE_1000BASE_X:
+ case MV88E6XXX_PORT_STS_CMODE_1000BASEX:
state->interface = PHY_INTERFACE_MODE_1000BASEX;
break;
case MV88E6XXX_PORT_STS_CMODE_SGMII:
@@ -1284,3 +1341,77 @@ int mv88e6390_port_tag_remap(struct mv88e6xxx_chip *chip, int port)
return 0;
}
+
+/* Offset 0x0E: Policy Control Register */
+
+int mv88e6352_port_set_policy(struct mv88e6xxx_chip *chip, int port,
+ enum mv88e6xxx_policy_mapping mapping,
+ enum mv88e6xxx_policy_action action)
+{
+ u16 reg, mask, val;
+ int shift;
+ int err;
+
+ switch (mapping) {
+ case MV88E6XXX_POLICY_MAPPING_DA:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_DA_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_DA_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_SA:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_SA_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_SA_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_VTU:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_VTU_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_VTU_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_ETYPE:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_ETYPE_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_ETYPE_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_PPPOE:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_PPPOE_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_PPPOE_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_VBAS:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_VBAS_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_VBAS_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_OPT82:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_OPT82_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_OPT82_MASK;
+ break;
+ case MV88E6XXX_POLICY_MAPPING_UDP:
+ shift = __bf_shf(MV88E6XXX_PORT_POLICY_CTL_UDP_MASK);
+ mask = MV88E6XXX_PORT_POLICY_CTL_UDP_MASK;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ switch (action) {
+ case MV88E6XXX_POLICY_ACTION_NORMAL:
+ val = MV88E6XXX_PORT_POLICY_CTL_NORMAL;
+ break;
+ case MV88E6XXX_POLICY_ACTION_MIRROR:
+ val = MV88E6XXX_PORT_POLICY_CTL_MIRROR;
+ break;
+ case MV88E6XXX_POLICY_ACTION_TRAP:
+ val = MV88E6XXX_PORT_POLICY_CTL_TRAP;
+ break;
+ case MV88E6XXX_POLICY_ACTION_DISCARD:
+ val = MV88E6XXX_PORT_POLICY_CTL_DISCARD;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ err = mv88e6xxx_port_read(chip, port, MV88E6XXX_PORT_POLICY_CTL, ®);
+ if (err)
+ return err;
+
+ reg &= ~mask;
+ reg |= (val << shift) & mask;
+
+ return mv88e6xxx_port_write(chip, port, MV88E6XXX_PORT_POLICY_CTL, reg);
+}
diff --git a/drivers/net/dsa/mv88e6xxx/port.h b/drivers/net/dsa/mv88e6xxx/port.h
index 1abf5ea..03a480c 100644
--- a/drivers/net/dsa/mv88e6xxx/port.h
+++ b/drivers/net/dsa/mv88e6xxx/port.h
@@ -43,8 +43,8 @@
#define MV88E6XXX_PORT_STS_FLOW_CTL 0x0010
#define MV88E6XXX_PORT_STS_CMODE_MASK 0x000f
#define MV88E6XXX_PORT_STS_CMODE_RGMII 0x0007
-#define MV88E6XXX_PORT_STS_CMODE_100BASE_X 0x0008
-#define MV88E6XXX_PORT_STS_CMODE_1000BASE_X 0x0009
+#define MV88E6XXX_PORT_STS_CMODE_100BASEX 0x0008
+#define MV88E6XXX_PORT_STS_CMODE_1000BASEX 0x0009
#define MV88E6XXX_PORT_STS_CMODE_SGMII 0x000a
#define MV88E6XXX_PORT_STS_CMODE_2500BASEX 0x000b
#define MV88E6XXX_PORT_STS_CMODE_XAUI 0x000c
@@ -222,7 +222,19 @@
#define MV88E6XXX_PORT_PRI_OVERRIDE 0x0d
/* Offset 0x0E: Policy Control Register */
-#define MV88E6XXX_PORT_POLICY_CTL 0x0e
+#define MV88E6XXX_PORT_POLICY_CTL 0x0e
+#define MV88E6XXX_PORT_POLICY_CTL_DA_MASK 0xc000
+#define MV88E6XXX_PORT_POLICY_CTL_SA_MASK 0x3000
+#define MV88E6XXX_PORT_POLICY_CTL_VTU_MASK 0x0c00
+#define MV88E6XXX_PORT_POLICY_CTL_ETYPE_MASK 0x0300
+#define MV88E6XXX_PORT_POLICY_CTL_PPPOE_MASK 0x00c0
+#define MV88E6XXX_PORT_POLICY_CTL_VBAS_MASK 0x0030
+#define MV88E6XXX_PORT_POLICY_CTL_OPT82_MASK 0x000c
+#define MV88E6XXX_PORT_POLICY_CTL_UDP_MASK 0x0003
+#define MV88E6XXX_PORT_POLICY_CTL_NORMAL 0x0000
+#define MV88E6XXX_PORT_POLICY_CTL_MIRROR 0x0001
+#define MV88E6XXX_PORT_POLICY_CTL_TRAP 0x0002
+#define MV88E6XXX_PORT_POLICY_CTL_DISCARD 0x0003
/* Offset 0x0F: Port Special Ether Type */
#define MV88E6XXX_PORT_ETH_TYPE 0x0f
@@ -261,14 +273,16 @@
#define MV88E6095_PORT_IEEE_PRIO_REMAP_4567 0x19
/* Offset 0x1a: Magic undocumented errata register */
-#define PORT_RESERVED_1A 0x1a
-#define PORT_RESERVED_1A_BUSY BIT(15)
-#define PORT_RESERVED_1A_WRITE BIT(14)
-#define PORT_RESERVED_1A_READ 0
-#define PORT_RESERVED_1A_PORT_SHIFT 5
-#define PORT_RESERVED_1A_BLOCK (0xf << 10)
-#define PORT_RESERVED_1A_CTRL_PORT 4
-#define PORT_RESERVED_1A_DATA_PORT 5
+#define MV88E6XXX_PORT_RESERVED_1A 0x1a
+#define MV88E6XXX_PORT_RESERVED_1A_BUSY 0x8000
+#define MV88E6XXX_PORT_RESERVED_1A_WRITE 0x4000
+#define MV88E6XXX_PORT_RESERVED_1A_READ 0x0000
+#define MV88E6XXX_PORT_RESERVED_1A_PORT_SHIFT 5
+#define MV88E6XXX_PORT_RESERVED_1A_BLOCK_SHIFT 10
+#define MV88E6XXX_PORT_RESERVED_1A_CTRL_PORT 0x04
+#define MV88E6XXX_PORT_RESERVED_1A_DATA_PORT 0x05
+#define MV88E6341_PORT_RESERVED_1A_FORCE_CMODE 0x8000
+#define MV88E6341_PORT_RESERVED_1A_SGMII_AN 0x2000
int mv88e6xxx_port_read(struct mv88e6xxx_chip *chip, int port, int reg,
u16 *val);
@@ -322,6 +336,9 @@ int mv88e6185_port_set_egress_floods(struct mv88e6xxx_chip *chip, int port,
bool unicast, bool multicast);
int mv88e6352_port_set_egress_floods(struct mv88e6xxx_chip *chip, int port,
bool unicast, bool multicast);
+int mv88e6352_port_set_policy(struct mv88e6xxx_chip *chip, int port,
+ enum mv88e6xxx_policy_mapping mapping,
+ enum mv88e6xxx_policy_action action);
int mv88e6351_port_set_ether_type(struct mv88e6xxx_chip *chip, int port,
u16 etype);
int mv88e6xxx_port_set_message_port(struct mv88e6xxx_chip *chip, int port,
@@ -334,6 +351,8 @@ int mv88e6097_port_pause_limit(struct mv88e6xxx_chip *chip, int port, u8 in,
u8 out);
int mv88e6390_port_pause_limit(struct mv88e6xxx_chip *chip, int port, u8 in,
u8 out);
+int mv88e6341_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
+ phy_interface_t mode);
int mv88e6390_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
phy_interface_t mode);
int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port,
@@ -353,4 +372,10 @@ int mv88e6095_port_set_upstream_port(struct mv88e6xxx_chip *chip, int port,
int mv88e6xxx_port_disable_learn_limit(struct mv88e6xxx_chip *chip, int port);
int mv88e6xxx_port_disable_pri_override(struct mv88e6xxx_chip *chip, int port);
+int mv88e6xxx_port_hidden_write(struct mv88e6xxx_chip *chip, int block,
+ int port, int reg, u16 val);
+int mv88e6xxx_port_hidden_wait(struct mv88e6xxx_chip *chip);
+int mv88e6xxx_port_hidden_read(struct mv88e6xxx_chip *chip, int block, int port,
+ int reg, u16 *val);
+
#endif /* _MV88E6XXX_PORT_H */
diff --git a/drivers/net/dsa/mv88e6xxx/port_hidden.c b/drivers/net/dsa/mv88e6xxx/port_hidden.c
new file mode 100644
index 0000000..b49d05f
--- /dev/null
+++ b/drivers/net/dsa/mv88e6xxx/port_hidden.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Marvell 88E6xxx Switch Hidden Registers support
+ *
+ * Copyright (c) 2008 Marvell Semiconductor
+ *
+ * Copyright (c) 2019 Andrew Lunn <andrew@lunn.ch>
+ */
+
+#include <linux/bitfield.h>
+
+#include "chip.h"
+#include "port.h"
+
+/* The mv88e6390 and mv88e6341 have some hidden registers used for debug and
+ * development. The errata also makes use of them.
+ */
+int mv88e6xxx_port_hidden_write(struct mv88e6xxx_chip *chip, int block,
+ int port, int reg, u16 val)
+{
+ u16 ctrl;
+ int err;
+
+ err = mv88e6xxx_port_write(chip, MV88E6XXX_PORT_RESERVED_1A_DATA_PORT,
+ MV88E6XXX_PORT_RESERVED_1A, val);
+ if (err)
+ return err;
+
+ ctrl = MV88E6XXX_PORT_RESERVED_1A_BUSY |
+ MV88E6XXX_PORT_RESERVED_1A_WRITE |
+ block << MV88E6XXX_PORT_RESERVED_1A_BLOCK_SHIFT |
+ port << MV88E6XXX_PORT_RESERVED_1A_PORT_SHIFT |
+ reg;
+
+ return mv88e6xxx_port_write(chip, MV88E6XXX_PORT_RESERVED_1A_CTRL_PORT,
+ MV88E6XXX_PORT_RESERVED_1A, ctrl);
+}
+
+int mv88e6xxx_port_hidden_wait(struct mv88e6xxx_chip *chip)
+{
+ int bit = __bf_shf(MV88E6XXX_PORT_RESERVED_1A_BUSY);
+
+ return mv88e6xxx_wait_bit(chip, MV88E6XXX_PORT_RESERVED_1A_CTRL_PORT,
+ MV88E6XXX_PORT_RESERVED_1A, bit, 0);
+}
+
+int mv88e6xxx_port_hidden_read(struct mv88e6xxx_chip *chip, int block, int port,
+ int reg, u16 *val)
+{
+ u16 ctrl;
+ int err;
+
+ ctrl = MV88E6XXX_PORT_RESERVED_1A_BUSY |
+ MV88E6XXX_PORT_RESERVED_1A_READ |
+ block << MV88E6XXX_PORT_RESERVED_1A_BLOCK_SHIFT |
+ port << MV88E6XXX_PORT_RESERVED_1A_PORT_SHIFT |
+ reg;
+
+ err = mv88e6xxx_port_write(chip, MV88E6XXX_PORT_RESERVED_1A_CTRL_PORT,
+ MV88E6XXX_PORT_RESERVED_1A, ctrl);
+ if (err)
+ return err;
+
+ err = mv88e6xxx_port_hidden_wait(chip);
+ if (err)
+ return err;
+
+ return mv88e6xxx_port_read(chip, MV88E6XXX_PORT_RESERVED_1A_DATA_PORT,
+ MV88E6XXX_PORT_RESERVED_1A, val);
+}
diff --git a/drivers/net/dsa/mv88e6xxx/serdes.c b/drivers/net/dsa/mv88e6xxx/serdes.c
index 20c526c..902feb3 100644
--- a/drivers/net/dsa/mv88e6xxx/serdes.c
+++ b/drivers/net/dsa/mv88e6xxx/serdes.c
@@ -49,7 +49,8 @@ static int mv88e6390_serdes_write(struct mv88e6xxx_chip *chip,
return mv88e6xxx_phy_write(chip, lane, reg_c45, val);
}
-static int mv88e6352_serdes_power_set(struct mv88e6xxx_chip *chip, bool on)
+int mv88e6352_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool up)
{
u16 val, new_val;
int err;
@@ -58,7 +59,7 @@ static int mv88e6352_serdes_power_set(struct mv88e6xxx_chip *chip, bool on)
if (err)
return err;
- if (on)
+ if (up)
new_val = val & ~BMCR_PDOWN;
else
new_val = val | BMCR_PDOWN;
@@ -69,31 +70,27 @@ static int mv88e6352_serdes_power_set(struct mv88e6xxx_chip *chip, bool on)
return err;
}
-static bool mv88e6352_port_has_serdes(struct mv88e6xxx_chip *chip, int port)
+u8 mv88e6352_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
{
u8 cmode = chip->ports[port].cmode;
+ u8 lane = 0;
- if ((cmode == MV88E6XXX_PORT_STS_CMODE_100BASE_X) ||
- (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASE_X) ||
+ if ((cmode == MV88E6XXX_PORT_STS_CMODE_100BASEX) ||
+ (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASEX) ||
(cmode == MV88E6XXX_PORT_STS_CMODE_SGMII))
+ lane = 0xff; /* Unused */
+
+ return lane;
+}
+
+static bool mv88e6352_port_has_serdes(struct mv88e6xxx_chip *chip, int port)
+{
+ if (mv88e6xxx_serdes_get_lane(chip, port))
return true;
return false;
}
-int mv88e6352_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on)
-{
- int err;
-
- if (mv88e6352_port_has_serdes(chip, port)) {
- err = mv88e6352_serdes_power_set(chip, on);
- if (err < 0)
- return err;
- }
-
- return 0;
-}
-
struct mv88e6352_serdes_hw_stat {
char string[ETH_GSTRING_LEN];
int sizeof_stat;
@@ -186,214 +183,178 @@ static void mv88e6352_serdes_irq_link(struct mv88e6xxx_chip *chip, int port)
struct dsa_switch *ds = chip->ds;
u16 status;
bool up;
+ int err;
- mv88e6352_serdes_read(chip, MII_BMSR, &status);
+ err = mv88e6352_serdes_read(chip, MII_BMSR, &status);
+ if (err)
+ return;
/* Status must be read twice in order to give the current link
* status. Otherwise the change in link status since the last
* read of the register is returned.
*/
- mv88e6352_serdes_read(chip, MII_BMSR, &status);
+ err = mv88e6352_serdes_read(chip, MII_BMSR, &status);
+ if (err)
+ return;
up = status & BMSR_LSTATUS;
dsa_port_phylink_mac_change(ds, port, up);
}
-static irqreturn_t mv88e6352_serdes_thread_fn(int irq, void *dev_id)
+irqreturn_t mv88e6352_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
+ u8 lane)
{
- struct mv88e6xxx_port *port = dev_id;
- struct mv88e6xxx_chip *chip = port->chip;
irqreturn_t ret = IRQ_NONE;
u16 status;
int err;
- mv88e6xxx_reg_lock(chip);
-
err = mv88e6352_serdes_read(chip, MV88E6352_SERDES_INT_STATUS, &status);
if (err)
- goto out;
+ return ret;
if (status & MV88E6352_SERDES_INT_LINK_CHANGE) {
ret = IRQ_HANDLED;
- mv88e6352_serdes_irq_link(chip, port->port);
+ mv88e6352_serdes_irq_link(chip, port);
}
-out:
- mv88e6xxx_reg_unlock(chip);
return ret;
}
-static int mv88e6352_serdes_irq_enable(struct mv88e6xxx_chip *chip)
+int mv88e6352_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable)
{
- return mv88e6352_serdes_write(chip, MV88E6352_SERDES_INT_ENABLE,
- MV88E6352_SERDES_INT_LINK_CHANGE);
+ u16 val = 0;
+
+ if (enable)
+ val |= MV88E6352_SERDES_INT_LINK_CHANGE;
+
+ return mv88e6352_serdes_write(chip, MV88E6352_SERDES_INT_ENABLE, val);
}
-static int mv88e6352_serdes_irq_disable(struct mv88e6xxx_chip *chip)
+unsigned int mv88e6352_serdes_irq_mapping(struct mv88e6xxx_chip *chip, int port)
{
- return mv88e6352_serdes_write(chip, MV88E6352_SERDES_INT_ENABLE, 0);
+ return irq_find_mapping(chip->g2_irq.domain, MV88E6352_SERDES_IRQ);
}
-int mv88e6352_serdes_irq_setup(struct mv88e6xxx_chip *chip, int port)
-{
- int err;
-
- if (!mv88e6352_port_has_serdes(chip, port))
- return 0;
-
- chip->ports[port].serdes_irq = irq_find_mapping(chip->g2_irq.domain,
- MV88E6352_SERDES_IRQ);
- if (chip->ports[port].serdes_irq < 0) {
- dev_err(chip->dev, "Unable to map SERDES irq: %d\n",
- chip->ports[port].serdes_irq);
- return chip->ports[port].serdes_irq;
- }
-
- /* Requesting the IRQ will trigger irq callbacks. So we cannot
- * hold the reg_lock.
- */
- mv88e6xxx_reg_unlock(chip);
- err = request_threaded_irq(chip->ports[port].serdes_irq, NULL,
- mv88e6352_serdes_thread_fn,
- IRQF_ONESHOT, "mv88e6xxx-serdes",
- &chip->ports[port]);
- mv88e6xxx_reg_lock(chip);
-
- if (err) {
- dev_err(chip->dev, "Unable to request SERDES interrupt: %d\n",
- err);
- return err;
- }
-
- return mv88e6352_serdes_irq_enable(chip);
-}
-
-void mv88e6352_serdes_irq_free(struct mv88e6xxx_chip *chip, int port)
-{
- if (!mv88e6352_port_has_serdes(chip, port))
- return;
-
- mv88e6352_serdes_irq_disable(chip);
-
- /* Freeing the IRQ will trigger irq callbacks. So we cannot
- * hold the reg_lock.
- */
- mv88e6xxx_reg_unlock(chip);
- free_irq(chip->ports[port].serdes_irq, &chip->ports[port]);
- mv88e6xxx_reg_lock(chip);
-
- chip->ports[port].serdes_irq = 0;
-}
-
-/* Return the SERDES lane address a port is using. Only Ports 9 and 10
- * have SERDES lanes. Returns -ENODEV if a port does not have a lane.
- */
-static int mv88e6390_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
+u8 mv88e6341_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
{
u8 cmode = chip->ports[port].cmode;
+ u8 lane = 0;
+
+ switch (port) {
+ case 5:
+ if (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
+ cmode == MV88E6XXX_PORT_STS_CMODE_SGMII ||
+ cmode == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
+ lane = MV88E6341_PORT5_LANE;
+ break;
+ }
+
+ return lane;
+}
+
+u8 mv88e6390_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
+{
+ u8 cmode = chip->ports[port].cmode;
+ u8 lane = 0;
switch (port) {
case 9:
- if (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
- return MV88E6390_PORT9_LANE0;
- return -ENODEV;
+ lane = MV88E6390_PORT9_LANE0;
+ break;
case 10:
- if (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
- return MV88E6390_PORT10_LANE0;
- return -ENODEV;
- default:
- return -ENODEV;
+ lane = MV88E6390_PORT10_LANE0;
+ break;
}
+
+ return lane;
}
-/* Return the SERDES lane address a port is using. Ports 9 and 10 can
- * use multiple lanes. If so, return the first lane the port uses.
- * Returns -ENODEV if a port does not have a lane.
- */
-int mv88e6390x_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
+u8 mv88e6390x_serdes_get_lane(struct mv88e6xxx_chip *chip, int port)
{
- u8 cmode_port9, cmode_port10, cmode_port;
-
- cmode_port9 = chip->ports[9].cmode;
- cmode_port10 = chip->ports[10].cmode;
- cmode_port = chip->ports[port].cmode;
+ u8 cmode_port = chip->ports[port].cmode;
+ u8 cmode_port10 = chip->ports[10].cmode;
+ u8 cmode_port9 = chip->ports[9].cmode;
+ u8 lane = 0;
switch (port) {
case 2:
- if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
- if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASE_X)
- return MV88E6390_PORT9_LANE1;
- return -ENODEV;
+ if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASEX)
+ lane = MV88E6390_PORT9_LANE1;
+ break;
case 3:
- if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_2500BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_RXAUI)
- if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASE_X)
- return MV88E6390_PORT9_LANE2;
- return -ENODEV;
+ if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASEX)
+ lane = MV88E6390_PORT9_LANE2;
+ break;
case 4:
- if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_2500BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_RXAUI)
- if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASE_X)
- return MV88E6390_PORT9_LANE3;
- return -ENODEV;
+ if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASEX)
+ lane = MV88E6390_PORT9_LANE3;
+ break;
case 5:
- if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
- if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASE_X)
- return MV88E6390_PORT10_LANE1;
- return -ENODEV;
+ if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASEX)
+ lane = MV88E6390_PORT10_LANE1;
+ break;
case 6:
- if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_2500BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_RXAUI)
- if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASE_X)
- return MV88E6390_PORT10_LANE2;
- return -ENODEV;
+ if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASEX)
+ lane = MV88E6390_PORT10_LANE2;
+ break;
case 7:
- if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_2500BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_RXAUI)
- if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASE_X)
- return MV88E6390_PORT10_LANE3;
- return -ENODEV;
+ if (cmode_port == MV88E6XXX_PORT_STS_CMODE_1000BASEX)
+ lane = MV88E6390_PORT10_LANE3;
+ break;
case 9:
- if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port9 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_2500BASEX ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_XAUI ||
cmode_port9 == MV88E6XXX_PORT_STS_CMODE_RXAUI)
- return MV88E6390_PORT9_LANE0;
- return -ENODEV;
+ lane = MV88E6390_PORT9_LANE0;
+ break;
case 10:
- if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
+ if (cmode_port10 == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_SGMII ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_2500BASEX ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_XAUI ||
cmode_port10 == MV88E6XXX_PORT_STS_CMODE_RXAUI)
- return MV88E6390_PORT10_LANE0;
- return -ENODEV;
- default:
- return -ENODEV;
+ lane = MV88E6390_PORT10_LANE0;
+ break;
}
+
+ return lane;
}
-/* Set the power on/off for 10GBASE-R and 10GBASE-X4/X2 */
-static int mv88e6390_serdes_power_10g(struct mv88e6xxx_chip *chip, int lane,
- bool on)
+/* Set power up/down for 10GBASE-R and 10GBASE-X4/X2 */
+static int mv88e6390_serdes_power_10g(struct mv88e6xxx_chip *chip, u8 lane,
+ bool up)
{
u16 val, new_val;
int err;
@@ -404,7 +365,7 @@ static int mv88e6390_serdes_power_10g(struct mv88e6xxx_chip *chip, int lane,
if (err)
return err;
- if (on)
+ if (up)
new_val = val & ~(MV88E6390_PCS_CONTROL_1_RESET |
MV88E6390_PCS_CONTROL_1_LOOPBACK |
MV88E6390_PCS_CONTROL_1_PDOWN);
@@ -418,9 +379,9 @@ static int mv88e6390_serdes_power_10g(struct mv88e6xxx_chip *chip, int lane,
return err;
}
-/* Set the power on/off for SGMII and 1000Base-X */
-static int mv88e6390_serdes_power_sgmii(struct mv88e6xxx_chip *chip, int lane,
- bool on)
+/* Set power up/down for SGMII and 1000Base-X */
+static int mv88e6390_serdes_power_sgmii(struct mv88e6xxx_chip *chip, u8 lane,
+ bool up)
{
u16 val, new_val;
int err;
@@ -430,7 +391,7 @@ static int mv88e6390_serdes_power_sgmii(struct mv88e6xxx_chip *chip, int lane,
if (err)
return err;
- if (on)
+ if (up)
new_val = val & ~(MV88E6390_SGMII_CONTROL_RESET |
MV88E6390_SGMII_CONTROL_LOOPBACK |
MV88E6390_SGMII_CONTROL_PDOWN);
@@ -444,70 +405,32 @@ static int mv88e6390_serdes_power_sgmii(struct mv88e6xxx_chip *chip, int lane,
return err;
}
-static int mv88e6390_serdes_power_lane(struct mv88e6xxx_chip *chip, int port,
- int lane, bool on)
+int mv88e6390_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool up)
{
u8 cmode = chip->ports[port].cmode;
switch (cmode) {
case MV88E6XXX_PORT_STS_CMODE_SGMII:
- case MV88E6XXX_PORT_STS_CMODE_1000BASE_X:
+ case MV88E6XXX_PORT_STS_CMODE_1000BASEX:
case MV88E6XXX_PORT_STS_CMODE_2500BASEX:
- return mv88e6390_serdes_power_sgmii(chip, lane, on);
+ return mv88e6390_serdes_power_sgmii(chip, lane, up);
case MV88E6XXX_PORT_STS_CMODE_XAUI:
case MV88E6XXX_PORT_STS_CMODE_RXAUI:
- return mv88e6390_serdes_power_10g(chip, lane, on);
- }
-
- return 0;
-}
-
-int mv88e6390_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on)
-{
- int lane;
-
- lane = mv88e6390_serdes_get_lane(chip, port);
- if (lane == -ENODEV)
- return 0;
-
- if (lane < 0)
- return lane;
-
- switch (port) {
- case 9 ... 10:
- return mv88e6390_serdes_power_lane(chip, port, lane, on);
- }
-
- return 0;
-}
-
-int mv88e6390x_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on)
-{
- int lane;
-
- lane = mv88e6390x_serdes_get_lane(chip, port);
- if (lane == -ENODEV)
- return 0;
-
- if (lane < 0)
- return lane;
-
- switch (port) {
- case 2 ... 4:
- case 5 ... 7:
- case 9 ... 10:
- return mv88e6390_serdes_power_lane(chip, port, lane, on);
+ return mv88e6390_serdes_power_10g(chip, lane, up);
}
return 0;
}
static void mv88e6390_serdes_irq_link_sgmii(struct mv88e6xxx_chip *chip,
- int port, int lane)
+ int port, u8 lane)
{
+ u8 cmode = chip->ports[port].cmode;
struct dsa_switch *ds = chip->ds;
int duplex = DUPLEX_UNKNOWN;
int speed = SPEED_UNKNOWN;
+ phy_interface_t mode;
int link, err;
u16 status;
@@ -527,7 +450,10 @@ static void mv88e6390_serdes_irq_link_sgmii(struct mv88e6xxx_chip *chip,
switch (status & MV88E6390_SGMII_PHY_STATUS_SPEED_MASK) {
case MV88E6390_SGMII_PHY_STATUS_SPEED_1000:
- speed = SPEED_1000;
+ if (cmode == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
+ speed = SPEED_2500;
+ else
+ speed = SPEED_1000;
break;
case MV88E6390_SGMII_PHY_STATUS_SPEED_100:
speed = SPEED_100;
@@ -541,8 +467,22 @@ static void mv88e6390_serdes_irq_link_sgmii(struct mv88e6xxx_chip *chip,
}
}
+ switch (cmode) {
+ case MV88E6XXX_PORT_STS_CMODE_SGMII:
+ mode = PHY_INTERFACE_MODE_SGMII;
+ break;
+ case MV88E6XXX_PORT_STS_CMODE_1000BASEX:
+ mode = PHY_INTERFACE_MODE_1000BASEX;
+ break;
+ case MV88E6XXX_PORT_STS_CMODE_2500BASEX:
+ mode = PHY_INTERFACE_MODE_2500BASEX;
+ break;
+ default:
+ mode = PHY_INTERFACE_MODE_NA;
+ }
+
err = mv88e6xxx_port_setup_mac(chip, port, link, speed, duplex,
- PAUSE_OFF, PHY_INTERFACE_MODE_NA);
+ PAUSE_OFF, mode);
if (err)
dev_err(chip->dev, "can't propagate PHY settings to MAC: %d\n",
err);
@@ -551,55 +491,35 @@ static void mv88e6390_serdes_irq_link_sgmii(struct mv88e6xxx_chip *chip,
}
static int mv88e6390_serdes_irq_enable_sgmii(struct mv88e6xxx_chip *chip,
- int lane)
+ u8 lane, bool enable)
{
+ u16 val = 0;
+
+ if (enable)
+ val |= MV88E6390_SGMII_INT_LINK_DOWN |
+ MV88E6390_SGMII_INT_LINK_UP;
+
return mv88e6390_serdes_write(chip, lane, MDIO_MMD_PHYXS,
- MV88E6390_SGMII_INT_ENABLE,
- MV88E6390_SGMII_INT_LINK_DOWN |
- MV88E6390_SGMII_INT_LINK_UP);
+ MV88E6390_SGMII_INT_ENABLE, val);
}
-static int mv88e6390_serdes_irq_disable_sgmii(struct mv88e6xxx_chip *chip,
- int lane)
-{
- return mv88e6390_serdes_write(chip, lane, MDIO_MMD_PHYXS,
- MV88E6390_SGMII_INT_ENABLE, 0);
-}
-
-int mv88e6390_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port,
- int lane)
+int mv88e6390_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable)
{
u8 cmode = chip->ports[port].cmode;
- int err = 0;
switch (cmode) {
case MV88E6XXX_PORT_STS_CMODE_SGMII:
- case MV88E6XXX_PORT_STS_CMODE_1000BASE_X:
+ case MV88E6XXX_PORT_STS_CMODE_1000BASEX:
case MV88E6XXX_PORT_STS_CMODE_2500BASEX:
- err = mv88e6390_serdes_irq_enable_sgmii(chip, lane);
+ return mv88e6390_serdes_irq_enable_sgmii(chip, lane, enable);
}
- return err;
-}
-
-int mv88e6390_serdes_irq_disable(struct mv88e6xxx_chip *chip, int port,
- int lane)
-{
- u8 cmode = chip->ports[port].cmode;
- int err = 0;
-
- switch (cmode) {
- case MV88E6XXX_PORT_STS_CMODE_SGMII:
- case MV88E6XXX_PORT_STS_CMODE_1000BASE_X:
- case MV88E6XXX_PORT_STS_CMODE_2500BASEX:
- err = mv88e6390_serdes_irq_disable_sgmii(chip, lane);
- }
-
- return err;
+ return 0;
}
static int mv88e6390_serdes_irq_status_sgmii(struct mv88e6xxx_chip *chip,
- int lane, u16 *status)
+ u8 lane, u16 *status)
{
int err;
@@ -609,129 +529,32 @@ static int mv88e6390_serdes_irq_status_sgmii(struct mv88e6xxx_chip *chip,
return err;
}
-static irqreturn_t mv88e6390_serdes_thread_fn(int irq, void *dev_id)
+irqreturn_t mv88e6390_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
+ u8 lane)
{
- struct mv88e6xxx_port *port = dev_id;
- struct mv88e6xxx_chip *chip = port->chip;
+ u8 cmode = chip->ports[port].cmode;
irqreturn_t ret = IRQ_NONE;
- u8 cmode = port->cmode;
u16 status;
- int lane;
int err;
- lane = mv88e6390x_serdes_get_lane(chip, port->port);
-
- mv88e6xxx_reg_lock(chip);
-
switch (cmode) {
case MV88E6XXX_PORT_STS_CMODE_SGMII:
- case MV88E6XXX_PORT_STS_CMODE_1000BASE_X:
+ case MV88E6XXX_PORT_STS_CMODE_1000BASEX:
case MV88E6XXX_PORT_STS_CMODE_2500BASEX:
err = mv88e6390_serdes_irq_status_sgmii(chip, lane, &status);
if (err)
- goto out;
+ return ret;
if (status & (MV88E6390_SGMII_INT_LINK_DOWN |
MV88E6390_SGMII_INT_LINK_UP)) {
ret = IRQ_HANDLED;
- mv88e6390_serdes_irq_link_sgmii(chip, port->port, lane);
+ mv88e6390_serdes_irq_link_sgmii(chip, port, lane);
}
}
-out:
- mv88e6xxx_reg_unlock(chip);
return ret;
}
-int mv88e6390x_serdes_irq_setup(struct mv88e6xxx_chip *chip, int port)
+unsigned int mv88e6390_serdes_irq_mapping(struct mv88e6xxx_chip *chip, int port)
{
- int lane;
- int err;
-
- lane = mv88e6390x_serdes_get_lane(chip, port);
-
- if (lane == -ENODEV)
- return 0;
-
- if (lane < 0)
- return lane;
-
- chip->ports[port].serdes_irq = irq_find_mapping(chip->g2_irq.domain,
- port);
- if (chip->ports[port].serdes_irq < 0) {
- dev_err(chip->dev, "Unable to map SERDES irq: %d\n",
- chip->ports[port].serdes_irq);
- return chip->ports[port].serdes_irq;
- }
-
- /* Requesting the IRQ will trigger irq callbacks. So we cannot
- * hold the reg_lock.
- */
- mv88e6xxx_reg_unlock(chip);
- err = request_threaded_irq(chip->ports[port].serdes_irq, NULL,
- mv88e6390_serdes_thread_fn,
- IRQF_ONESHOT, "mv88e6xxx-serdes",
- &chip->ports[port]);
- mv88e6xxx_reg_lock(chip);
-
- if (err) {
- dev_err(chip->dev, "Unable to request SERDES interrupt: %d\n",
- err);
- return err;
- }
-
- return mv88e6390_serdes_irq_enable(chip, port, lane);
-}
-
-int mv88e6390_serdes_irq_setup(struct mv88e6xxx_chip *chip, int port)
-{
- if (port < 9)
- return 0;
-
- return mv88e6390x_serdes_irq_setup(chip, port);
-}
-
-void mv88e6390x_serdes_irq_free(struct mv88e6xxx_chip *chip, int port)
-{
- int lane = mv88e6390x_serdes_get_lane(chip, port);
-
- if (lane == -ENODEV)
- return;
-
- if (lane < 0)
- return;
-
- mv88e6390_serdes_irq_disable(chip, port, lane);
-
- /* Freeing the IRQ will trigger irq callbacks. So we cannot
- * hold the reg_lock.
- */
- mv88e6xxx_reg_unlock(chip);
- free_irq(chip->ports[port].serdes_irq, &chip->ports[port]);
- mv88e6xxx_reg_lock(chip);
-
- chip->ports[port].serdes_irq = 0;
-}
-
-void mv88e6390_serdes_irq_free(struct mv88e6xxx_chip *chip, int port)
-{
- if (port < 9)
- return;
-
- mv88e6390x_serdes_irq_free(chip, port);
-}
-
-int mv88e6341_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on)
-{
- u8 cmode = chip->ports[port].cmode;
-
- if (port != 5)
- return 0;
-
- if (cmode == MV88E6XXX_PORT_STS_CMODE_1000BASE_X ||
- cmode == MV88E6XXX_PORT_STS_CMODE_SGMII ||
- cmode == MV88E6XXX_PORT_STS_CMODE_2500BASEX)
- return mv88e6390_serdes_power_sgmii(chip, MV88E6341_ADDR_SERDES,
- on);
-
- return 0;
+ return irq_find_mapping(chip->g2_irq.domain, port);
}
diff --git a/drivers/net/dsa/mv88e6xxx/serdes.h b/drivers/net/dsa/mv88e6xxx/serdes.h
index ff5b944..bd8df36 100644
--- a/drivers/net/dsa/mv88e6xxx/serdes.h
+++ b/drivers/net/dsa/mv88e6xxx/serdes.h
@@ -28,7 +28,7 @@
#define MV88E6352_SERDES_INT_STATUS 0x13
-#define MV88E6341_ADDR_SERDES 0x15
+#define MV88E6341_PORT5_LANE 0x15
#define MV88E6390_PORT9_LANE0 0x09
#define MV88E6390_PORT9_LANE1 0x12
@@ -74,26 +74,94 @@
#define MV88E6390_SGMII_PHY_STATUS_SPD_DPL_VALID BIT(11)
#define MV88E6390_SGMII_PHY_STATUS_LINK BIT(10)
-int mv88e6390x_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
-int mv88e6341_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on);
-int mv88e6352_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on);
-int mv88e6390_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on);
-int mv88e6390x_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on);
-int mv88e6390_serdes_irq_setup(struct mv88e6xxx_chip *chip, int port);
-void mv88e6390_serdes_irq_free(struct mv88e6xxx_chip *chip, int port);
-int mv88e6390x_serdes_irq_setup(struct mv88e6xxx_chip *chip, int port);
-void mv88e6390x_serdes_irq_free(struct mv88e6xxx_chip *chip, int port);
+u8 mv88e6341_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
+u8 mv88e6352_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
+u8 mv88e6390_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
+u8 mv88e6390x_serdes_get_lane(struct mv88e6xxx_chip *chip, int port);
+unsigned int mv88e6352_serdes_irq_mapping(struct mv88e6xxx_chip *chip,
+ int port);
+unsigned int mv88e6390_serdes_irq_mapping(struct mv88e6xxx_chip *chip,
+ int port);
+int mv88e6352_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool on);
+int mv88e6390_serdes_power(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool on);
+int mv88e6352_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable);
+int mv88e6390_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port, u8 lane,
+ bool enable);
+irqreturn_t mv88e6352_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
+ u8 lane);
+irqreturn_t mv88e6390_serdes_irq_status(struct mv88e6xxx_chip *chip, int port,
+ u8 lane);
int mv88e6352_serdes_get_sset_count(struct mv88e6xxx_chip *chip, int port);
int mv88e6352_serdes_get_strings(struct mv88e6xxx_chip *chip,
int port, uint8_t *data);
int mv88e6352_serdes_get_stats(struct mv88e6xxx_chip *chip, int port,
uint64_t *data);
-int mv88e6390_serdes_irq_enable(struct mv88e6xxx_chip *chip, int port,
- int lane);
-int mv88e6390_serdes_irq_disable(struct mv88e6xxx_chip *chip, int port,
- int lane);
-int mv88e6352_serdes_irq_setup(struct mv88e6xxx_chip *chip, int port);
-void mv88e6352_serdes_irq_free(struct mv88e6xxx_chip *chip, int port);
+/* Return the (first) SERDES lane address a port is using, 0 otherwise. */
+static inline u8 mv88e6xxx_serdes_get_lane(struct mv88e6xxx_chip *chip,
+ int port)
+{
+ if (!chip->info->ops->serdes_get_lane)
+ return 0;
+
+ return chip->info->ops->serdes_get_lane(chip, port);
+}
+
+static inline int mv88e6xxx_serdes_power_up(struct mv88e6xxx_chip *chip,
+ int port, u8 lane)
+{
+ if (!chip->info->ops->serdes_power)
+ return -EOPNOTSUPP;
+
+ return chip->info->ops->serdes_power(chip, port, lane, true);
+}
+
+static inline int mv88e6xxx_serdes_power_down(struct mv88e6xxx_chip *chip,
+ int port, u8 lane)
+{
+ if (!chip->info->ops->serdes_power)
+ return -EOPNOTSUPP;
+
+ return chip->info->ops->serdes_power(chip, port, lane, false);
+}
+
+static inline unsigned int
+mv88e6xxx_serdes_irq_mapping(struct mv88e6xxx_chip *chip, int port)
+{
+ if (!chip->info->ops->serdes_irq_mapping)
+ return 0;
+
+ return chip->info->ops->serdes_irq_mapping(chip, port);
+}
+
+static inline int mv88e6xxx_serdes_irq_enable(struct mv88e6xxx_chip *chip,
+ int port, u8 lane)
+{
+ if (!chip->info->ops->serdes_irq_enable)
+ return -EOPNOTSUPP;
+
+ return chip->info->ops->serdes_irq_enable(chip, port, lane, true);
+}
+
+static inline int mv88e6xxx_serdes_irq_disable(struct mv88e6xxx_chip *chip,
+ int port, u8 lane)
+{
+ if (!chip->info->ops->serdes_irq_enable)
+ return -EOPNOTSUPP;
+
+ return chip->info->ops->serdes_irq_enable(chip, port, lane, false);
+}
+
+static inline irqreturn_t
+mv88e6xxx_serdes_irq_status(struct mv88e6xxx_chip *chip, int port, u8 lane)
+{
+ if (!chip->info->ops->serdes_irq_status)
+ return IRQ_NONE;
+
+ return chip->info->ops->serdes_irq_status(chip, port, lane);
+}
#endif
diff --git a/drivers/net/dsa/sja1105/sja1105_main.c b/drivers/net/dsa/sja1105/sja1105_main.c
index df976b25..d8cff01 100644
--- a/drivers/net/dsa/sja1105/sja1105_main.c
+++ b/drivers/net/dsa/sja1105/sja1105_main.c
@@ -1728,6 +1728,21 @@ static void sja1105_teardown(struct dsa_switch *ds)
sja1105_static_config_free(&priv->static_config);
}
+static int sja1105_port_enable(struct dsa_switch *ds, int port,
+ struct phy_device *phy)
+{
+ struct net_device *slave;
+
+ if (!dsa_is_user_port(ds, port))
+ return 0;
+
+ slave = ds->ports[port].slave;
+
+ slave->features &= ~NETIF_F_HW_VLAN_CTAG_FILTER;
+
+ return 0;
+}
+
static int sja1105_mgmt_xmit(struct dsa_switch *ds, int port, int slot,
struct sk_buff *skb, bool takets)
{
@@ -2049,6 +2064,7 @@ static const struct dsa_switch_ops sja1105_switch_ops = {
.get_ethtool_stats = sja1105_get_ethtool_stats,
.get_sset_count = sja1105_get_sset_count,
.get_ts_info = sja1105_get_ts_info,
+ .port_enable = sja1105_port_enable,
.port_fdb_dump = sja1105_fdb_dump,
.port_fdb_add = sja1105_fdb_add,
.port_fdb_del = sja1105_fdb_del,
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 93a2d4d..2830dc2 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -168,6 +168,7 @@
source "drivers/net/ethernet/packetengines/Kconfig"
source "drivers/net/ethernet/pasemi/Kconfig"
+source "drivers/net/ethernet/pensando/Kconfig"
source "drivers/net/ethernet/qlogic/Kconfig"
source "drivers/net/ethernet/qualcomm/Kconfig"
source "drivers/net/ethernet/rdc/Kconfig"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index fb9155c..061edd2 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -97,3 +97,4 @@
obj-$(CONFIG_NET_VENDOR_XILINX) += xilinx/
obj-$(CONFIG_NET_VENDOR_XIRCOM) += xircom/
obj-$(CONFIG_NET_VENDOR_SYNOPSYS) += synopsys/
+obj-$(CONFIG_NET_VENDOR_PENSANDO) += pensando/
diff --git a/drivers/net/ethernet/amd/ni65.c b/drivers/net/ethernet/amd/ni65.c
index 87ff5d6..c6c2a54 100644
--- a/drivers/net/ethernet/amd/ni65.c
+++ b/drivers/net/ethernet/amd/ni65.c
@@ -697,16 +697,14 @@ static void ni65_free_buffer(struct priv *p)
for(i=0;i<TMDNUM;i++) {
kfree(p->tmdbounce[i]);
#ifdef XMT_VIA_SKB
- if(p->tmd_skb[i])
- dev_kfree_skb(p->tmd_skb[i]);
+ dev_kfree_skb(p->tmd_skb[i]);
#endif
}
for(i=0;i<RMDNUM;i++)
{
#ifdef RCV_VIA_SKB
- if(p->recv_skb[i])
- dev_kfree_skb(p->recv_skb[i]);
+ dev_kfree_skb(p->recv_skb[i]);
#else
kfree(p->recvbounce[i]);
#endif
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-main.c b/drivers/net/ethernet/amd/xgbe/xgbe-main.c
index b41f236..7ce9c69 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-main.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-main.c
@@ -469,13 +469,19 @@ static int __init xgbe_mod_init(void)
ret = xgbe_platform_init();
if (ret)
- return ret;
+ goto err_platform_init;
ret = xgbe_pci_init();
if (ret)
- return ret;
+ goto err_pci_init;
return 0;
+
+err_pci_init:
+ xgbe_platform_exit();
+err_platform_init:
+ unregister_netdevice_notifier(&xgbe_netdev_notifier);
+ return ret;
}
static void __exit xgbe_mod_exit(void)
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_filters.c b/drivers/net/ethernet/aquantia/atlantic/aq_filters.c
index 440690b..aee827f 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_filters.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_filters.c
@@ -431,7 +431,8 @@ int aq_del_fvlan_by_vlan(struct aq_nic_s *aq_nic, u16 vlan_id)
if (be16_to_cpu(rule->aq_fsp.h_ext.vlan_tci) == vlan_id)
break;
}
- if (rule && be16_to_cpu(rule->aq_fsp.h_ext.vlan_tci) == vlan_id) {
+ if (rule && rule->type == aq_rx_filter_vlan &&
+ be16_to_cpu(rule->aq_fsp.h_ext.vlan_tci) == vlan_id) {
struct ethtool_rxnfc cmd;
cmd.fs.location = rule->aq_fsp.location;
@@ -843,7 +844,7 @@ int aq_filters_vlans_update(struct aq_nic_s *aq_nic)
return err;
if (aq_nic->ndev->features & NETIF_F_HW_VLAN_CTAG_FILTER) {
- if (hweight < AQ_VLAN_MAX_FILTERS && hweight > 0) {
+ if (hweight <= AQ_VLAN_MAX_FILTERS && hweight > 0) {
err = aq_hw_ops->hw_filter_vlan_ctrl(aq_hw,
!(aq_nic->packet_filter & IFF_PROMISC));
aq_nic->aq_nic_cfg.is_vlan_force_promisc = false;
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_main.c b/drivers/net/ethernet/aquantia/atlantic/aq_main.c
index 100722a..b4a0fb2 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_main.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_main.c
@@ -61,6 +61,10 @@ static int aq_ndev_open(struct net_device *ndev)
if (err < 0)
goto err_exit;
+ err = aq_filters_vlans_update(aq_nic);
+ if (err < 0)
+ goto err_exit;
+
err = aq_nic_start(aq_nic);
if (err < 0)
goto err_exit;
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
index e139276..8f66e78 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_nic.c
@@ -393,7 +393,7 @@ int aq_nic_start(struct aq_nic_s *self)
self->aq_nic_cfg.link_irq_vec);
err = request_threaded_irq(irqvec, NULL,
aq_linkstate_threaded_isr,
- IRQF_SHARED,
+ IRQF_SHARED | IRQF_ONESHOT,
self->ndev->name, self);
if (err < 0)
goto err_exit;
diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c
index 715685a..28892b8 100644
--- a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c
+++ b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c
@@ -86,6 +86,7 @@ static int aq_vec_poll(struct napi_struct *napi, int budget)
}
}
+err_exit:
if (!was_tx_cleaned)
work_done = budget;
@@ -95,7 +96,7 @@ static int aq_vec_poll(struct napi_struct *napi, int budget)
1U << self->aq_ring_param.vec_idx);
}
}
-err_exit:
+
return work_done;
}
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index cae66ba..7df887e 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -708,8 +708,7 @@ static int bcm_sysport_alloc_rx_bufs(struct bcm_sysport_priv *priv)
for (i = 0; i < priv->num_rx_bds; i++) {
cb = &priv->rx_cbs[i];
skb = bcm_sysport_rx_refill(priv, cb);
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
if (!cb->skb)
return -ENOMEM;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 4c790ff..402d9f5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -254,6 +254,8 @@ static const u16 bnxt_async_events_arr[] = {
ASYNC_EVENT_CMPL_EVENT_ID_PORT_CONN_NOT_ALLOWED,
ASYNC_EVENT_CMPL_EVENT_ID_VF_CFG_CHANGE,
ASYNC_EVENT_CMPL_EVENT_ID_LINK_SPEED_CFG_CHANGE,
+ ASYNC_EVENT_CMPL_EVENT_ID_RESET_NOTIFY,
+ ASYNC_EVENT_CMPL_EVENT_ID_ERROR_RECOVERY,
};
static struct workqueue_struct *bnxt_pf_wq;
@@ -1138,6 +1140,14 @@ static int bnxt_discard_rx(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
return 0;
}
+static void bnxt_queue_fw_reset_work(struct bnxt *bp, unsigned long delay)
+{
+ if (BNXT_PF(bp))
+ queue_delayed_work(bnxt_pf_wq, &bp->fw_reset_task, delay);
+ else
+ schedule_delayed_work(&bp->fw_reset_task, delay);
+}
+
static void bnxt_queue_sp_work(struct bnxt *bp)
{
if (BNXT_PF(bp))
@@ -1896,6 +1906,33 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
return bnxt_rx_pkt(bp, cpr, raw_cons, event);
}
+u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ u32 reg = fw_health->regs[reg_idx];
+ u32 reg_type, reg_off, val = 0;
+
+ reg_type = BNXT_FW_HEALTH_REG_TYPE(reg);
+ reg_off = BNXT_FW_HEALTH_REG_OFF(reg);
+ switch (reg_type) {
+ case BNXT_FW_HEALTH_REG_TYPE_CFG:
+ pci_read_config_dword(bp->pdev, reg_off, &val);
+ break;
+ case BNXT_FW_HEALTH_REG_TYPE_GRC:
+ reg_off = fw_health->mapped_regs[reg_idx];
+ /* fall through */
+ case BNXT_FW_HEALTH_REG_TYPE_BAR0:
+ val = readl(bp->bar0 + reg_off);
+ break;
+ case BNXT_FW_HEALTH_REG_TYPE_BAR1:
+ val = readl(bp->bar1 + reg_off);
+ break;
+ }
+ if (reg_idx == BNXT_FW_RESET_INPROG_REG)
+ val &= fw_health->fw_reset_inprog_reg_mask;
+ return val;
+}
+
#define BNXT_GET_EVENT_PORT(data) \
((data) & \
ASYNC_EVENT_CMPL_PORT_CONN_NOT_ALLOWED_EVENT_DATA1_PORT_ID_MASK)
@@ -1951,6 +1988,55 @@ static int bnxt_async_event_process(struct bnxt *bp,
goto async_event_process_exit;
set_bit(BNXT_RESET_TASK_SILENT_SP_EVENT, &bp->sp_event);
break;
+ case ASYNC_EVENT_CMPL_EVENT_ID_RESET_NOTIFY: {
+ u32 data1 = le32_to_cpu(cmpl->event_data1);
+
+ bp->fw_reset_timestamp = jiffies;
+ bp->fw_reset_min_dsecs = cmpl->timestamp_lo;
+ if (!bp->fw_reset_min_dsecs)
+ bp->fw_reset_min_dsecs = BNXT_DFLT_FW_RST_MIN_DSECS;
+ bp->fw_reset_max_dsecs = le16_to_cpu(cmpl->timestamp_hi);
+ if (!bp->fw_reset_max_dsecs)
+ bp->fw_reset_max_dsecs = BNXT_DFLT_FW_RST_MAX_DSECS;
+ if (EVENT_DATA1_RESET_NOTIFY_FATAL(data1)) {
+ netdev_warn(bp->dev, "Firmware fatal reset event received\n");
+ set_bit(BNXT_STATE_FW_FATAL_COND, &bp->state);
+ } else {
+ netdev_warn(bp->dev, "Firmware non-fatal reset event received, max wait time %d msec\n",
+ bp->fw_reset_max_dsecs * 100);
+ }
+ set_bit(BNXT_FW_RESET_NOTIFY_SP_EVENT, &bp->sp_event);
+ break;
+ }
+ case ASYNC_EVENT_CMPL_EVENT_ID_ERROR_RECOVERY: {
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ u32 data1 = le32_to_cpu(cmpl->event_data1);
+
+ if (!fw_health)
+ goto async_event_process_exit;
+
+ fw_health->enabled = EVENT_DATA1_RECOVERY_ENABLED(data1);
+ fw_health->master = EVENT_DATA1_RECOVERY_MASTER_FUNC(data1);
+ if (!fw_health->enabled)
+ break;
+
+ if (netif_msg_drv(bp))
+ netdev_info(bp->dev, "Error recovery info: error recovery[%d], master[%d], reset count[0x%x], health status: 0x%x\n",
+ fw_health->enabled, fw_health->master,
+ bnxt_fw_health_readl(bp,
+ BNXT_FW_RESET_CNT_REG),
+ bnxt_fw_health_readl(bp,
+ BNXT_FW_HEALTH_REG));
+ fw_health->tmr_multiplier =
+ DIV_ROUND_UP(fw_health->polling_dsecs * HZ,
+ bp->current_interval * 10);
+ fw_health->tmr_counter = fw_health->tmr_multiplier;
+ fw_health->last_fw_heartbeat =
+ bnxt_fw_health_readl(bp, BNXT_FW_HEARTBEAT_REG);
+ fw_health->last_fw_reset_cnt =
+ bnxt_fw_health_readl(bp, BNXT_FW_RESET_CNT_REG);
+ goto async_event_process_exit;
+ }
default:
goto async_event_process_exit;
}
@@ -3555,6 +3641,9 @@ static int bnxt_alloc_kong_hwrm_resources(struct bnxt *bp)
{
struct pci_dev *pdev = bp->pdev;
+ if (bp->hwrm_cmd_kong_resp_addr)
+ return 0;
+
bp->hwrm_cmd_kong_resp_addr =
dma_alloc_coherent(&pdev->dev, PAGE_SIZE,
&bp->hwrm_cmd_kong_resp_dma_addr,
@@ -3594,6 +3683,9 @@ static int bnxt_alloc_hwrm_short_cmd_req(struct bnxt *bp)
{
struct pci_dev *pdev = bp->pdev;
+ if (bp->hwrm_short_cmd_req_addr)
+ return 0;
+
bp->hwrm_short_cmd_req_addr =
dma_alloc_coherent(&pdev->dev, bp->hwrm_max_ext_req_len,
&bp->hwrm_short_cmd_req_dma_addr,
@@ -4048,6 +4140,32 @@ void bnxt_hwrm_cmd_hdr_init(struct bnxt *bp, void *request, u16 req_type,
req->resp_addr = cpu_to_le64(bp->hwrm_cmd_resp_dma_addr);
}
+static int bnxt_hwrm_to_stderr(u32 hwrm_err)
+{
+ switch (hwrm_err) {
+ case HWRM_ERR_CODE_SUCCESS:
+ return 0;
+ case HWRM_ERR_CODE_RESOURCE_ACCESS_DENIED:
+ return -EACCES;
+ case HWRM_ERR_CODE_RESOURCE_ALLOC_ERROR:
+ return -ENOSPC;
+ case HWRM_ERR_CODE_INVALID_PARAMS:
+ case HWRM_ERR_CODE_INVALID_FLAGS:
+ case HWRM_ERR_CODE_INVALID_ENABLES:
+ case HWRM_ERR_CODE_UNSUPPORTED_TLV:
+ case HWRM_ERR_CODE_UNSUPPORTED_OPTION_ERR:
+ return -EINVAL;
+ case HWRM_ERR_CODE_NO_BUFFER:
+ return -ENOMEM;
+ case HWRM_ERR_CODE_HOT_RESET_PROGRESS:
+ return -EAGAIN;
+ case HWRM_ERR_CODE_CMD_NOT_SUPPORTED:
+ return -EOPNOTSUPP;
+ default:
+ return -EIO;
+ }
+}
+
static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
int timeout, bool silent)
{
@@ -4065,6 +4183,9 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
u32 bar_offset = BNXT_GRCPF_REG_CHIMP_COMM;
u16 dst = BNXT_HWRM_CHNL_CHIMP;
+ if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state))
+ return -EBUSY;
+
if (msg_len > BNXT_HWRM_MAX_REQ_LEN) {
if (msg_len > bp->hwrm_max_ext_req_len ||
!bp->hwrm_short_cmd_req_addr)
@@ -4129,6 +4250,9 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
/* Ring channel doorbell */
writel(1, bp->bar0 + doorbell_offset);
+ if (!pci_is_enabled(bp->pdev))
+ return 0;
+
if (!timeout)
timeout = DFLT_HWRM_CMD_TIMEOUT;
/* convert timeout to usec */
@@ -4160,9 +4284,10 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
}
if (bp->hwrm_intr_seq_id != (u16)~seq_id) {
- netdev_err(bp->dev, "Resp cmpl intr err msg: 0x%x\n",
- le16_to_cpu(req->req_type));
- return -1;
+ if (!silent)
+ netdev_err(bp->dev, "Resp cmpl intr err msg: 0x%x\n",
+ le16_to_cpu(req->req_type));
+ return -EBUSY;
}
len = (le32_to_cpu(*resp_len) & HWRM_RESP_LEN_MASK) >>
HWRM_RESP_LEN_SFT;
@@ -4186,11 +4311,12 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
}
if (i >= tmo_count) {
- netdev_err(bp->dev, "Error (timeout: %d) msg {0x%x 0x%x} len:%d\n",
- HWRM_TOTAL_TIMEOUT(i),
- le16_to_cpu(req->req_type),
- le16_to_cpu(req->seq_id), len);
- return -1;
+ if (!silent)
+ netdev_err(bp->dev, "Error (timeout: %d) msg {0x%x 0x%x} len:%d\n",
+ HWRM_TOTAL_TIMEOUT(i),
+ le16_to_cpu(req->req_type),
+ le16_to_cpu(req->seq_id), len);
+ return -EBUSY;
}
/* Last byte of resp contains valid bit */
@@ -4204,11 +4330,13 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
}
if (j >= HWRM_VALID_BIT_DELAY_USEC) {
- netdev_err(bp->dev, "Error (timeout: %d) msg {0x%x 0x%x} len:%d v:%d\n",
- HWRM_TOTAL_TIMEOUT(i),
- le16_to_cpu(req->req_type),
- le16_to_cpu(req->seq_id), len, *valid);
- return -1;
+ if (!silent)
+ netdev_err(bp->dev, "Error (timeout: %d) msg {0x%x 0x%x} len:%d v:%d\n",
+ HWRM_TOTAL_TIMEOUT(i),
+ le16_to_cpu(req->req_type),
+ le16_to_cpu(req->seq_id), len,
+ *valid);
+ return -EBUSY;
}
}
@@ -4222,7 +4350,7 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len,
netdev_err(bp->dev, "hwrm req_type 0x%x seq id 0x%x error 0x%x\n",
le16_to_cpu(resp->req_type),
le16_to_cpu(resp->seq_id), rc);
- return rc;
+ return bnxt_hwrm_to_stderr(rc);
}
int _hwrm_send_message(struct bnxt *bp, void *msg, u32 msg_len, int timeout)
@@ -4271,9 +4399,14 @@ int bnxt_hwrm_func_rgtr_async_events(struct bnxt *bp, unsigned long *bmap,
cpu_to_le32(FUNC_DRV_RGTR_REQ_ENABLES_ASYNC_EVENT_FWD);
memset(async_events_bmap, 0, sizeof(async_events_bmap));
- for (i = 0; i < ARRAY_SIZE(bnxt_async_events_arr); i++)
- __set_bit(bnxt_async_events_arr[i], async_events_bmap);
+ for (i = 0; i < ARRAY_SIZE(bnxt_async_events_arr); i++) {
+ u16 event_id = bnxt_async_events_arr[i];
+ if (event_id == ASYNC_EVENT_CMPL_EVENT_ID_ERROR_RECOVERY &&
+ !(bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY))
+ continue;
+ __set_bit(bnxt_async_events_arr[i], async_events_bmap);
+ }
if (bmap && bmap_size) {
for (i = 0; i < bmap_size; i++) {
if (test_bit(i, bmap))
@@ -4291,6 +4424,7 @@ static int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp)
{
struct hwrm_func_drv_rgtr_output *resp = bp->hwrm_cmd_resp_addr;
struct hwrm_func_drv_rgtr_input req = {0};
+ u32 flags;
int rc;
bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_DRV_RGTR, -1, -1);
@@ -4300,7 +4434,11 @@ static int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp)
FUNC_DRV_RGTR_REQ_ENABLES_VER);
req.os_type = cpu_to_le16(FUNC_DRV_RGTR_REQ_OS_TYPE_LINUX);
- req.flags = cpu_to_le32(FUNC_DRV_RGTR_REQ_FLAGS_16BIT_VER_MODE);
+ flags = FUNC_DRV_RGTR_REQ_FLAGS_16BIT_VER_MODE |
+ FUNC_DRV_RGTR_REQ_FLAGS_HOT_RESET_SUPPORT;
+ if (bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY)
+ flags |= FUNC_DRV_RGTR_REQ_FLAGS_ERROR_RECOVERY_SUPPORT;
+ req.flags = cpu_to_le32(flags);
req.ver_maj_8b = DRV_VER_MAJ;
req.ver_min_8b = DRV_VER_MIN;
req.ver_upd_8b = DRV_VER_UPD;
@@ -4335,10 +4473,8 @@ static int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp)
mutex_lock(&bp->hwrm_cmd_lock);
rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
- else if (resp->flags &
- cpu_to_le32(FUNC_DRV_RGTR_RESP_FLAGS_IF_CHANGE_SUPPORTED))
+ if (!rc && (resp->flags &
+ cpu_to_le32(FUNC_DRV_RGTR_RESP_FLAGS_IF_CHANGE_SUPPORTED)))
bp->fw_cap |= BNXT_FW_CAP_IF_CHANGE;
mutex_unlock(&bp->hwrm_cmd_lock);
return rc;
@@ -4761,7 +4897,7 @@ static int bnxt_hwrm_vnic_set_rss_p5(struct bnxt *bp, u16 vnic_id, bool set_rss)
}
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
- return -EIO;
+ return rc;
}
return 0;
}
@@ -4924,8 +5060,6 @@ static int bnxt_hwrm_vnic_free_one(struct bnxt *bp, u16 vnic_id)
cpu_to_le32(bp->vnic_info[vnic_id].fw_vnic_id);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- return rc;
bp->vnic_info[vnic_id].fw_vnic_id = INVALID_HW_RING_ID;
}
return rc;
@@ -4985,6 +5119,8 @@ static int bnxt_hwrm_vnic_qcaps(struct bnxt *bp)
struct hwrm_vnic_qcaps_input req = {0};
int rc;
+ bp->hw_ring_stats_size = sizeof(struct ctx_hw_stats);
+ bp->flags &= ~(BNXT_FLAG_NEW_RSS_CAP | BNXT_FLAG_ROCE_MIRROR_CAP);
if (bp->hwrm_spec_code < 0x10600)
return 0;
@@ -5004,8 +5140,6 @@ static int bnxt_hwrm_vnic_qcaps(struct bnxt *bp)
if (bp->max_tpa_v2)
bp->hw_ring_stats_size =
sizeof(struct ctx_hw_stats_ext);
- else
- bp->hw_ring_stats_size = sizeof(struct ctx_hw_stats);
}
mutex_unlock(&bp->hwrm_cmd_lock);
return rc;
@@ -5065,8 +5199,6 @@ static int bnxt_hwrm_ring_grp_free(struct bnxt *bp)
rc = _hwrm_send_message(bp, &req, sizeof(req),
HWRM_CMD_TIMEOUT);
- if (rc)
- break;
bp->grp_info[i].fw_grp_id = INVALID_HW_RING_ID;
}
mutex_unlock(&bp->hwrm_cmd_lock);
@@ -5385,6 +5517,9 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,
struct hwrm_ring_free_output *resp = bp->hwrm_cmd_resp_addr;
u16 error_code;
+ if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state))
+ return 0;
+
bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_RING_FREE, cmpl_ring_id, -1);
req.ring_type = ring_type;
req.ring_id = cpu_to_le16(ring->fw_ring_id);
@@ -5522,7 +5657,7 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp)
rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc) {
mutex_unlock(&bp->hwrm_cmd_lock);
- return -EIO;
+ return rc;
}
hw_resc->resv_tx_rings = le16_to_cpu(resp->alloc_tx_rings);
@@ -5686,7 +5821,7 @@ bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
- return -ENOMEM;
+ return rc;
if (bp->hwrm_spec_code < 0x10601)
bp->hw_resc.resv_tx_rings = tx_rings;
@@ -5711,7 +5846,7 @@ bnxt_hwrm_reserve_vf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
cp_rings, stats, vnics);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
- return -ENOMEM;
+ return rc;
rc = bnxt_hwrm_get_rings(bp);
return rc;
@@ -5892,9 +6027,7 @@ static int bnxt_hwrm_check_vf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
req.flags = cpu_to_le32(flags);
rc = hwrm_send_message_silent(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- return -ENOMEM;
- return 0;
+ return rc;
}
static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
@@ -5922,9 +6055,7 @@ static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
req.flags = cpu_to_le32(flags);
rc = hwrm_send_message_silent(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- return -ENOMEM;
- return 0;
+ return rc;
}
static int bnxt_hwrm_check_rings(struct bnxt *bp, int tx_rings, int rx_rings,
@@ -6186,8 +6317,6 @@ static int bnxt_hwrm_stat_ctx_free(struct bnxt *bp)
rc = _hwrm_send_message(bp, &req, sizeof(req),
HWRM_CMD_TIMEOUT);
- if (rc)
- break;
cpr->hw_stats_ctx_id = INVALID_STATS_CTX_ID;
}
@@ -6249,6 +6378,8 @@ static int bnxt_hwrm_func_qcfg(struct bnxt *bp)
struct bnxt_vf_info *vf = &bp->vf;
vf->vlan = le16_to_cpu(resp->vlan) & VLAN_VID_MASK;
+ } else {
+ bp->pf.registered_vfs = le16_to_cpu(resp->registered_vfs);
}
#endif
flags = le16_to_cpu(resp->flags);
@@ -6484,8 +6615,6 @@ static int bnxt_hwrm_func_backing_store_cfg(struct bnxt *bp, u32 enables)
}
req.flags = cpu_to_le32(flags);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -6747,10 +6876,8 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)
mutex_lock(&bp->hwrm_cmd_lock);
rc = _hwrm_send_message_silent(bp, &req, sizeof(req),
HWRM_CMD_TIMEOUT);
- if (rc) {
- rc = -EIO;
+ if (rc)
goto hwrm_func_resc_qcaps_exit;
- }
hw_resc->max_tx_sch_inputs = le16_to_cpu(resp->max_tx_scheduler_inputs);
if (!all)
@@ -6818,6 +6945,8 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)
bp->fw_cap |= BNXT_FW_CAP_PCIE_STATS_SUPPORTED;
if (flags & FUNC_QCAPS_RESP_FLAGS_EXT_STATS_SUPPORTED)
bp->fw_cap |= BNXT_FW_CAP_EXT_STATS_SUPPORTED;
+ if (flags & FUNC_QCAPS_RESP_FLAGS_ERROR_RECOVERY_CAPABLE)
+ bp->fw_cap |= BNXT_FW_CAP_ERROR_RECOVERY;
bp->tx_push_thresh = 0;
if (flags & FUNC_QCAPS_RESP_FLAGS_PUSH_MODE_SUPPORTED)
@@ -6849,6 +6978,7 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)
pf->max_tx_wm_flows = le32_to_cpu(resp->max_tx_wm_flows);
pf->max_rx_em_flows = le32_to_cpu(resp->max_rx_em_flows);
pf->max_rx_wm_flows = le32_to_cpu(resp->max_rx_wm_flows);
+ bp->flags &= ~BNXT_FLAG_WOL_CAP;
if (flags & FUNC_QCAPS_RESP_FLAGS_WOL_MAGICPKT_SUPPORTED)
bp->flags |= BNXT_FLAG_WOL_CAP;
} else {
@@ -6918,6 +7048,103 @@ static int bnxt_hwrm_cfa_adv_flow_mgnt_qcaps(struct bnxt *bp)
return rc;
}
+static int bnxt_map_fw_health_regs(struct bnxt *bp)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ u32 reg_base = 0xffffffff;
+ int i;
+
+ /* Only pre-map the monitoring GRC registers using window 3 */
+ for (i = 0; i < 4; i++) {
+ u32 reg = fw_health->regs[i];
+
+ if (BNXT_FW_HEALTH_REG_TYPE(reg) != BNXT_FW_HEALTH_REG_TYPE_GRC)
+ continue;
+ if (reg_base == 0xffffffff)
+ reg_base = reg & BNXT_GRC_BASE_MASK;
+ if ((reg & BNXT_GRC_BASE_MASK) != reg_base)
+ return -ERANGE;
+ fw_health->mapped_regs[i] = BNXT_FW_HEALTH_WIN_BASE +
+ (reg & BNXT_GRC_OFFSET_MASK);
+ }
+ if (reg_base == 0xffffffff)
+ return 0;
+
+ writel(reg_base, bp->bar0 + BNXT_GRCPF_REG_WINDOW_BASE_OUT +
+ BNXT_FW_HEALTH_WIN_MAP_OFF);
+ return 0;
+}
+
+static int bnxt_hwrm_error_recovery_qcfg(struct bnxt *bp)
+{
+ struct hwrm_error_recovery_qcfg_output *resp = bp->hwrm_cmd_resp_addr;
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ struct hwrm_error_recovery_qcfg_input req = {0};
+ int rc, i;
+
+ if (!(bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY))
+ return 0;
+
+ bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_ERROR_RECOVERY_QCFG, -1, -1);
+ mutex_lock(&bp->hwrm_cmd_lock);
+ rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ if (rc)
+ goto err_recovery_out;
+ if (!fw_health) {
+ fw_health = kzalloc(sizeof(*fw_health), GFP_KERNEL);
+ bp->fw_health = fw_health;
+ if (!fw_health) {
+ rc = -ENOMEM;
+ goto err_recovery_out;
+ }
+ }
+ fw_health->flags = le32_to_cpu(resp->flags);
+ if ((fw_health->flags & ERROR_RECOVERY_QCFG_RESP_FLAGS_CO_CPU) &&
+ !(bp->fw_cap & BNXT_FW_CAP_KONG_MB_CHNL)) {
+ rc = -EINVAL;
+ goto err_recovery_out;
+ }
+ fw_health->polling_dsecs = le32_to_cpu(resp->driver_polling_freq);
+ fw_health->master_func_wait_dsecs =
+ le32_to_cpu(resp->master_func_wait_period);
+ fw_health->normal_func_wait_dsecs =
+ le32_to_cpu(resp->normal_func_wait_period);
+ fw_health->post_reset_wait_dsecs =
+ le32_to_cpu(resp->master_func_wait_period_after_reset);
+ fw_health->post_reset_max_wait_dsecs =
+ le32_to_cpu(resp->max_bailout_time_after_reset);
+ fw_health->regs[BNXT_FW_HEALTH_REG] =
+ le32_to_cpu(resp->fw_health_status_reg);
+ fw_health->regs[BNXT_FW_HEARTBEAT_REG] =
+ le32_to_cpu(resp->fw_heartbeat_reg);
+ fw_health->regs[BNXT_FW_RESET_CNT_REG] =
+ le32_to_cpu(resp->fw_reset_cnt_reg);
+ fw_health->regs[BNXT_FW_RESET_INPROG_REG] =
+ le32_to_cpu(resp->reset_inprogress_reg);
+ fw_health->fw_reset_inprog_reg_mask =
+ le32_to_cpu(resp->reset_inprogress_reg_mask);
+ fw_health->fw_reset_seq_cnt = resp->reg_array_cnt;
+ if (fw_health->fw_reset_seq_cnt >= 16) {
+ rc = -EINVAL;
+ goto err_recovery_out;
+ }
+ for (i = 0; i < fw_health->fw_reset_seq_cnt; i++) {
+ fw_health->fw_reset_seq_regs[i] =
+ le32_to_cpu(resp->reset_reg[i]);
+ fw_health->fw_reset_seq_vals[i] =
+ le32_to_cpu(resp->reset_reg_val[i]);
+ fw_health->fw_reset_seq_delay_msec[i] =
+ resp->delay_after_reset[i];
+ }
+err_recovery_out:
+ mutex_unlock(&bp->hwrm_cmd_lock);
+ if (!rc)
+ rc = bnxt_map_fw_health_regs(bp);
+ if (rc)
+ bp->fw_cap &= ~BNXT_FW_CAP_ERROR_RECOVERY;
+ return rc;
+}
+
static int bnxt_hwrm_func_reset(struct bnxt *bp)
{
struct hwrm_func_reset_input req = {0};
@@ -6977,20 +7204,30 @@ static int bnxt_hwrm_queue_qportcfg(struct bnxt *bp)
return rc;
}
-static int bnxt_hwrm_ver_get(struct bnxt *bp)
+static int __bnxt_hwrm_ver_get(struct bnxt *bp, bool silent)
{
- int rc;
struct hwrm_ver_get_input req = {0};
- struct hwrm_ver_get_output *resp = bp->hwrm_cmd_resp_addr;
- u32 dev_caps_cfg;
+ int rc;
- bp->hwrm_max_req_len = HWRM_MAX_REQ_LEN;
bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_VER_GET, -1, -1);
req.hwrm_intf_maj = HWRM_VERSION_MAJOR;
req.hwrm_intf_min = HWRM_VERSION_MINOR;
req.hwrm_intf_upd = HWRM_VERSION_UPDATE;
+
+ rc = bnxt_hwrm_do_send_msg(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT,
+ silent);
+ return rc;
+}
+
+static int bnxt_hwrm_ver_get(struct bnxt *bp)
+{
+ struct hwrm_ver_get_output *resp = bp->hwrm_cmd_resp_addr;
+ u32 dev_caps_cfg;
+ int rc;
+
+ bp->hwrm_max_req_len = HWRM_MAX_REQ_LEN;
mutex_lock(&bp->hwrm_cmd_lock);
- rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ rc = __bnxt_hwrm_ver_get(bp, false);
if (rc)
goto hwrm_ver_get_exit;
@@ -7193,6 +7430,8 @@ static int bnxt_set_tpa(struct bnxt *bp, bool set_tpa)
if (set_tpa)
tpa_flags = bp->flags & BNXT_FLAG_TPA;
+ else if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state))
+ return 0;
for (i = 0; i < bp->nr_vnics; i++) {
rc = bnxt_hwrm_vnic_set_tpa(bp, i, tpa_flags);
if (rc) {
@@ -7258,8 +7497,6 @@ static int bnxt_hwrm_set_br_mode(struct bnxt *bp, u16 br_mode)
else
return -EINVAL;
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -7279,8 +7516,6 @@ static int bnxt_hwrm_set_cache_line_size(struct bnxt *bp, int size)
req.options = FUNC_CFG_REQ_OPTIONS_CACHE_LINESIZE_SIZE_128;
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -8163,6 +8398,9 @@ static int bnxt_hwrm_phy_qcaps(struct bnxt *bp)
struct hwrm_port_phy_qcaps_output *resp = bp->hwrm_cmd_resp_addr;
struct bnxt_link_info *link_info = &bp->link_info;
+ bp->flags &= ~BNXT_FLAG_EEE_CAP;
+ if (bp->test_info)
+ bp->test_info->flags &= ~BNXT_TEST_FL_EXT_LPBK;
if (bp->hwrm_spec_code < 0x10201)
return 0;
@@ -8484,11 +8722,14 @@ static int bnxt_hwrm_shutdown_link(struct bnxt *bp)
return hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
}
+static int bnxt_fw_init_one(struct bnxt *bp);
+
static int bnxt_hwrm_if_change(struct bnxt *bp, bool up)
{
struct hwrm_func_drv_if_change_output *resp = bp->hwrm_cmd_resp_addr;
struct hwrm_func_drv_if_change_input req = {0};
- bool resc_reinit = false;
+ bool resc_reinit = false, fw_reset = false;
+ u32 flags = 0;
int rc;
if (!(bp->fw_cap & BNXT_FW_CAP_IF_CHANGE))
@@ -8499,26 +8740,57 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up)
req.flags = cpu_to_le32(FUNC_DRV_IF_CHANGE_REQ_FLAGS_UP);
mutex_lock(&bp->hwrm_cmd_lock);
rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (!rc && (resp->flags &
- cpu_to_le32(FUNC_DRV_IF_CHANGE_RESP_FLAGS_RESC_CHANGE)))
- resc_reinit = true;
+ if (!rc)
+ flags = le32_to_cpu(resp->flags);
mutex_unlock(&bp->hwrm_cmd_lock);
+ if (rc)
+ return rc;
- if (up && resc_reinit && BNXT_NEW_RM(bp)) {
- struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+ if (!up)
+ return 0;
- rc = bnxt_hwrm_func_resc_qcaps(bp, true);
- hw_resc->resv_cp_rings = 0;
- hw_resc->resv_stat_ctxs = 0;
- hw_resc->resv_irqs = 0;
- hw_resc->resv_tx_rings = 0;
- hw_resc->resv_rx_rings = 0;
- hw_resc->resv_hw_ring_grps = 0;
- hw_resc->resv_vnics = 0;
- bp->tx_nr_rings = 0;
- bp->rx_nr_rings = 0;
+ if (flags & FUNC_DRV_IF_CHANGE_RESP_FLAGS_RESC_CHANGE)
+ resc_reinit = true;
+ if (flags & FUNC_DRV_IF_CHANGE_RESP_FLAGS_HOT_FW_RESET_DONE)
+ fw_reset = true;
+
+ if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state) && !fw_reset) {
+ netdev_err(bp->dev, "RESET_DONE not set during FW reset.\n");
+ return -ENODEV;
}
- return rc;
+ if (resc_reinit || fw_reset) {
+ if (fw_reset) {
+ rc = bnxt_fw_init_one(bp);
+ if (rc) {
+ set_bit(BNXT_STATE_ABORT_ERR, &bp->state);
+ return rc;
+ }
+ bnxt_clear_int_mode(bp);
+ rc = bnxt_init_int_mode(bp);
+ if (rc) {
+ netdev_err(bp->dev, "init int mode failed\n");
+ return rc;
+ }
+ set_bit(BNXT_STATE_FW_RESET_DET, &bp->state);
+ }
+ if (BNXT_NEW_RM(bp)) {
+ struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+
+ rc = bnxt_hwrm_func_resc_qcaps(bp, true);
+ hw_resc->resv_cp_rings = 0;
+ hw_resc->resv_stat_ctxs = 0;
+ hw_resc->resv_irqs = 0;
+ hw_resc->resv_tx_rings = 0;
+ hw_resc->resv_rx_rings = 0;
+ hw_resc->resv_hw_ring_grps = 0;
+ hw_resc->resv_vnics = 0;
+ if (!fw_reset) {
+ bp->tx_nr_rings = 0;
+ bp->rx_nr_rings = 0;
+ }
+ }
+ }
+ return 0;
}
static int bnxt_hwrm_port_led_qcaps(struct bnxt *bp)
@@ -8528,6 +8800,7 @@ static int bnxt_hwrm_port_led_qcaps(struct bnxt *bp)
struct bnxt_pf_info *pf = &bp->pf;
int rc;
+ bp->num_leds = 0;
if (BNXT_VF(bp) || bp->hwrm_spec_code < 0x10601)
return 0;
@@ -8622,6 +8895,7 @@ static void bnxt_get_wol_settings(struct bnxt *bp)
{
u16 handle = 0;
+ bp->wol = 0;
if (!BNXT_PF(bp) || !(bp->flags & BNXT_FLAG_WOL_CAP))
return;
@@ -8668,6 +8942,9 @@ static void bnxt_hwmon_open(struct bnxt *bp)
{
struct pci_dev *pdev = bp->pdev;
+ if (bp->hwmon_dev)
+ return;
+
bp->hwmon_dev = hwmon_device_register_with_groups(&pdev->dev,
DRV_MODULE_NAME, bp,
bnxt_groups);
@@ -8933,12 +9210,28 @@ static int bnxt_open(struct net_device *dev)
struct bnxt *bp = netdev_priv(dev);
int rc;
- bnxt_hwrm_if_change(bp, true);
- rc = __bnxt_open_nic(bp, true, true);
- if (rc)
- bnxt_hwrm_if_change(bp, false);
+ if (test_bit(BNXT_STATE_ABORT_ERR, &bp->state)) {
+ netdev_err(bp->dev, "A previous firmware reset did not complete, aborting\n");
+ return -ENODEV;
+ }
- bnxt_hwmon_open(bp);
+ rc = bnxt_hwrm_if_change(bp, true);
+ if (rc)
+ return rc;
+ rc = __bnxt_open_nic(bp, true, true);
+ if (rc) {
+ bnxt_hwrm_if_change(bp, false);
+ } else {
+ if (test_and_clear_bit(BNXT_STATE_FW_RESET_DET, &bp->state) &&
+ BNXT_PF(bp)) {
+ struct bnxt_pf_info *pf = &bp->pf;
+ int n = pf->active_vfs;
+
+ if (n)
+ bnxt_cfg_hw_sriov(bp, &n, true);
+ }
+ bnxt_hwmon_open(bp);
+ }
return rc;
}
@@ -8975,6 +9268,10 @@ static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,
bnxt_debug_dev_exit(bp);
bnxt_disable_napi(bp);
del_timer_sync(&bp->timer);
+ if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state) &&
+ pci_is_enabled(bp->pdev))
+ pci_disable_device(bp->pdev);
+
bnxt_free_skbs(bp);
/* Save ring stats before shutdown */
@@ -8991,6 +9288,18 @@ int bnxt_close_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
{
int rc = 0;
+ if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) {
+ /* If we get here, it means firmware reset is in progress
+ * while we are trying to close. We can safely proceed with
+ * the close because we are holding rtnl_lock(). Some firmware
+ * messages may fail as we proceed to close. We set the
+ * ABORT_ERR flag here so that the FW reset thread will later
+ * abort when it gets the rtnl_lock() and sees the flag.
+ */
+ netdev_warn(bp->dev, "FW reset in progress during close, FW reset will be aborted\n");
+ set_bit(BNXT_STATE_ABORT_ERR, &bp->state);
+ }
+
#ifdef CONFIG_BNXT_SRIOV
if (bp->sriov_cfg) {
rc = wait_event_interruptible_timeout(bp->sriov_cfg_wait,
@@ -9632,6 +9941,38 @@ static void bnxt_tx_timeout(struct net_device *dev)
bnxt_queue_sp_work(bp);
}
+static void bnxt_fw_health_check(struct bnxt *bp)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ u32 val;
+
+ if (!fw_health || !fw_health->enabled ||
+ test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
+ return;
+
+ if (fw_health->tmr_counter) {
+ fw_health->tmr_counter--;
+ return;
+ }
+
+ val = bnxt_fw_health_readl(bp, BNXT_FW_HEARTBEAT_REG);
+ if (val == fw_health->last_fw_heartbeat)
+ goto fw_reset;
+
+ fw_health->last_fw_heartbeat = val;
+
+ val = bnxt_fw_health_readl(bp, BNXT_FW_RESET_CNT_REG);
+ if (val != fw_health->last_fw_reset_cnt)
+ goto fw_reset;
+
+ fw_health->tmr_counter = fw_health->tmr_multiplier;
+ return;
+
+fw_reset:
+ set_bit(BNXT_FW_EXCEPTION_SP_EVENT, &bp->sp_event);
+ bnxt_queue_sp_work(bp);
+}
+
static void bnxt_timer(struct timer_list *t)
{
struct bnxt *bp = from_timer(bp, t, timer);
@@ -9643,6 +9984,9 @@ static void bnxt_timer(struct timer_list *t)
if (atomic_read(&bp->intr_sem) != 0)
goto bnxt_restart_timer;
+ if (bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY)
+ bnxt_fw_health_check(bp);
+
if (bp->link_info.link_up && (bp->flags & BNXT_FLAG_PORT_STATS) &&
bp->stats_coal_ticks) {
set_bit(BNXT_PERIODIC_STATS_SP_EVENT, &bp->sp_event);
@@ -9698,6 +10042,130 @@ static void bnxt_reset(struct bnxt *bp, bool silent)
bnxt_rtnl_unlock_sp(bp);
}
+static void bnxt_fw_reset_close(struct bnxt *bp)
+{
+ __bnxt_close_nic(bp, true, false);
+ bnxt_ulp_irq_stop(bp);
+ bnxt_clear_int_mode(bp);
+ bnxt_hwrm_func_drv_unrgtr(bp);
+ bnxt_free_ctx_mem(bp);
+ kfree(bp->ctx);
+ bp->ctx = NULL;
+}
+
+static bool is_bnxt_fw_ok(struct bnxt *bp)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ bool no_heartbeat = false, has_reset = false;
+ u32 val;
+
+ val = bnxt_fw_health_readl(bp, BNXT_FW_HEARTBEAT_REG);
+ if (val == fw_health->last_fw_heartbeat)
+ no_heartbeat = true;
+
+ val = bnxt_fw_health_readl(bp, BNXT_FW_RESET_CNT_REG);
+ if (val != fw_health->last_fw_reset_cnt)
+ has_reset = true;
+
+ if (!no_heartbeat && has_reset)
+ return true;
+
+ return false;
+}
+
+/* rtnl_lock is acquired before calling this function */
+static void bnxt_force_fw_reset(struct bnxt *bp)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ u32 wait_dsecs;
+
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state) ||
+ test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
+ return;
+
+ set_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ bnxt_fw_reset_close(bp);
+ wait_dsecs = fw_health->master_func_wait_dsecs;
+ if (fw_health->master) {
+ if (fw_health->flags & ERROR_RECOVERY_QCFG_RESP_FLAGS_CO_CPU)
+ wait_dsecs = 0;
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_RESET_FW;
+ } else {
+ bp->fw_reset_timestamp = jiffies + wait_dsecs * HZ / 10;
+ wait_dsecs = fw_health->normal_func_wait_dsecs;
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_ENABLE_DEV;
+ }
+ bp->fw_reset_max_dsecs = fw_health->post_reset_max_wait_dsecs;
+ bnxt_queue_fw_reset_work(bp, wait_dsecs * HZ / 10);
+}
+
+void bnxt_fw_exception(struct bnxt *bp)
+{
+ set_bit(BNXT_STATE_FW_FATAL_COND, &bp->state);
+ bnxt_rtnl_lock_sp(bp);
+ bnxt_force_fw_reset(bp);
+ bnxt_rtnl_unlock_sp(bp);
+}
+
+/* Returns the number of registered VFs, or 1 if VF configuration is pending, or
+ * < 0 on error.
+ */
+static int bnxt_get_registered_vfs(struct bnxt *bp)
+{
+#ifdef CONFIG_BNXT_SRIOV
+ int rc;
+
+ if (!BNXT_PF(bp))
+ return 0;
+
+ rc = bnxt_hwrm_func_qcfg(bp);
+ if (rc) {
+ netdev_err(bp->dev, "func_qcfg cmd failed, rc = %d\n", rc);
+ return rc;
+ }
+ if (bp->pf.registered_vfs)
+ return bp->pf.registered_vfs;
+ if (bp->sriov_cfg)
+ return 1;
+#endif
+ return 0;
+}
+
+void bnxt_fw_reset(struct bnxt *bp)
+{
+ bnxt_rtnl_lock_sp(bp);
+ if (test_bit(BNXT_STATE_OPEN, &bp->state) &&
+ !test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) {
+ int n = 0;
+
+ set_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ if (bp->pf.active_vfs &&
+ !test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state))
+ n = bnxt_get_registered_vfs(bp);
+ if (n < 0) {
+ netdev_err(bp->dev, "Firmware reset aborted, rc = %d\n",
+ n);
+ clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ dev_close(bp->dev);
+ goto fw_reset_exit;
+ } else if (n > 0) {
+ u16 vf_tmo_dsecs = n * 10;
+
+ if (bp->fw_reset_max_dsecs < vf_tmo_dsecs)
+ bp->fw_reset_max_dsecs = vf_tmo_dsecs;
+ bp->fw_reset_state =
+ BNXT_FW_RESET_STATE_POLL_VF;
+ bnxt_queue_fw_reset_work(bp, HZ / 10);
+ goto fw_reset_exit;
+ }
+ bnxt_fw_reset_close(bp);
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_ENABLE_DEV;
+ bnxt_queue_fw_reset_work(bp, bp->fw_reset_min_dsecs * HZ / 10);
+ }
+fw_reset_exit:
+ bnxt_rtnl_unlock_sp(bp);
+}
+
static void bnxt_chk_missed_irq(struct bnxt *bp)
{
int i;
@@ -9828,6 +10296,15 @@ static void bnxt_sp_task(struct work_struct *work)
if (test_and_clear_bit(BNXT_RESET_TASK_SILENT_SP_EVENT, &bp->sp_event))
bnxt_reset(bp, true);
+ if (test_and_clear_bit(BNXT_FW_RESET_NOTIFY_SP_EVENT, &bp->sp_event))
+ bnxt_devlink_health_report(bp, BNXT_FW_RESET_NOTIFY_SP_EVENT);
+
+ if (test_and_clear_bit(BNXT_FW_EXCEPTION_SP_EVENT, &bp->sp_event)) {
+ if (!is_bnxt_fw_ok(bp))
+ bnxt_devlink_health_report(bp,
+ BNXT_FW_EXCEPTION_SP_EVENT);
+ }
+
smp_mb__before_atomic();
clear_bit(BNXT_STATE_IN_SP_TASK, &bp->state);
}
@@ -9968,6 +10445,11 @@ static int bnxt_fw_init_one_p2(struct bnxt *bp)
netdev_warn(bp->dev, "hwrm query adv flow mgnt failure rc: %d\n",
rc);
+ rc = bnxt_hwrm_error_recovery_qcfg(bp);
+ if (rc)
+ netdev_warn(bp->dev, "hwrm query error recovery failure rc: %d\n",
+ rc);
+
rc = bnxt_hwrm_func_drv_rgtr(bp);
if (rc)
return -ENODEV;
@@ -9984,6 +10466,241 @@ static int bnxt_fw_init_one_p2(struct bnxt *bp)
return 0;
}
+static void bnxt_set_dflt_rss_hash_type(struct bnxt *bp)
+{
+ bp->flags &= ~BNXT_FLAG_UDP_RSS_CAP;
+ bp->rss_hash_cfg = VNIC_RSS_CFG_REQ_HASH_TYPE_IPV4 |
+ VNIC_RSS_CFG_REQ_HASH_TYPE_TCP_IPV4 |
+ VNIC_RSS_CFG_REQ_HASH_TYPE_IPV6 |
+ VNIC_RSS_CFG_REQ_HASH_TYPE_TCP_IPV6;
+ if (BNXT_CHIP_P4(bp) && bp->hwrm_spec_code >= 0x10501) {
+ bp->flags |= BNXT_FLAG_UDP_RSS_CAP;
+ bp->rss_hash_cfg |= VNIC_RSS_CFG_REQ_HASH_TYPE_UDP_IPV4 |
+ VNIC_RSS_CFG_REQ_HASH_TYPE_UDP_IPV6;
+ }
+}
+
+static void bnxt_set_dflt_rfs(struct bnxt *bp)
+{
+ struct net_device *dev = bp->dev;
+
+ dev->hw_features &= ~NETIF_F_NTUPLE;
+ dev->features &= ~NETIF_F_NTUPLE;
+ bp->flags &= ~BNXT_FLAG_RFS;
+ if (bnxt_rfs_supported(bp)) {
+ dev->hw_features |= NETIF_F_NTUPLE;
+ if (bnxt_rfs_capable(bp)) {
+ bp->flags |= BNXT_FLAG_RFS;
+ dev->features |= NETIF_F_NTUPLE;
+ }
+ }
+}
+
+static void bnxt_fw_init_one_p3(struct bnxt *bp)
+{
+ struct pci_dev *pdev = bp->pdev;
+
+ bnxt_set_dflt_rss_hash_type(bp);
+ bnxt_set_dflt_rfs(bp);
+
+ bnxt_get_wol_settings(bp);
+ if (bp->flags & BNXT_FLAG_WOL_CAP)
+ device_set_wakeup_enable(&pdev->dev, bp->wol);
+ else
+ device_set_wakeup_capable(&pdev->dev, false);
+
+ bnxt_hwrm_set_cache_line_size(bp, cache_line_size());
+ bnxt_hwrm_coal_params_qcaps(bp);
+}
+
+static int bnxt_fw_init_one(struct bnxt *bp)
+{
+ int rc;
+
+ rc = bnxt_fw_init_one_p1(bp);
+ if (rc) {
+ netdev_err(bp->dev, "Firmware init phase 1 failed\n");
+ return rc;
+ }
+ rc = bnxt_fw_init_one_p2(bp);
+ if (rc) {
+ netdev_err(bp->dev, "Firmware init phase 2 failed\n");
+ return rc;
+ }
+ rc = bnxt_approve_mac(bp, bp->dev->dev_addr, false);
+ if (rc)
+ return rc;
+ bnxt_fw_init_one_p3(bp);
+ return 0;
+}
+
+static void bnxt_fw_reset_writel(struct bnxt *bp, int reg_idx)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ u32 reg = fw_health->fw_reset_seq_regs[reg_idx];
+ u32 val = fw_health->fw_reset_seq_vals[reg_idx];
+ u32 reg_type, reg_off, delay_msecs;
+
+ delay_msecs = fw_health->fw_reset_seq_delay_msec[reg_idx];
+ reg_type = BNXT_FW_HEALTH_REG_TYPE(reg);
+ reg_off = BNXT_FW_HEALTH_REG_OFF(reg);
+ switch (reg_type) {
+ case BNXT_FW_HEALTH_REG_TYPE_CFG:
+ pci_write_config_dword(bp->pdev, reg_off, val);
+ break;
+ case BNXT_FW_HEALTH_REG_TYPE_GRC:
+ writel(reg_off & BNXT_GRC_BASE_MASK,
+ bp->bar0 + BNXT_GRCPF_REG_WINDOW_BASE_OUT + 4);
+ reg_off = (reg_off & BNXT_GRC_OFFSET_MASK) + 0x2000;
+ /* fall through */
+ case BNXT_FW_HEALTH_REG_TYPE_BAR0:
+ writel(val, bp->bar0 + reg_off);
+ break;
+ case BNXT_FW_HEALTH_REG_TYPE_BAR1:
+ writel(val, bp->bar1 + reg_off);
+ break;
+ }
+ if (delay_msecs) {
+ pci_read_config_dword(bp->pdev, 0, &val);
+ msleep(delay_msecs);
+ }
+}
+
+static void bnxt_reset_all(struct bnxt *bp)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ int i;
+
+ if (fw_health->flags & ERROR_RECOVERY_QCFG_RESP_FLAGS_HOST) {
+ for (i = 0; i < fw_health->fw_reset_seq_cnt; i++)
+ bnxt_fw_reset_writel(bp, i);
+ } else if (fw_health->flags & ERROR_RECOVERY_QCFG_RESP_FLAGS_CO_CPU) {
+ struct hwrm_fw_reset_input req = {0};
+ int rc;
+
+ bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FW_RESET, -1, -1);
+ req.resp_addr = cpu_to_le64(bp->hwrm_cmd_kong_resp_dma_addr);
+ req.embedded_proc_type = FW_RESET_REQ_EMBEDDED_PROC_TYPE_CHIP;
+ req.selfrst_status = FW_RESET_REQ_SELFRST_STATUS_SELFRSTASAP;
+ req.flags = FW_RESET_REQ_FLAGS_RESET_GRACEFUL;
+ rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ if (rc)
+ netdev_warn(bp->dev, "Unable to reset FW rc=%d\n", rc);
+ }
+ bp->fw_reset_timestamp = jiffies;
+}
+
+static void bnxt_fw_reset_task(struct work_struct *work)
+{
+ struct bnxt *bp = container_of(work, struct bnxt, fw_reset_task.work);
+ int rc;
+
+ if (!test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) {
+ netdev_err(bp->dev, "bnxt_fw_reset_task() called when not in fw reset mode!\n");
+ return;
+ }
+
+ switch (bp->fw_reset_state) {
+ case BNXT_FW_RESET_STATE_POLL_VF: {
+ int n = bnxt_get_registered_vfs(bp);
+
+ if (n < 0) {
+ netdev_err(bp->dev, "Firmware reset aborted, subsequent func_qcfg cmd failed, rc = %d, %d msecs since reset timestamp\n",
+ n, jiffies_to_msecs(jiffies -
+ bp->fw_reset_timestamp));
+ goto fw_reset_abort;
+ } else if (n > 0) {
+ if (time_after(jiffies, bp->fw_reset_timestamp +
+ (bp->fw_reset_max_dsecs * HZ / 10))) {
+ clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ bp->fw_reset_state = 0;
+ netdev_err(bp->dev, "Firmware reset aborted, bnxt_get_registered_vfs() returns %d\n",
+ n);
+ return;
+ }
+ bnxt_queue_fw_reset_work(bp, HZ / 10);
+ return;
+ }
+ bp->fw_reset_timestamp = jiffies;
+ rtnl_lock();
+ bnxt_fw_reset_close(bp);
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_ENABLE_DEV;
+ rtnl_unlock();
+ bnxt_queue_fw_reset_work(bp, bp->fw_reset_min_dsecs * HZ / 10);
+ return;
+ }
+ case BNXT_FW_RESET_STATE_RESET_FW: {
+ u32 wait_dsecs = bp->fw_health->post_reset_wait_dsecs;
+
+ bnxt_reset_all(bp);
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_ENABLE_DEV;
+ bnxt_queue_fw_reset_work(bp, wait_dsecs * HZ / 10);
+ return;
+ }
+ case BNXT_FW_RESET_STATE_ENABLE_DEV:
+ if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state) &&
+ bp->fw_health) {
+ u32 val;
+
+ val = bnxt_fw_health_readl(bp,
+ BNXT_FW_RESET_INPROG_REG);
+ if (val)
+ netdev_warn(bp->dev, "FW reset inprog %x after min wait time.\n",
+ val);
+ }
+ clear_bit(BNXT_STATE_FW_FATAL_COND, &bp->state);
+ if (pci_enable_device(bp->pdev)) {
+ netdev_err(bp->dev, "Cannot re-enable PCI device\n");
+ goto fw_reset_abort;
+ }
+ pci_set_master(bp->pdev);
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_POLL_FW;
+ /* fall through */
+ case BNXT_FW_RESET_STATE_POLL_FW:
+ bp->hwrm_cmd_timeout = SHORT_HWRM_CMD_TIMEOUT;
+ rc = __bnxt_hwrm_ver_get(bp, true);
+ if (rc) {
+ if (time_after(jiffies, bp->fw_reset_timestamp +
+ (bp->fw_reset_max_dsecs * HZ / 10))) {
+ netdev_err(bp->dev, "Firmware reset aborted\n");
+ goto fw_reset_abort;
+ }
+ bnxt_queue_fw_reset_work(bp, HZ / 5);
+ return;
+ }
+ bp->hwrm_cmd_timeout = DFLT_HWRM_CMD_TIMEOUT;
+ bp->fw_reset_state = BNXT_FW_RESET_STATE_OPENING;
+ /* fall through */
+ case BNXT_FW_RESET_STATE_OPENING:
+ while (!rtnl_trylock()) {
+ bnxt_queue_fw_reset_work(bp, HZ / 10);
+ return;
+ }
+ rc = bnxt_open(bp->dev);
+ if (rc) {
+ netdev_err(bp->dev, "bnxt_open_nic() failed\n");
+ clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ dev_close(bp->dev);
+ }
+ bnxt_ulp_irq_restart(bp, rc);
+ rtnl_unlock();
+
+ bp->fw_reset_state = 0;
+ /* Make sure fw_reset_state is 0 before clearing the flag */
+ smp_mb__before_atomic();
+ clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ break;
+ }
+ return;
+
+fw_reset_abort:
+ clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+ bp->fw_reset_state = 0;
+ rtnl_lock();
+ dev_close(bp->dev);
+ rtnl_unlock();
+}
+
static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev)
{
int rc;
@@ -10046,6 +10763,7 @@ static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev)
pci_enable_pcie_error_reporting(pdev);
INIT_WORK(&bp->sp_task, bnxt_sp_task);
+ INIT_DELAYED_WORK(&bp->fw_reset_task, bnxt_fw_reset_task);
spin_lock_init(&bp->ntp_fltr_lock);
#if BITS_PER_LONG == 32
@@ -10589,7 +11307,7 @@ static void bnxt_remove_one(struct pci_dev *pdev)
free_netdev(dev);
}
-static int bnxt_probe_phy(struct bnxt *bp)
+static int bnxt_probe_phy(struct bnxt *bp, bool fw_dflt)
{
int rc = 0;
struct bnxt_link_info *link_info = &bp->link_info;
@@ -10600,8 +11318,6 @@ static int bnxt_probe_phy(struct bnxt *bp)
rc);
return rc;
}
- mutex_init(&bp->link_lock);
-
rc = bnxt_update_link(bp, false);
if (rc) {
netdev_err(bp->dev, "Probe phy can't update link (rc: %x)\n",
@@ -10615,6 +11331,9 @@ static int bnxt_probe_phy(struct bnxt *bp)
if (link_info->auto_link_speeds && !link_info->support_auto_speeds)
link_info->support_auto_speeds = link_info->support_speeds;
+ if (!fw_dflt)
+ return 0;
+
/*initialize the ethool setting copy with NVM settings */
if (BNXT_AUTO_MODE(link_info->auto_mode)) {
link_info->autoneg = BNXT_AUTONEG_SPEED;
@@ -10635,7 +11354,7 @@ static int bnxt_probe_phy(struct bnxt *bp)
link_info->auto_pause_setting & BNXT_LINK_PAUSE_BOTH;
else
link_info->req_flow_ctrl = link_info->force_pause_setting;
- return rc;
+ return 0;
}
static int bnxt_get_max_irq(struct pci_dev *pdev)
@@ -10939,6 +11658,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
goto init_err_pci_clean;
mutex_init(&bp->hwrm_cmd_lock);
+ mutex_init(&bp->link_lock);
rc = bnxt_fw_init_one_p1(bp);
if (rc)
@@ -11014,7 +11734,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
dev->min_mtu = ETH_ZLEN;
dev->max_mtu = bp->max_mtu;
- rc = bnxt_probe_phy(bp);
+ rc = bnxt_probe_phy(bp, true);
if (rc)
goto init_err_pci_clean;
@@ -11028,24 +11748,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
goto init_err_pci_clean;
}
- /* Default RSS hash cfg. */
- bp->rss_hash_cfg = VNIC_RSS_CFG_REQ_HASH_TYPE_IPV4 |
- VNIC_RSS_CFG_REQ_HASH_TYPE_TCP_IPV4 |
- VNIC_RSS_CFG_REQ_HASH_TYPE_IPV6 |
- VNIC_RSS_CFG_REQ_HASH_TYPE_TCP_IPV6;
- if (BNXT_CHIP_P4(bp) && bp->hwrm_spec_code >= 0x10501) {
- bp->flags |= BNXT_FLAG_UDP_RSS_CAP;
- bp->rss_hash_cfg |= VNIC_RSS_CFG_REQ_HASH_TYPE_UDP_IPV4 |
- VNIC_RSS_CFG_REQ_HASH_TYPE_UDP_IPV6;
- }
-
- if (bnxt_rfs_supported(bp)) {
- dev->hw_features |= NETIF_F_NTUPLE;
- if (bnxt_rfs_capable(bp)) {
- bp->flags |= BNXT_FLAG_RFS;
- dev->features |= NETIF_F_NTUPLE;
- }
- }
+ bnxt_fw_init_one_p3(bp);
if (dev->hw_features & NETIF_F_HW_VLAN_CTAG_RX)
bp->flags |= BNXT_FLAG_STRIP_VLAN;
@@ -11059,16 +11762,6 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
*/
bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
- bnxt_get_wol_settings(bp);
- if (bp->flags & BNXT_FLAG_WOL_CAP)
- device_set_wakeup_enable(&pdev->dev, bp->wol);
- else
- device_set_wakeup_capable(&pdev->dev, false);
-
- bnxt_hwrm_set_cache_line_size(bp, cache_line_size());
-
- bnxt_hwrm_coal_params_qcaps(bp);
-
if (BNXT_PF(bp)) {
if (!bnxt_pf_wq) {
bnxt_pf_wq =
@@ -11105,6 +11798,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
bnxt_free_ctx_mem(bp);
kfree(bp->ctx);
bp->ctx = NULL;
+ kfree(bp->fw_health);
+ bp->fw_health = NULL;
bnxt_cleanup_pci(bp);
init_err_free:
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 1b1610d..333b0a8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -472,6 +472,19 @@ struct rx_tpa_end_cmp_ext {
((le32_to_cpu((rx_tpa_end_ext)->rx_tpa_end_cmp_dup_acks) & \
RX_TPA_END_CMP_AGG_BUFS_P5) >> RX_TPA_END_CMP_AGG_BUFS_SHIFT_P5)
+#define EVENT_DATA1_RESET_NOTIFY_FATAL(data1) \
+ (((data1) & \
+ ASYNC_EVENT_CMPL_RESET_NOTIFY_EVENT_DATA1_REASON_CODE_MASK) ==\
+ ASYNC_EVENT_CMPL_RESET_NOTIFY_EVENT_DATA1_REASON_CODE_FW_EXCEPTION_FATAL)
+
+#define EVENT_DATA1_RECOVERY_MASTER_FUNC(data1) \
+ !!((data1) & \
+ ASYNC_EVENT_CMPL_ERROR_RECOVERY_EVENT_DATA1_FLAGS_MASTER_FUNC)
+
+#define EVENT_DATA1_RECOVERY_ENABLED(data1) \
+ !!((data1) & \
+ ASYNC_EVENT_CMPL_ERROR_RECOVERY_EVENT_DATA1_FLAGS_RECOVERY_ENABLED)
+
struct nqe_cn {
__le16 type;
#define NQ_CN_TYPE_MASK 0x3fUL
@@ -632,6 +645,7 @@ struct nqe_cn {
#define BNXT_HWRM_MAX_REQ_LEN (bp->hwrm_max_req_len)
#define BNXT_HWRM_SHORT_REQ_LEN sizeof(struct hwrm_short_input)
#define DFLT_HWRM_CMD_TIMEOUT 500
+#define SHORT_HWRM_CMD_TIMEOUT 20
#define HWRM_CMD_TIMEOUT (bp->hwrm_cmd_timeout)
#define HWRM_RESET_TIMEOUT ((HWRM_CMD_TIMEOUT) * 4)
#define HWRM_RESP_ERR_CODE_MASK 0xffff
@@ -1058,6 +1072,7 @@ struct bnxt_pf_info {
u8 mac_addr[ETH_ALEN];
u32 first_vf_id;
u16 active_vfs;
+ u16 registered_vfs;
u16 max_vfs;
u32 max_encap_records;
u32 max_decap_records;
@@ -1217,6 +1232,9 @@ struct bnxt_test_info {
#define BNXT_GRCPF_REG_KONG_COMM 0xA00
#define BNXT_GRCPF_REG_KONG_COMM_TRIGGER 0xB00
+#define BNXT_GRC_BASE_MASK 0xfffff000
+#define BNXT_GRC_OFFSET_MASK 0x00000ffc
+
struct bnxt_tc_flow_stats {
u64 packets;
u64 bytes;
@@ -1333,6 +1351,53 @@ struct bnxt_ctx_mem_info {
struct bnxt_ctx_pg_info *tqm_mem[9];
};
+struct bnxt_fw_health {
+ u32 flags;
+ u32 polling_dsecs;
+ u32 master_func_wait_dsecs;
+ u32 normal_func_wait_dsecs;
+ u32 post_reset_wait_dsecs;
+ u32 post_reset_max_wait_dsecs;
+ u32 regs[4];
+ u32 mapped_regs[4];
+#define BNXT_FW_HEALTH_REG 0
+#define BNXT_FW_HEARTBEAT_REG 1
+#define BNXT_FW_RESET_CNT_REG 2
+#define BNXT_FW_RESET_INPROG_REG 3
+ u32 fw_reset_inprog_reg_mask;
+ u32 last_fw_heartbeat;
+ u32 last_fw_reset_cnt;
+ u8 enabled:1;
+ u8 master:1;
+ u8 tmr_multiplier;
+ u8 tmr_counter;
+ u8 fw_reset_seq_cnt;
+ u32 fw_reset_seq_regs[16];
+ u32 fw_reset_seq_vals[16];
+ u32 fw_reset_seq_delay_msec[16];
+ struct devlink_health_reporter *fw_reporter;
+ struct devlink_health_reporter *fw_reset_reporter;
+ struct devlink_health_reporter *fw_fatal_reporter;
+};
+
+struct bnxt_fw_reporter_ctx {
+ unsigned long sp_event;
+};
+
+#define BNXT_FW_HEALTH_REG_TYPE_MASK 3
+#define BNXT_FW_HEALTH_REG_TYPE_CFG 0
+#define BNXT_FW_HEALTH_REG_TYPE_GRC 1
+#define BNXT_FW_HEALTH_REG_TYPE_BAR0 2
+#define BNXT_FW_HEALTH_REG_TYPE_BAR1 3
+
+#define BNXT_FW_HEALTH_REG_TYPE(reg) ((reg) & BNXT_FW_HEALTH_REG_TYPE_MASK)
+#define BNXT_FW_HEALTH_REG_OFF(reg) ((reg) & ~BNXT_FW_HEALTH_REG_TYPE_MASK)
+
+#define BNXT_FW_HEALTH_WIN_BASE 0x3000
+#define BNXT_FW_HEALTH_WIN_MAP_OFF 8
+
+#define BNXT_FW_STATUS_HEALTHY 0x8000
+
struct bnxt {
void __iomem *bar0;
void __iomem *bar1;
@@ -1555,6 +1620,10 @@ struct bnxt {
#define BNXT_STATE_OPEN 0
#define BNXT_STATE_IN_SP_TASK 1
#define BNXT_STATE_READ_STATS 2
+#define BNXT_STATE_FW_RESET_DET 3
+#define BNXT_STATE_IN_FW_RESET 4
+#define BNXT_STATE_ABORT_ERR 5
+#define BNXT_STATE_FW_FATAL_COND 6
struct bnxt_irq *irq_tbl;
int total_irqs;
@@ -1579,6 +1648,7 @@ struct bnxt {
#define BNXT_FW_CAP_KONG_MB_CHNL 0x00000080
#define BNXT_FW_CAP_OVS_64BIT_HANDLE 0x00000400
#define BNXT_FW_CAP_TRUSTED_VF 0x00000800
+ #define BNXT_FW_CAP_ERROR_RECOVERY 0x00002000
#define BNXT_FW_CAP_PKG_VER 0x00004000
#define BNXT_FW_CAP_CFA_ADV_FLOW 0x00008000
#define BNXT_FW_CAP_CFA_RFS_RING_TBL_IDX 0x00010000
@@ -1663,6 +1733,24 @@ struct bnxt {
#define BNXT_FLOW_STATS_SP_EVENT 15
#define BNXT_UPDATE_PHY_SP_EVENT 16
#define BNXT_RING_COAL_NOW_SP_EVENT 17
+#define BNXT_FW_RESET_NOTIFY_SP_EVENT 18
+#define BNXT_FW_EXCEPTION_SP_EVENT 19
+
+ struct delayed_work fw_reset_task;
+ int fw_reset_state;
+#define BNXT_FW_RESET_STATE_POLL_VF 1
+#define BNXT_FW_RESET_STATE_RESET_FW 2
+#define BNXT_FW_RESET_STATE_ENABLE_DEV 3
+#define BNXT_FW_RESET_STATE_POLL_FW 4
+#define BNXT_FW_RESET_STATE_OPENING 5
+
+ u16 fw_reset_min_dsecs;
+#define BNXT_DFLT_FW_RST_MIN_DSECS 20
+ u16 fw_reset_max_dsecs;
+#define BNXT_DFLT_FW_RST_MAX_DSECS 60
+ unsigned long fw_reset_timestamp;
+
+ struct bnxt_fw_health *fw_health;
struct bnxt_hw_resc hw_resc;
struct bnxt_pf_info pf;
@@ -1868,6 +1956,7 @@ extern const u16 bnxt_lhint_arr[];
int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
u16 prod, gfp_t gfp);
void bnxt_reuse_rx_data(struct bnxt_rx_ring_info *rxr, u16 cons, void *data);
+u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx);
void bnxt_set_tpa_flags(struct bnxt *bp);
void bnxt_set_ring_params(struct bnxt *);
int bnxt_set_rx_skb_mode(struct bnxt *bp, bool page_mode);
@@ -1900,6 +1989,8 @@ int bnxt_open_nic(struct bnxt *, bool, bool);
int bnxt_half_open_nic(struct bnxt *bp);
void bnxt_half_close_nic(struct bnxt *bp);
int bnxt_close_nic(struct bnxt *, bool, bool);
+void bnxt_fw_exception(struct bnxt *bp);
+void bnxt_fw_reset(struct bnxt *bp);
int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
int tx_xdp);
int bnxt_setup_mq_tc(struct net_device *dev, u8 tc);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c
index 07301cb..fb6f30d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c
@@ -377,8 +377,6 @@ static int bnxt_hwrm_set_dcbx_app(struct bnxt *bp, struct dcb_app *app,
set.data_len = cpu_to_le16(sizeof(*data) + sizeof(*fw_app) * n);
set.hdr_cnt = 1;
rc = hwrm_send_message(bp, &set, sizeof(set), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
set_app_exit:
dma_free_coherent(&bp->pdev->dev, data_len, data, mapping);
@@ -391,6 +389,7 @@ static int bnxt_hwrm_queue_dscp_qcaps(struct bnxt *bp)
struct hwrm_queue_dscp_qcaps_input req = {0};
int rc;
+ bp->max_dscp_value = 0;
if (bp->hwrm_spec_code < 0x10800 || BNXT_VF(bp))
return 0;
@@ -433,8 +432,6 @@ static int bnxt_hwrm_queue_dscp2pri_cfg(struct bnxt *bp, struct dcb_app *app,
dscp2pri->pri = app->priority;
req.entry_cnt = cpu_to_le16(1);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
dma_free_coherent(&bp->pdev->dev, sizeof(*dscp2pri), dscp2pri,
mapping);
return rc;
@@ -722,6 +719,7 @@ static const struct dcbnl_rtnl_ops dcbnl_ops = {
void bnxt_dcb_init(struct bnxt *bp)
{
+ bp->dcbx_cap = 0;
if (bp->hwrm_spec_code < 0x10501)
return;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
index c05d663..e664392 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c
@@ -15,6 +15,192 @@
#include "bnxt_vfr.h"
#include "bnxt_devlink.h"
+static int bnxt_fw_reporter_diagnose(struct devlink_health_reporter *reporter,
+ struct devlink_fmsg *fmsg)
+{
+ struct bnxt *bp = devlink_health_reporter_priv(reporter);
+ struct bnxt_fw_health *health = bp->fw_health;
+ u32 val, health_status;
+ int rc;
+
+ if (!health || test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
+ return 0;
+
+ val = bnxt_fw_health_readl(bp, BNXT_FW_HEALTH_REG);
+ health_status = val & 0xffff;
+
+ if (health_status == BNXT_FW_STATUS_HEALTHY) {
+ rc = devlink_fmsg_string_pair_put(fmsg, "FW status",
+ "Healthy;");
+ if (rc)
+ return rc;
+ } else if (health_status < BNXT_FW_STATUS_HEALTHY) {
+ rc = devlink_fmsg_string_pair_put(fmsg, "FW status",
+ "Not yet completed initialization;");
+ if (rc)
+ return rc;
+ } else if (health_status > BNXT_FW_STATUS_HEALTHY) {
+ rc = devlink_fmsg_string_pair_put(fmsg, "FW status",
+ "Encountered fatal error and cannot recover;");
+ if (rc)
+ return rc;
+ }
+
+ if (val >> 16) {
+ rc = devlink_fmsg_u32_pair_put(fmsg, "Error", val >> 16);
+ if (rc)
+ return rc;
+ }
+
+ val = bnxt_fw_health_readl(bp, BNXT_FW_RESET_CNT_REG);
+ rc = devlink_fmsg_u32_pair_put(fmsg, "Reset count", val);
+ if (rc)
+ return rc;
+
+ return 0;
+}
+
+static const struct devlink_health_reporter_ops bnxt_dl_fw_reporter_ops = {
+ .name = "fw",
+ .diagnose = bnxt_fw_reporter_diagnose,
+};
+
+static int bnxt_fw_reset_recover(struct devlink_health_reporter *reporter,
+ void *priv_ctx)
+{
+ struct bnxt *bp = devlink_health_reporter_priv(reporter);
+
+ if (!priv_ctx)
+ return -EOPNOTSUPP;
+
+ bnxt_fw_reset(bp);
+ return 0;
+}
+
+static const
+struct devlink_health_reporter_ops bnxt_dl_fw_reset_reporter_ops = {
+ .name = "fw_reset",
+ .recover = bnxt_fw_reset_recover,
+};
+
+static int bnxt_fw_fatal_recover(struct devlink_health_reporter *reporter,
+ void *priv_ctx)
+{
+ struct bnxt *bp = devlink_health_reporter_priv(reporter);
+ struct bnxt_fw_reporter_ctx *fw_reporter_ctx = priv_ctx;
+ unsigned long event;
+
+ if (!priv_ctx)
+ return -EOPNOTSUPP;
+
+ event = fw_reporter_ctx->sp_event;
+ if (event == BNXT_FW_RESET_NOTIFY_SP_EVENT)
+ bnxt_fw_reset(bp);
+ else if (event == BNXT_FW_EXCEPTION_SP_EVENT)
+ bnxt_fw_exception(bp);
+
+ return 0;
+}
+
+static const
+struct devlink_health_reporter_ops bnxt_dl_fw_fatal_reporter_ops = {
+ .name = "fw_fatal",
+ .recover = bnxt_fw_fatal_recover,
+};
+
+static void bnxt_dl_fw_reporters_create(struct bnxt *bp)
+{
+ struct bnxt_fw_health *health = bp->fw_health;
+
+ if (!health)
+ return;
+
+ health->fw_reporter =
+ devlink_health_reporter_create(bp->dl, &bnxt_dl_fw_reporter_ops,
+ 0, false, bp);
+ if (IS_ERR(health->fw_reporter)) {
+ netdev_warn(bp->dev, "Failed to create FW health reporter, rc = %ld\n",
+ PTR_ERR(health->fw_reporter));
+ health->fw_reporter = NULL;
+ }
+
+ health->fw_reset_reporter =
+ devlink_health_reporter_create(bp->dl,
+ &bnxt_dl_fw_reset_reporter_ops,
+ 0, true, bp);
+ if (IS_ERR(health->fw_reset_reporter)) {
+ netdev_warn(bp->dev, "Failed to create FW fatal health reporter, rc = %ld\n",
+ PTR_ERR(health->fw_reset_reporter));
+ health->fw_reset_reporter = NULL;
+ }
+
+ health->fw_fatal_reporter =
+ devlink_health_reporter_create(bp->dl,
+ &bnxt_dl_fw_fatal_reporter_ops,
+ 0, true, bp);
+ if (IS_ERR(health->fw_fatal_reporter)) {
+ netdev_warn(bp->dev, "Failed to create FW fatal health reporter, rc = %ld\n",
+ PTR_ERR(health->fw_fatal_reporter));
+ health->fw_fatal_reporter = NULL;
+ }
+}
+
+static void bnxt_dl_fw_reporters_destroy(struct bnxt *bp)
+{
+ struct bnxt_fw_health *health = bp->fw_health;
+
+ if (!health)
+ return;
+
+ if (health->fw_reporter)
+ devlink_health_reporter_destroy(health->fw_reporter);
+
+ if (health->fw_reset_reporter)
+ devlink_health_reporter_destroy(health->fw_reset_reporter);
+
+ if (health->fw_fatal_reporter)
+ devlink_health_reporter_destroy(health->fw_fatal_reporter);
+}
+
+void bnxt_devlink_health_report(struct bnxt *bp, unsigned long event)
+{
+ struct bnxt_fw_health *fw_health = bp->fw_health;
+ struct bnxt_fw_reporter_ctx fw_reporter_ctx;
+
+ if (!fw_health)
+ return;
+
+ fw_reporter_ctx.sp_event = event;
+ switch (event) {
+ case BNXT_FW_RESET_NOTIFY_SP_EVENT:
+ if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state)) {
+ if (!fw_health->fw_fatal_reporter)
+ return;
+
+ devlink_health_report(fw_health->fw_fatal_reporter,
+ "FW fatal async event received",
+ &fw_reporter_ctx);
+ return;
+ }
+ if (!fw_health->fw_reset_reporter)
+ return;
+
+ devlink_health_report(fw_health->fw_reset_reporter,
+ "FW non-fatal reset event received",
+ &fw_reporter_ctx);
+ return;
+
+ case BNXT_FW_EXCEPTION_SP_EVENT:
+ if (!fw_health->fw_fatal_reporter)
+ return;
+
+ devlink_health_report(fw_health->fw_fatal_reporter,
+ "FW fatal error reported",
+ &fw_reporter_ctx);
+ return;
+ }
+}
+
static const struct devlink_ops bnxt_dl_ops = {
#ifdef CONFIG_BNXT_SRIOV
.eswitch_mode_set = bnxt_dl_eswitch_mode_set,
@@ -109,13 +295,9 @@ static int bnxt_hwrm_nvm_req(struct bnxt *bp, u32 param_id, void *msg,
memcpy(buf, data_addr, bytesize);
dma_free_coherent(&bp->pdev->dev, bytesize, data_addr, data_dma_addr);
- if (rc == HWRM_ERR_CODE_RESOURCE_ACCESS_DENIED) {
+ if (rc == -EACCES)
netdev_err(bp->dev, "PF does not have admin privileges to modify NVM config\n");
- return -EACCES;
- } else if (rc) {
- return -EIO;
- }
- return 0;
+ return rc;
}
static int bnxt_dl_nvm_param_get(struct devlink *dl, u32 id,
@@ -251,6 +433,8 @@ int bnxt_dl_register(struct bnxt *bp)
devlink_params_publish(dl);
+ bnxt_dl_fw_reporters_create(bp);
+
return 0;
err_dl_port_unreg:
@@ -273,6 +457,7 @@ void bnxt_dl_unregister(struct bnxt *bp)
if (!dl)
return;
+ bnxt_dl_fw_reporters_destroy(bp);
devlink_port_params_unregister(&bp->dl_port, bnxt_dl_port_params,
ARRAY_SIZE(bnxt_dl_port_params));
devlink_port_unregister(&bp->dl_port);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.h
index 5b6b2c7..b97e0ba 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.h
@@ -55,6 +55,7 @@ struct bnxt_dl_nvm_param {
u16 num_bits;
};
+void bnxt_devlink_health_report(struct bnxt *bp, unsigned long event);
int bnxt_dl_register(struct bnxt *bp);
void bnxt_dl_unregister(struct bnxt *bp);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index b624174..235265e 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -1699,6 +1699,11 @@ static u32 bnxt_get_link(struct net_device *dev)
return bp->link_info.link_up;
}
+static void bnxt_print_admin_err(struct bnxt *bp)
+{
+ netdev_info(bp->dev, "PF does not have admin privileges to flash or reset the device\n");
+}
+
static int bnxt_find_nvram_item(struct net_device *dev, u16 type, u16 ordinal,
u16 ext, u16 *index, u32 *item_length,
u32 *data_length);
@@ -1738,13 +1743,8 @@ static int bnxt_flash_nvram(struct net_device *dev,
rc = hwrm_send_message(bp, &req, sizeof(req), FLASH_NVRAM_TIMEOUT);
dma_free_coherent(&bp->pdev->dev, data_len, kmem, dma_handle);
- if (rc == HWRM_ERR_CODE_RESOURCE_ACCESS_DENIED) {
- netdev_info(dev,
- "PF does not have admin privileges to flash the device\n");
- rc = -EACCES;
- } else if (rc) {
- rc = -EIO;
- }
+ if (rc == -EACCES)
+ bnxt_print_admin_err(bp);
return rc;
}
@@ -1794,13 +1794,8 @@ static int bnxt_firmware_reset(struct net_device *dev,
}
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc == HWRM_ERR_CODE_RESOURCE_ACCESS_DENIED) {
- netdev_info(dev,
- "PF does not have admin privileges to reset the device\n");
- rc = -EACCES;
- } else if (rc) {
- rc = -EIO;
- }
+ if (rc == -EACCES)
+ bnxt_print_admin_err(bp);
return rc;
}
@@ -2095,13 +2090,8 @@ static int bnxt_flash_package_from_file(struct net_device *dev,
flash_pkg_exit:
mutex_unlock(&bp->hwrm_cmd_lock);
err_exit:
- if (hwrm_err == HWRM_ERR_CODE_RESOURCE_ACCESS_DENIED) {
- netdev_info(dev,
- "PF does not have admin privileges to flash the device\n");
- rc = -EACCES;
- } else if (hwrm_err) {
- rc = -EOPNOTSUPP;
- }
+ if (hwrm_err == -EACCES)
+ bnxt_print_admin_err(bp);
return rc;
}
@@ -2640,8 +2630,6 @@ static int bnxt_set_phys_id(struct net_device *dev,
led_cfg->led_group_id = bp->leds[i].led_group_id;
}
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -3362,6 +3350,7 @@ void bnxt_ethtool_init(struct bnxt *bp)
if (!(bp->fw_cap & BNXT_FW_CAP_PKG_VER))
bnxt_get_pkgver(dev);
+ bp->num_tests = 0;
if (bp->hwrm_spec_code < 0x10704 || !BNXT_SINGLE_PF(bp))
return;
@@ -3371,7 +3360,9 @@ void bnxt_ethtool_init(struct bnxt *bp)
if (rc)
goto ethtool_init_exit;
- test_info = kzalloc(sizeof(*bp->test_info), GFP_KERNEL);
+ test_info = bp->test_info;
+ if (!test_info)
+ test_info = kzalloc(sizeof(*bp->test_info), GFP_KERNEL);
if (!test_info)
goto ethtool_init_exit;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
index 2b90a2b..f6f3454 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
@@ -25,7 +25,6 @@
static int bnxt_hwrm_fwd_async_event_cmpl(struct bnxt *bp,
struct bnxt_vf_info *vf, u16 event_id)
{
- struct hwrm_fwd_async_event_cmpl_output *resp = bp->hwrm_cmd_resp_addr;
struct hwrm_fwd_async_event_cmpl_input req = {0};
struct hwrm_async_event_cmpl *async_cmpl;
int rc = 0;
@@ -40,23 +39,10 @@ static int bnxt_hwrm_fwd_async_event_cmpl(struct bnxt *bp,
async_cmpl->type = cpu_to_le16(ASYNC_EVENT_CMPL_TYPE_HWRM_ASYNC_EVENT);
async_cmpl->event_id = cpu_to_le16(event_id);
- mutex_lock(&bp->hwrm_cmd_lock);
- rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
-
- if (rc) {
+ rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ if (rc)
netdev_err(bp->dev, "hwrm_fwd_async_event_cmpl failed. rc:%d\n",
rc);
- goto fwd_async_event_cmpl_exit;
- }
-
- if (resp->error_code) {
- netdev_err(bp->dev, "hwrm_fwd_async_event_cmpl error %d\n",
- resp->error_code);
- rc = -1;
- }
-
-fwd_async_event_cmpl_exit:
- mutex_unlock(&bp->hwrm_cmd_lock);
return rc;
}
@@ -133,7 +119,7 @@ static int bnxt_hwrm_func_qcfg_flags(struct bnxt *bp, struct bnxt_vf_info *vf)
rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc) {
mutex_unlock(&bp->hwrm_cmd_lock);
- return -EIO;
+ return rc;
}
vf->func_qcfg_flags = le16_to_cpu(resp->flags);
mutex_unlock(&bp->hwrm_cmd_lock);
@@ -164,9 +150,7 @@ static int bnxt_hwrm_set_trusted_vf(struct bnxt *bp, struct bnxt_vf_info *vf)
else
req.flags = cpu_to_le32(FUNC_CFG_REQ_FLAGS_TRUSTED_VF_DISABLE);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
- if (rc)
- return -EIO;
- return 0;
+ return rc;
}
int bnxt_set_vf_trust(struct net_device *dev, int vf_id, bool trusted)
@@ -486,10 +470,43 @@ static int bnxt_hwrm_func_buf_rgtr(struct bnxt *bp)
return hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
}
+/* Caller holds bp->hwrm_cmd_lock mutex lock */
+static void __bnxt_set_vf_params(struct bnxt *bp, int vf_id)
+{
+ struct hwrm_func_cfg_input req = {0};
+ struct bnxt_vf_info *vf;
+
+ vf = &bp->pf.vf[vf_id];
+ bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_CFG, -1, -1);
+ req.fid = cpu_to_le16(vf->fw_fid);
+ req.flags = cpu_to_le32(vf->func_flags);
+
+ if (is_valid_ether_addr(vf->mac_addr)) {
+ req.enables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_DFLT_MAC_ADDR);
+ memcpy(req.dflt_mac_addr, vf->mac_addr, ETH_ALEN);
+ }
+ if (vf->vlan) {
+ req.enables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_DFLT_VLAN);
+ req.dflt_vlan = cpu_to_le16(vf->vlan);
+ }
+ if (vf->max_tx_rate) {
+ req.enables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_MAX_BW);
+ req.max_bw = cpu_to_le32(vf->max_tx_rate);
+#ifdef HAVE_IFLA_TX_RATE
+ req.enables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_MIN_BW);
+ req.min_bw = cpu_to_le32(vf->min_tx_rate);
+#endif
+ }
+ if (vf->flags & BNXT_VF_TRUST)
+ req.flags |= cpu_to_le32(FUNC_CFG_REQ_FLAGS_TRUSTED_VF_ENABLE);
+
+ _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+}
+
/* Only called by PF to reserve resources for VFs, returns actual number of
* VFs configured, or < 0 on error.
*/
-static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs)
+static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs, bool reset)
{
struct hwrm_func_vf_resource_cfg_input req = {0};
struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
@@ -561,13 +578,14 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs)
mutex_lock(&bp->hwrm_cmd_lock);
for (i = 0; i < num_vfs; i++) {
+ if (reset)
+ __bnxt_set_vf_params(bp, i);
+
req.vf_id = cpu_to_le16(pf->first_vf_id + i);
rc = _hwrm_send_message(bp, &req, sizeof(req),
HWRM_CMD_TIMEOUT);
- if (rc) {
- rc = -ENOMEM;
+ if (rc)
break;
- }
pf->active_vfs = i + 1;
pf->vf[i].fw_fid = pf->first_vf_id + i;
}
@@ -664,8 +682,6 @@ static int bnxt_hwrm_func_cfg(struct bnxt *bp, int num_vfs)
total_vf_tx_rings += vf_tx_rsvd;
}
mutex_unlock(&bp->hwrm_cmd_lock);
- if (rc)
- rc = -ENOMEM;
if (pf->active_vfs) {
hw_resc->max_tx_rings -= total_vf_tx_rings;
hw_resc->max_rx_rings -= vf_rx_rings * num_vfs;
@@ -679,14 +695,40 @@ static int bnxt_hwrm_func_cfg(struct bnxt *bp, int num_vfs)
return rc;
}
-static int bnxt_func_cfg(struct bnxt *bp, int num_vfs)
+static int bnxt_func_cfg(struct bnxt *bp, int num_vfs, bool reset)
{
if (BNXT_NEW_RM(bp))
- return bnxt_hwrm_func_vf_resc_cfg(bp, num_vfs);
+ return bnxt_hwrm_func_vf_resc_cfg(bp, num_vfs, reset);
else
return bnxt_hwrm_func_cfg(bp, num_vfs);
}
+int bnxt_cfg_hw_sriov(struct bnxt *bp, int *num_vfs, bool reset)
+{
+ int rc;
+
+ /* Register buffers for VFs */
+ rc = bnxt_hwrm_func_buf_rgtr(bp);
+ if (rc)
+ return rc;
+
+ /* Reserve resources for VFs */
+ rc = bnxt_func_cfg(bp, *num_vfs, reset);
+ if (rc != *num_vfs) {
+ if (rc <= 0) {
+ netdev_warn(bp->dev, "Unable to reserve resources for SRIOV.\n");
+ *num_vfs = 0;
+ return rc;
+ }
+ netdev_warn(bp->dev, "Only able to reserve resources for %d VFs.\n",
+ rc);
+ *num_vfs = rc;
+ }
+
+ bnxt_ulp_sriov_cfg(bp, *num_vfs);
+ return 0;
+}
+
static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs)
{
int rc = 0, vfs_supported;
@@ -752,25 +794,10 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs)
if (rc)
goto err_out1;
- /* Reserve resources for VFs */
- rc = bnxt_func_cfg(bp, *num_vfs);
- if (rc != *num_vfs) {
- if (rc <= 0) {
- netdev_warn(bp->dev, "Unable to reserve resources for SRIOV.\n");
- *num_vfs = 0;
- goto err_out2;
- }
- netdev_warn(bp->dev, "Only able to reserve resources for %d VFs.\n", rc);
- *num_vfs = rc;
- }
-
- /* Register buffers for VFs */
- rc = bnxt_hwrm_func_buf_rgtr(bp);
+ rc = bnxt_cfg_hw_sriov(bp, num_vfs, false);
if (rc)
goto err_out2;
- bnxt_ulp_sriov_cfg(bp, *num_vfs);
-
rc = pci_enable_sriov(bp->pdev, *num_vfs);
if (rc)
goto err_out2;
@@ -837,6 +864,11 @@ int bnxt_sriov_configure(struct pci_dev *pdev, int num_vfs)
rtnl_unlock();
return 0;
}
+ if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state)) {
+ netdev_warn(dev, "Reject SRIOV config request when FW reset is in progress\n");
+ rtnl_unlock();
+ return 0;
+ }
bp->sriov_cfg = true;
rtnl_unlock();
@@ -870,7 +902,6 @@ static int bnxt_hwrm_fwd_resp(struct bnxt *bp, struct bnxt_vf_info *vf,
{
int rc = 0;
struct hwrm_fwd_resp_input req = {0};
- struct hwrm_fwd_resp_output *resp = bp->hwrm_cmd_resp_addr;
if (BNXT_FWD_RESP_SIZE_ERR(msg_size))
return -EINVAL;
@@ -885,22 +916,9 @@ static int bnxt_hwrm_fwd_resp(struct bnxt *bp, struct bnxt_vf_info *vf,
req.encap_resp_cmpl_ring = encap_resp_cpr;
memcpy(req.encap_resp, encap_resp, msg_size);
- mutex_lock(&bp->hwrm_cmd_lock);
- rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
-
- if (rc) {
+ rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ if (rc)
netdev_err(bp->dev, "hwrm_fwd_resp failed. rc:%d\n", rc);
- goto fwd_resp_exit;
- }
-
- if (resp->error_code) {
- netdev_err(bp->dev, "hwrm_fwd_resp error %d\n",
- resp->error_code);
- rc = -1;
- }
-
-fwd_resp_exit:
- mutex_unlock(&bp->hwrm_cmd_lock);
return rc;
}
@@ -909,7 +927,6 @@ static int bnxt_hwrm_fwd_err_resp(struct bnxt *bp, struct bnxt_vf_info *vf,
{
int rc = 0;
struct hwrm_reject_fwd_resp_input req = {0};
- struct hwrm_reject_fwd_resp_output *resp = bp->hwrm_cmd_resp_addr;
if (BNXT_REJ_FWD_RESP_SIZE_ERR(msg_size))
return -EINVAL;
@@ -920,22 +937,9 @@ static int bnxt_hwrm_fwd_err_resp(struct bnxt *bp, struct bnxt_vf_info *vf,
req.encap_resp_target_id = cpu_to_le16(vf->fw_fid);
memcpy(req.encap_request, vf->hwrm_cmd_req_addr, msg_size);
- mutex_lock(&bp->hwrm_cmd_lock);
- rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
-
- if (rc) {
+ rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ if (rc)
netdev_err(bp->dev, "hwrm_fwd_err_resp failed. rc:%d\n", rc);
- goto fwd_err_resp_exit;
- }
-
- if (resp->error_code) {
- netdev_err(bp->dev, "hwrm_fwd_err_resp error %d\n",
- resp->error_code);
- rc = -1;
- }
-
-fwd_err_resp_exit:
- mutex_unlock(&bp->hwrm_cmd_lock);
return rc;
}
@@ -944,7 +948,6 @@ static int bnxt_hwrm_exec_fwd_resp(struct bnxt *bp, struct bnxt_vf_info *vf,
{
int rc = 0;
struct hwrm_exec_fwd_resp_input req = {0};
- struct hwrm_exec_fwd_resp_output *resp = bp->hwrm_cmd_resp_addr;
if (BNXT_EXEC_FWD_RESP_SIZE_ERR(msg_size))
return -EINVAL;
@@ -955,22 +958,9 @@ static int bnxt_hwrm_exec_fwd_resp(struct bnxt *bp, struct bnxt_vf_info *vf,
req.encap_resp_target_id = cpu_to_le16(vf->fw_fid);
memcpy(req.encap_request, vf->hwrm_cmd_req_addr, msg_size);
- mutex_lock(&bp->hwrm_cmd_lock);
- rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
-
- if (rc) {
+ rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
+ if (rc)
netdev_err(bp->dev, "hwrm_exec_fw_resp failed. rc:%d\n", rc);
- goto exec_fwd_resp_exit;
- }
-
- if (resp->error_code) {
- netdev_err(bp->dev, "hwrm_exec_fw_resp error %d\n",
- resp->error_code);
- rc = -1;
- }
-
-exec_fwd_resp_exit:
- mutex_unlock(&bp->hwrm_cmd_lock);
return rc;
}
@@ -1190,6 +1180,13 @@ int bnxt_approve_mac(struct bnxt *bp, u8 *mac, bool strict)
}
#else
+int bnxt_cfg_hw_sriov(struct bnxt *bp, int *num_vfs, bool reset)
+{
+ if (*num_vfs)
+ return -EOPNOTSUPP;
+ return 0;
+}
+
void bnxt_sriov_disable(struct bnxt *bp)
{
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h
index 2eed9ed..629641b 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h
@@ -36,6 +36,7 @@ int bnxt_set_vf_link_state(struct net_device *, int, int);
int bnxt_set_vf_spoofchk(struct net_device *, int, bool);
int bnxt_set_vf_trust(struct net_device *dev, int vf_id, bool trust);
int bnxt_sriov_configure(struct pci_dev *pdev, int num_vfs);
+int bnxt_cfg_hw_sriov(struct bnxt *bp, int *num_vfs, bool reset);
void bnxt_sriov_disable(struct bnxt *);
void bnxt_hwrm_exec_fwd_req(struct bnxt *);
void bnxt_update_vf_mac(struct bnxt *);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c
index dd621f6..c8062d0 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c
@@ -319,8 +319,6 @@ static int bnxt_hwrm_cfa_flow_free(struct bnxt *bp,
if (rc)
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -515,11 +513,6 @@ static int bnxt_hwrm_cfa_flow_alloc(struct bnxt *bp, struct bnxt_tc_flow *flow,
}
}
mutex_unlock(&bp->hwrm_cmd_lock);
-
- if (rc == HWRM_ERR_CODE_RESOURCE_ALLOC_ERROR)
- rc = -ENOSPC;
- else if (rc)
- rc = -EIO;
return rc;
}
@@ -591,8 +584,6 @@ static int hwrm_cfa_decap_filter_alloc(struct bnxt *bp,
}
mutex_unlock(&bp->hwrm_cmd_lock);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -609,8 +600,6 @@ static int hwrm_cfa_decap_filter_free(struct bnxt *bp,
if (rc)
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -660,8 +649,6 @@ static int hwrm_cfa_encap_record_alloc(struct bnxt *bp,
}
mutex_unlock(&bp->hwrm_cmd_lock);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -678,8 +665,6 @@ static int hwrm_cfa_encap_record_free(struct bnxt *bp,
if (rc)
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
- if (rc)
- rc = -EIO;
return rc;
}
@@ -1457,8 +1442,6 @@ bnxt_hwrm_cfa_flow_stats_get(struct bnxt *bp, int num_flows,
}
mutex_unlock(&bp->hwrm_cmd_lock);
- if (rc)
- rc = -EIO;
return rc;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
index fc77caf..b2c1609 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c
@@ -226,6 +226,9 @@ static int bnxt_send_msg(struct bnxt_en_dev *edev, int ulp_id,
struct input *req;
int rc;
+ if (ulp_id != BNXT_ROCE_ULP && bp->fw_reset_state)
+ return -EBUSY;
+
mutex_lock(&bp->hwrm_cmd_lock);
req = fw_msg->msg;
req->resp_addr = cpu_to_le64(bp->hwrm_cmd_resp_dma_addr);
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 2108e59..12cb77e 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -1124,6 +1124,7 @@ static const struct ethtool_ops bcmgenet_ethtool_ops = {
.set_coalesce = bcmgenet_set_coalesce,
.get_link_ksettings = bcmgenet_get_link_ksettings,
.set_link_ksettings = bcmgenet_set_link_ksettings,
+ .get_ts_info = ethtool_op_get_ts_info,
};
/* Power down the unimac, based on mode. */
@@ -2515,19 +2516,14 @@ static int bcmgenet_dma_teardown(struct bcmgenet_priv *priv)
static void bcmgenet_fini_dma(struct bcmgenet_priv *priv)
{
struct netdev_queue *txq;
- struct sk_buff *skb;
- struct enet_cb *cb;
int i;
bcmgenet_fini_rx_napi(priv);
bcmgenet_fini_tx_napi(priv);
- for (i = 0; i < priv->num_tx_bds; i++) {
- cb = priv->tx_cbs + i;
- skb = bcmgenet_free_tx_cb(&priv->pdev->dev, cb);
- if (skb)
- dev_kfree_skb(skb);
- }
+ for (i = 0; i < priv->num_tx_bds; i++)
+ dev_kfree_skb(bcmgenet_free_tx_cb(&priv->pdev->dev,
+ priv->tx_cbs + i));
for (i = 0; i < priv->hw_params->tx_queues; i++) {
txq = netdev_get_tx_queue(priv->dev, priv->tx_rings[i].queue);
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 5ca17e6..35b59b5 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -4154,7 +4154,7 @@ static const struct of_device_id macb_dt_ids[] = {
{ .compatible = "cdns,emac", .data = &emac_config },
{ .compatible = "cdns,zynqmp-gem", .data = &zynqmp_config},
{ .compatible = "cdns,zynq-gem", .data = &zynq_config },
- { .compatible = "sifive,fu540-macb", .data = &fu540_c000_config },
+ { .compatible = "sifive,fu540-c000-gem", .data = &fu540_c000_config },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, macb_dt_ids);
diff --git a/drivers/net/ethernet/cirrus/cs89x0.c b/drivers/net/ethernet/cirrus/cs89x0.c
index 2d30972..c9aebcd 100644
--- a/drivers/net/ethernet/cirrus/cs89x0.c
+++ b/drivers/net/ethernet/cirrus/cs89x0.c
@@ -1844,15 +1844,12 @@ cleanup_module(void)
static int __init cs89x0_platform_probe(struct platform_device *pdev)
{
struct net_device *dev = alloc_etherdev(sizeof(struct net_local));
- struct net_local *lp;
void __iomem *virt_addr;
int err;
if (!dev)
return -ENOMEM;
- lp = netdev_priv(dev);
-
dev->irq = platform_get_irq(pdev, 0);
if (dev->irq <= 0) {
dev_warn(&dev->dev, "interrupt resource missing\n");
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c b/drivers/net/ethernet/emulex/benet/be_cmds.c
index 323976c..701c12c 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.c
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.c
@@ -2756,7 +2756,7 @@ static int be_flash_BEx(struct be_adapter *adapter,
bool crc_match;
const u8 *p;
- struct flash_comp gen3_flash_types[] = {
+ static const struct flash_comp gen3_flash_types[] = {
{ BE3_ISCSI_PRIMARY_IMAGE_START, OPTYPE_ISCSI_ACTIVE,
BE3_COMP_MAX_SIZE, IMAGE_FIRMWARE_ISCSI},
{ BE3_REDBOOT_START, OPTYPE_REDBOOT,
@@ -2779,7 +2779,7 @@ static int be_flash_BEx(struct be_adapter *adapter,
BE3_PHY_FW_COMP_MAX_SIZE, IMAGE_FIRMWARE_PHY}
};
- struct flash_comp gen2_flash_types[] = {
+ static const struct flash_comp gen2_flash_types[] = {
{ BE2_ISCSI_PRIMARY_IMAGE_START, OPTYPE_ISCSI_ACTIVE,
BE2_COMP_MAX_SIZE, IMAGE_FIRMWARE_ISCSI},
{ BE2_REDBOOT_START, OPTYPE_REDBOOT,
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
index 0acb115..162d7d8 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
@@ -1208,9 +1208,37 @@ static void disable_ch_napi(struct dpaa2_eth_priv *priv)
}
}
+static void dpaa2_eth_set_rx_taildrop(struct dpaa2_eth_priv *priv, bool enable)
+{
+ struct dpni_taildrop td = {0};
+ int i, err;
+
+ if (priv->rx_td_enabled == enable)
+ return;
+
+ td.enable = enable;
+ td.threshold = DPAA2_ETH_TAILDROP_THRESH;
+
+ for (i = 0; i < priv->num_fqs; i++) {
+ if (priv->fq[i].type != DPAA2_RX_FQ)
+ continue;
+ err = dpni_set_taildrop(priv->mc_io, 0, priv->mc_token,
+ DPNI_CP_QUEUE, DPNI_QUEUE_RX, 0,
+ priv->fq[i].flowid, &td);
+ if (err) {
+ netdev_err(priv->net_dev,
+ "dpni_set_taildrop() failed\n");
+ break;
+ }
+ }
+
+ priv->rx_td_enabled = enable;
+}
+
static int link_state_update(struct dpaa2_eth_priv *priv)
{
struct dpni_link_state state = {0};
+ bool tx_pause;
int err;
err = dpni_get_link_state(priv->mc_io, 0, priv->mc_token, &state);
@@ -1220,11 +1248,18 @@ static int link_state_update(struct dpaa2_eth_priv *priv)
return err;
}
+ /* If Tx pause frame settings have changed, we need to update
+ * Rx FQ taildrop configuration as well. We configure taildrop
+ * only when pause frame generation is disabled.
+ */
+ tx_pause = !!(state.options & DPNI_LINK_OPT_PAUSE) ^
+ !!(state.options & DPNI_LINK_OPT_ASYM_PAUSE);
+ dpaa2_eth_set_rx_taildrop(priv, !tx_pause);
+
/* Chech link state; speed / duplex changes are not treated yet */
if (priv->link_state.up == state.up)
- return 0;
+ goto out;
- priv->link_state = state;
if (state.up) {
netif_carrier_on(priv->net_dev);
netif_tx_start_all_queues(priv->net_dev);
@@ -1236,6 +1271,9 @@ static int link_state_update(struct dpaa2_eth_priv *priv)
netdev_info(priv->net_dev, "Link Event: state %s\n",
state.up ? "up" : "down");
+out:
+ priv->link_state = state;
+
return 0;
}
@@ -1310,7 +1348,7 @@ static u32 ingress_fq_count(struct dpaa2_eth_priv *priv)
return total;
}
-static void wait_for_fq_empty(struct dpaa2_eth_priv *priv)
+static void wait_for_ingress_fq_empty(struct dpaa2_eth_priv *priv)
{
int retries = 10;
u32 pending;
@@ -1322,6 +1360,31 @@ static void wait_for_fq_empty(struct dpaa2_eth_priv *priv)
} while (pending && --retries);
}
+#define DPNI_TX_PENDING_VER_MAJOR 7
+#define DPNI_TX_PENDING_VER_MINOR 13
+static void wait_for_egress_fq_empty(struct dpaa2_eth_priv *priv)
+{
+ union dpni_statistics stats;
+ int retries = 10;
+ int err;
+
+ if (dpaa2_eth_cmp_dpni_ver(priv, DPNI_TX_PENDING_VER_MAJOR,
+ DPNI_TX_PENDING_VER_MINOR) < 0)
+ goto out;
+
+ do {
+ err = dpni_get_statistics(priv->mc_io, 0, priv->mc_token, 6,
+ &stats);
+ if (err)
+ goto out;
+ if (stats.page_6.tx_pending_frames == 0)
+ return;
+ } while (--retries);
+
+out:
+ msleep(500);
+}
+
static int dpaa2_eth_stop(struct net_device *net_dev)
{
struct dpaa2_eth_priv *priv = netdev_priv(net_dev);
@@ -1341,7 +1404,7 @@ static int dpaa2_eth_stop(struct net_device *net_dev)
* on WRIOP. After it finishes, wait until all remaining frames on Rx
* and Tx conf queues are consumed on NAPI poll.
*/
- msleep(500);
+ wait_for_egress_fq_empty(priv);
do {
dpni_disable(priv->mc_io, 0, priv->mc_token);
@@ -1357,7 +1420,7 @@ static int dpaa2_eth_stop(struct net_device *net_dev)
*/
}
- wait_for_fq_empty(priv);
+ wait_for_ingress_fq_empty(priv);
disable_ch_napi(priv);
/* Empty the buffer pool */
@@ -2443,6 +2506,33 @@ static void set_enqueue_mode(struct dpaa2_eth_priv *priv)
priv->enqueue = dpaa2_eth_enqueue_fq;
}
+static int set_pause(struct dpaa2_eth_priv *priv)
+{
+ struct device *dev = priv->net_dev->dev.parent;
+ struct dpni_link_cfg link_cfg = {0};
+ int err;
+
+ /* Get the default link options so we don't override other flags */
+ err = dpni_get_link_cfg(priv->mc_io, 0, priv->mc_token, &link_cfg);
+ if (err) {
+ dev_err(dev, "dpni_get_link_cfg() failed\n");
+ return err;
+ }
+
+ /* By default, enable both Rx and Tx pause frames */
+ link_cfg.options |= DPNI_LINK_OPT_PAUSE;
+ link_cfg.options &= ~DPNI_LINK_OPT_ASYM_PAUSE;
+ err = dpni_set_link_cfg(priv->mc_io, 0, priv->mc_token, &link_cfg);
+ if (err) {
+ dev_err(dev, "dpni_set_link_cfg() failed\n");
+ return err;
+ }
+
+ priv->link_state.options = link_cfg.options;
+
+ return 0;
+}
+
/* Configure the DPNI object this interface is associated with */
static int setup_dpni(struct fsl_mc_device *ls_dev)
{
@@ -2498,6 +2588,13 @@ static int setup_dpni(struct fsl_mc_device *ls_dev)
set_enqueue_mode(priv);
+ /* Enable pause frame support */
+ if (dpaa2_eth_has_pause_support(priv)) {
+ err = set_pause(priv);
+ if (err)
+ goto close;
+ }
+
priv->cls_rules = devm_kzalloc(dev, sizeof(struct dpaa2_eth_cls_rule) *
dpaa2_eth_fs_count(priv), GFP_KERNEL);
if (!priv->cls_rules)
@@ -2529,7 +2626,6 @@ static int setup_rx_flow(struct dpaa2_eth_priv *priv,
struct device *dev = priv->net_dev->dev.parent;
struct dpni_queue queue;
struct dpni_queue_id qid;
- struct dpni_taildrop td;
int err;
err = dpni_get_queue(priv->mc_io, 0, priv->mc_token,
@@ -2554,15 +2650,6 @@ static int setup_rx_flow(struct dpaa2_eth_priv *priv,
return err;
}
- td.enable = 1;
- td.threshold = DPAA2_ETH_TAILDROP_THRESH;
- err = dpni_set_taildrop(priv->mc_io, 0, priv->mc_token, DPNI_CP_QUEUE,
- DPNI_QUEUE_RX, 0, fq->flowid, &td);
- if (err) {
- dev_err(dev, "dpni_set_threshold() failed\n");
- return err;
- }
-
/* xdp_rxq setup */
err = xdp_rxq_info_reg(&fq->channel->xdp_rxq, priv->net_dev,
fq->flowid);
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.h b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.h
index 9af18c2..8a0e65b 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.h
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.h
@@ -392,6 +392,7 @@ struct dpaa2_eth_priv {
struct dpaa2_eth_drv_stats __percpu *percpu_extras;
u16 mc_token;
+ u8 rx_td_enabled;
struct dpni_link_state link_state;
bool do_link_poll;
@@ -476,6 +477,12 @@ enum dpaa2_eth_rx_dist {
#define DPAA2_ETH_DIST_L4DST BIT(8)
#define DPAA2_ETH_DIST_ALL (~0ULL)
+#define DPNI_PAUSE_VER_MAJOR 7
+#define DPNI_PAUSE_VER_MINOR 13
+#define dpaa2_eth_has_pause_support(priv) \
+ (dpaa2_eth_cmp_dpni_ver((priv), DPNI_PAUSE_VER_MAJOR, \
+ DPNI_PAUSE_VER_MINOR) >= 0)
+
static inline
unsigned int dpaa2_eth_needed_headroom(struct dpaa2_eth_priv *priv,
struct sk_buff *skb)
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-ethtool.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-ethtool.c
index 7b182f4..0aa1c34 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-ethtool.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-ethtool.c
@@ -28,6 +28,11 @@ static char dpaa2_ethtool_stats[][ETH_GSTRING_LEN] = {
"[hw] rx nobuffer discards",
"[hw] tx discarded frames",
"[hw] tx confirmed frames",
+ "[hw] tx dequeued bytes",
+ "[hw] tx dequeued frames",
+ "[hw] tx rejected bytes",
+ "[hw] tx rejected frames",
+ "[hw] tx pending frames",
};
#define DPAA2_ETH_NUM_STATS ARRAY_SIZE(dpaa2_ethtool_stats)
@@ -78,71 +83,67 @@ static int
dpaa2_eth_get_link_ksettings(struct net_device *net_dev,
struct ethtool_link_ksettings *link_settings)
{
- struct dpni_link_state state = {0};
- int err = 0;
struct dpaa2_eth_priv *priv = netdev_priv(net_dev);
- err = dpni_get_link_state(priv->mc_io, 0, priv->mc_token, &state);
- if (err) {
- netdev_err(net_dev, "ERROR %d getting link state\n", err);
- goto out;
- }
-
- /* At the moment, we have no way of interrogating the DPMAC
- * from the DPNI side - and for that matter there may exist
- * no DPMAC at all. So for now we just don't report anything
- * beyond the DPNI attributes.
- */
- if (state.options & DPNI_LINK_OPT_AUTONEG)
- link_settings->base.autoneg = AUTONEG_ENABLE;
- if (!(state.options & DPNI_LINK_OPT_HALF_DUPLEX))
+ link_settings->base.autoneg = AUTONEG_DISABLE;
+ if (!(priv->link_state.options & DPNI_LINK_OPT_HALF_DUPLEX))
link_settings->base.duplex = DUPLEX_FULL;
- link_settings->base.speed = state.rate;
+ link_settings->base.speed = priv->link_state.rate;
-out:
- return err;
+ return 0;
}
-#define DPNI_DYNAMIC_LINK_SET_VER_MAJOR 7
-#define DPNI_DYNAMIC_LINK_SET_VER_MINOR 1
-static int
-dpaa2_eth_set_link_ksettings(struct net_device *net_dev,
- const struct ethtool_link_ksettings *link_settings)
+static void dpaa2_eth_get_pauseparam(struct net_device *net_dev,
+ struct ethtool_pauseparam *pause)
{
- struct dpni_link_cfg cfg = {0};
struct dpaa2_eth_priv *priv = netdev_priv(net_dev);
- int err = 0;
+ u64 link_options = priv->link_state.options;
- /* If using an older MC version, the DPNI must be down
- * in order to be able to change link settings. Taking steps to let
- * the user know that.
- */
- if (dpaa2_eth_cmp_dpni_ver(priv, DPNI_DYNAMIC_LINK_SET_VER_MAJOR,
- DPNI_DYNAMIC_LINK_SET_VER_MINOR) < 0) {
- if (netif_running(net_dev)) {
- netdev_info(net_dev, "Interface must be brought down first.\n");
- return -EACCES;
- }
+ pause->rx_pause = !!(link_options & DPNI_LINK_OPT_PAUSE);
+ pause->tx_pause = pause->rx_pause ^
+ !!(link_options & DPNI_LINK_OPT_ASYM_PAUSE);
+ pause->autoneg = AUTONEG_DISABLE;
+}
+
+static int dpaa2_eth_set_pauseparam(struct net_device *net_dev,
+ struct ethtool_pauseparam *pause)
+{
+ struct dpaa2_eth_priv *priv = netdev_priv(net_dev);
+ struct dpni_link_cfg cfg = {0};
+ int err;
+
+ if (!dpaa2_eth_has_pause_support(priv)) {
+ netdev_info(net_dev, "No pause frame support for DPNI version < %d.%d\n",
+ DPNI_PAUSE_VER_MAJOR, DPNI_PAUSE_VER_MINOR);
+ return -EOPNOTSUPP;
}
- cfg.rate = link_settings->base.speed;
- if (link_settings->base.autoneg == AUTONEG_ENABLE)
- cfg.options |= DPNI_LINK_OPT_AUTONEG;
+ if (pause->autoneg)
+ return -EOPNOTSUPP;
+
+ cfg.rate = priv->link_state.rate;
+ cfg.options = priv->link_state.options;
+ if (pause->rx_pause)
+ cfg.options |= DPNI_LINK_OPT_PAUSE;
else
- cfg.options &= ~DPNI_LINK_OPT_AUTONEG;
- if (link_settings->base.duplex == DUPLEX_HALF)
- cfg.options |= DPNI_LINK_OPT_HALF_DUPLEX;
+ cfg.options &= ~DPNI_LINK_OPT_PAUSE;
+ if (!!pause->rx_pause ^ !!pause->tx_pause)
+ cfg.options |= DPNI_LINK_OPT_ASYM_PAUSE;
else
- cfg.options &= ~DPNI_LINK_OPT_HALF_DUPLEX;
+ cfg.options &= ~DPNI_LINK_OPT_ASYM_PAUSE;
+
+ if (cfg.options == priv->link_state.options)
+ return 0;
err = dpni_set_link_cfg(priv->mc_io, 0, priv->mc_token, &cfg);
- if (err)
- /* ethtool will be loud enough if we return an error; no point
- * in putting our own error message on the console by default
- */
- netdev_dbg(net_dev, "ERROR %d setting link cfg\n", err);
+ if (err) {
+ netdev_err(net_dev, "dpni_set_link_state failed\n");
+ return err;
+ }
- return err;
+ priv->link_state.options = cfg.options;
+
+ return 0;
}
static void dpaa2_eth_get_strings(struct net_device *netdev, u32 stringset,
@@ -192,27 +193,33 @@ static void dpaa2_eth_get_ethtool_stats(struct net_device *net_dev,
struct dpaa2_eth_priv *priv = netdev_priv(net_dev);
struct dpaa2_eth_drv_stats *extras;
struct dpaa2_eth_ch_stats *ch_stats;
+ int dpni_stats_page_size[DPNI_STATISTICS_CNT] = {
+ sizeof(dpni_stats.page_0),
+ sizeof(dpni_stats.page_1),
+ sizeof(dpni_stats.page_2),
+ sizeof(dpni_stats.page_3),
+ sizeof(dpni_stats.page_4),
+ sizeof(dpni_stats.page_5),
+ sizeof(dpni_stats.page_6),
+ };
memset(data, 0,
sizeof(u64) * (DPAA2_ETH_NUM_STATS + DPAA2_ETH_NUM_EXTRA_STATS));
/* Print standard counters, from DPNI statistics */
- for (j = 0; j <= 2; j++) {
+ for (j = 0; j <= 6; j++) {
+ /* We're not interested in pages 4 & 5 for now */
+ if (j == 4 || j == 5)
+ continue;
err = dpni_get_statistics(priv->mc_io, 0, priv->mc_token,
j, &dpni_stats);
- if (err != 0)
+ if (err == -EINVAL)
+ /* Older firmware versions don't support all pages */
+ memset(&dpni_stats, 0, sizeof(dpni_stats));
+ else
netdev_warn(net_dev, "dpni_get_stats(%d) failed\n", j);
- switch (j) {
- case 0:
- num_cnt = sizeof(dpni_stats.page_0) / sizeof(u64);
- break;
- case 1:
- num_cnt = sizeof(dpni_stats.page_1) / sizeof(u64);
- break;
- case 2:
- num_cnt = sizeof(dpni_stats.page_2) / sizeof(u64);
- break;
- }
+
+ num_cnt = dpni_stats_page_size[j] / sizeof(u64);
for (k = 0; k < num_cnt; k++)
*(data + i++) = dpni_stats.raw.counter[k];
}
@@ -721,7 +728,8 @@ const struct ethtool_ops dpaa2_ethtool_ops = {
.get_drvinfo = dpaa2_eth_get_drvinfo,
.get_link = ethtool_op_get_link,
.get_link_ksettings = dpaa2_eth_get_link_ksettings,
- .set_link_ksettings = dpaa2_eth_set_link_ksettings,
+ .get_pauseparam = dpaa2_eth_get_pauseparam,
+ .set_pauseparam = dpaa2_eth_set_pauseparam,
.get_sset_count = dpaa2_eth_get_sset_count,
.get_ethtool_stats = dpaa2_eth_get_ethtool_stats,
.get_strings = dpaa2_eth_get_strings,
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpni-cmd.h b/drivers/net/ethernet/freescale/dpaa2/dpni-cmd.h
index 7b44d7d..d9b6918 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpni-cmd.h
+++ b/drivers/net/ethernet/freescale/dpaa2/dpni-cmd.h
@@ -84,6 +84,7 @@
#define DPNI_CMDID_SET_RX_FS_DIST DPNI_CMD(0x273)
#define DPNI_CMDID_SET_RX_HASH_DIST DPNI_CMD(0x274)
+#define DPNI_CMDID_GET_LINK_CFG DPNI_CMD(0x278)
/* Macros for accessing command fields smaller than 1byte */
#define DPNI_MASK(field) \
@@ -284,7 +285,7 @@ struct dpni_rsp_get_statistics {
__le64 counter[DPNI_STATISTICS_CNT];
};
-struct dpni_cmd_set_link_cfg {
+struct dpni_cmd_link_cfg {
/* cmd word 0 */
__le64 pad0;
/* cmd word 1 */
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpni.c b/drivers/net/ethernet/freescale/dpaa2/dpni.c
index 220dfc8..dd54e69 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpni.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpni.c
@@ -838,13 +838,13 @@ int dpni_set_link_cfg(struct fsl_mc_io *mc_io,
const struct dpni_link_cfg *cfg)
{
struct fsl_mc_command cmd = { 0 };
- struct dpni_cmd_set_link_cfg *cmd_params;
+ struct dpni_cmd_link_cfg *cmd_params;
/* prepare command */
cmd.header = mc_encode_cmd_header(DPNI_CMDID_SET_LINK_CFG,
cmd_flags,
token);
- cmd_params = (struct dpni_cmd_set_link_cfg *)cmd.params;
+ cmd_params = (struct dpni_cmd_link_cfg *)cmd.params;
cmd_params->rate = cpu_to_le32(cfg->rate);
cmd_params->options = cpu_to_le64(cfg->options);
@@ -853,6 +853,42 @@ int dpni_set_link_cfg(struct fsl_mc_io *mc_io,
}
/**
+ * dpni_get_link_cfg() - return the link configuration
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPNI object
+ * @cfg: Link configuration from dpni object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpni_get_link_cfg(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpni_link_cfg *cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpni_cmd_link_cfg *rsp_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPNI_CMDID_GET_LINK_CFG,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ rsp_params = (struct dpni_cmd_link_cfg *)cmd.params;
+ cfg->rate = le32_to_cpu(rsp_params->rate);
+ cfg->options = le64_to_cpu(rsp_params->options);
+
+ return err;
+}
+
+/**
* dpni_get_link_state() - Return the link state (either up or down)
* @mc_io: Pointer to MC portal's I/O object
* @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
@@ -1434,7 +1470,7 @@ int dpni_get_queue(struct fsl_mc_io *mc_io,
* @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
* @token: Token of DPNI object
* @page: Selects the statistics page to retrieve, see
- * DPNI_GET_STATISTICS output. Pages are numbered 0 to 2.
+ * DPNI_GET_STATISTICS output. Pages are numbered 0 to 6.
* @stat: Structure containing the statistics
*
* Return: '0' on Success; Error code otherwise.
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpni.h b/drivers/net/ethernet/freescale/dpaa2/dpni.h
index a521242..fd583911 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpni.h
+++ b/drivers/net/ethernet/freescale/dpaa2/dpni.h
@@ -416,6 +416,26 @@ int dpni_get_tx_data_offset(struct fsl_mc_io *mc_io,
* lack of buffers
* @page_2.egress_discarded_frames: Egress discarded frame count
* @page_2.egress_confirmed_frames: Egress confirmed frame count
+ * @page3: Page_3 statistics structure
+ * @page_3.egress_dequeue_bytes: Cumulative count of the number of bytes
+ * dequeued from egress FQs
+ * @page_3.egress_dequeue_frames: Cumulative count of the number of frames
+ * dequeued from egress FQs
+ * @page_3.egress_reject_bytes: Cumulative count of the number of bytes in
+ * egress frames whose enqueue was rejected
+ * @page_3.egress_reject_frames: Cumulative count of the number of egress
+ * frames whose enqueue was rejected
+ * @page_4: Page_4 statistics structure: congestion points
+ * @page_4.cgr_reject_frames: number of rejected frames due to congestion point
+ * @page_4.cgr_reject_bytes: number of rejected bytes due to congestion point
+ * @page_5: Page_5 statistics structure: policer
+ * @page_5.policer_cnt_red: NUmber of red colored frames
+ * @page_5.policer_cnt_yellow: number of yellow colored frames
+ * @page_5.policer_cnt_green: number of green colored frames
+ * @page_5.policer_cnt_re_red: number of recolored red frames
+ * @page_5.policer_cnt_re_yellow: number of recolored yellow frames
+ * @page_6: Page_6 statistics structure
+ * @page_6.tx_pending_frames: total number of frames pending in egress FQs
* @raw: raw statistics structure, used to index counters
*/
union dpni_statistics {
@@ -443,6 +463,26 @@ union dpni_statistics {
u64 egress_confirmed_frames;
} page_2;
struct {
+ u64 egress_dequeue_bytes;
+ u64 egress_dequeue_frames;
+ u64 egress_reject_bytes;
+ u64 egress_reject_frames;
+ } page_3;
+ struct {
+ u64 cgr_reject_frames;
+ u64 cgr_reject_bytes;
+ } page_4;
+ struct {
+ u64 policer_cnt_red;
+ u64 policer_cnt_yellow;
+ u64 policer_cnt_green;
+ u64 policer_cnt_re_red;
+ u64 policer_cnt_re_yellow;
+ } page_5;
+ struct {
+ u64 tx_pending_frames;
+ } page_6;
+ struct {
u64 counter[DPNI_STATISTICS_CNT];
} raw;
};
@@ -485,6 +525,11 @@ int dpni_set_link_cfg(struct fsl_mc_io *mc_io,
u16 token,
const struct dpni_link_cfg *cfg);
+int dpni_get_link_cfg(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpni_link_cfg *cfg);
+
/**
* struct dpni_link_state - Structure representing DPNI link state
* @rate: Rate
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ptp.c b/drivers/net/ethernet/freescale/enetc/enetc_ptp.c
index 2fd2586..bc59489 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_ptp.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_ptp.c
@@ -82,7 +82,7 @@ static int enetc_ptp_probe(struct pci_dev *pdev,
n = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSIX);
if (n != 1) {
err = -EPERM;
- goto err_irq;
+ goto err_irq_vectors;
}
ptp_qoriq->irq = pci_irq_vector(pdev, 0);
@@ -107,6 +107,8 @@ static int enetc_ptp_probe(struct pci_dev *pdev,
err_no_clock:
free_irq(ptp_qoriq->irq, ptp_qoriq);
err_irq:
+ pci_free_irq_vectors(pdev);
+err_irq_vectors:
iounmap(base);
err_ioremap:
kfree(ptp_qoriq);
@@ -125,6 +127,7 @@ static void enetc_ptp_remove(struct pci_dev *pdev)
enetc_phc_index = -1;
ptp_qoriq_free(ptp_qoriq);
+ pci_free_irq_vectors(pdev);
kfree(ptp_qoriq);
pci_release_mem_regions(pdev);
diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c
index cacc671..d4d4c72 100644
--- a/drivers/net/ethernet/freescale/fec_main.c
+++ b/drivers/net/ethernet/freescale/fec_main.c
@@ -208,8 +208,11 @@ MODULE_PARM_DESC(macaddr, "FEC Ethernet MAC address");
/* FEC MII MMFR bits definition */
#define FEC_MMFR_ST (1 << 30)
+#define FEC_MMFR_ST_C45 (0)
#define FEC_MMFR_OP_READ (2 << 28)
+#define FEC_MMFR_OP_READ_C45 (3 << 28)
#define FEC_MMFR_OP_WRITE (1 << 28)
+#define FEC_MMFR_OP_ADDR_WRITE (0)
#define FEC_MMFR_PA(v) ((v & 0x1f) << 23)
#define FEC_MMFR_RA(v) ((v & 0x1f) << 18)
#define FEC_MMFR_TA (2 << 16)
@@ -1767,7 +1770,8 @@ static int fec_enet_mdio_read(struct mii_bus *bus, int mii_id, int regnum)
struct fec_enet_private *fep = bus->priv;
struct device *dev = &fep->pdev->dev;
unsigned long time_left;
- int ret = 0;
+ int ret = 0, frame_start, frame_addr, frame_op;
+ bool is_c45 = !!(regnum & MII_ADDR_C45);
ret = pm_runtime_get_sync(dev);
if (ret < 0)
@@ -1775,9 +1779,37 @@ static int fec_enet_mdio_read(struct mii_bus *bus, int mii_id, int regnum)
reinit_completion(&fep->mdio_done);
+ if (is_c45) {
+ frame_start = FEC_MMFR_ST_C45;
+
+ /* write address */
+ frame_addr = (regnum >> 16);
+ writel(frame_start | FEC_MMFR_OP_ADDR_WRITE |
+ FEC_MMFR_PA(mii_id) | FEC_MMFR_RA(frame_addr) |
+ FEC_MMFR_TA | (regnum & 0xFFFF),
+ fep->hwp + FEC_MII_DATA);
+
+ /* wait for end of transfer */
+ time_left = wait_for_completion_timeout(&fep->mdio_done,
+ usecs_to_jiffies(FEC_MII_TIMEOUT));
+ if (time_left == 0) {
+ netdev_err(fep->netdev, "MDIO address write timeout\n");
+ ret = -ETIMEDOUT;
+ goto out;
+ }
+
+ frame_op = FEC_MMFR_OP_READ_C45;
+
+ } else {
+ /* C22 read */
+ frame_op = FEC_MMFR_OP_READ;
+ frame_start = FEC_MMFR_ST;
+ frame_addr = regnum;
+ }
+
/* start a read op */
- writel(FEC_MMFR_ST | FEC_MMFR_OP_READ |
- FEC_MMFR_PA(mii_id) | FEC_MMFR_RA(regnum) |
+ writel(frame_start | frame_op |
+ FEC_MMFR_PA(mii_id) | FEC_MMFR_RA(frame_addr) |
FEC_MMFR_TA, fep->hwp + FEC_MII_DATA);
/* wait for end of transfer */
@@ -1804,7 +1836,8 @@ static int fec_enet_mdio_write(struct mii_bus *bus, int mii_id, int regnum,
struct fec_enet_private *fep = bus->priv;
struct device *dev = &fep->pdev->dev;
unsigned long time_left;
- int ret;
+ int ret, frame_start, frame_addr;
+ bool is_c45 = !!(regnum & MII_ADDR_C45);
ret = pm_runtime_get_sync(dev);
if (ret < 0)
@@ -1814,9 +1847,33 @@ static int fec_enet_mdio_write(struct mii_bus *bus, int mii_id, int regnum,
reinit_completion(&fep->mdio_done);
+ if (is_c45) {
+ frame_start = FEC_MMFR_ST_C45;
+
+ /* write address */
+ frame_addr = (regnum >> 16);
+ writel(frame_start | FEC_MMFR_OP_ADDR_WRITE |
+ FEC_MMFR_PA(mii_id) | FEC_MMFR_RA(frame_addr) |
+ FEC_MMFR_TA | (regnum & 0xFFFF),
+ fep->hwp + FEC_MII_DATA);
+
+ /* wait for end of transfer */
+ time_left = wait_for_completion_timeout(&fep->mdio_done,
+ usecs_to_jiffies(FEC_MII_TIMEOUT));
+ if (time_left == 0) {
+ netdev_err(fep->netdev, "MDIO address write timeout\n");
+ ret = -ETIMEDOUT;
+ goto out;
+ }
+ } else {
+ /* C22 write */
+ frame_start = FEC_MMFR_ST;
+ frame_addr = regnum;
+ }
+
/* start a write op */
- writel(FEC_MMFR_ST | FEC_MMFR_OP_WRITE |
- FEC_MMFR_PA(mii_id) | FEC_MMFR_RA(regnum) |
+ writel(frame_start | FEC_MMFR_OP_WRITE |
+ FEC_MMFR_PA(mii_id) | FEC_MMFR_RA(frame_addr) |
FEC_MMFR_TA | FEC_MMFR_DATA(value),
fep->hwp + FEC_MII_DATA);
@@ -1828,6 +1885,7 @@ static int fec_enet_mdio_write(struct mii_bus *bus, int mii_id, int regnum,
ret = -ETIMEDOUT;
}
+out:
pm_runtime_mark_last_busy(dev);
pm_runtime_put_autosuspend(dev);
diff --git a/drivers/net/ethernet/freescale/gianfar.c b/drivers/net/ethernet/freescale/gianfar.c
index 7ea19e1..24bf7f6 100644
--- a/drivers/net/ethernet/freescale/gianfar.c
+++ b/drivers/net/ethernet/freescale/gianfar.c
@@ -105,43 +105,6 @@
const char gfar_driver_version[] = "2.0";
-static int gfar_enet_open(struct net_device *dev);
-static netdev_tx_t gfar_start_xmit(struct sk_buff *skb, struct net_device *dev);
-static void gfar_reset_task(struct work_struct *work);
-static void gfar_timeout(struct net_device *dev);
-static int gfar_close(struct net_device *dev);
-static void gfar_alloc_rx_buffs(struct gfar_priv_rx_q *rx_queue,
- int alloc_cnt);
-static int gfar_set_mac_address(struct net_device *dev);
-static int gfar_change_mtu(struct net_device *dev, int new_mtu);
-static irqreturn_t gfar_error(int irq, void *dev_id);
-static irqreturn_t gfar_transmit(int irq, void *dev_id);
-static irqreturn_t gfar_interrupt(int irq, void *dev_id);
-static void adjust_link(struct net_device *dev);
-static noinline void gfar_update_link_state(struct gfar_private *priv);
-static int init_phy(struct net_device *dev);
-static int gfar_probe(struct platform_device *ofdev);
-static int gfar_remove(struct platform_device *ofdev);
-static void free_skb_resources(struct gfar_private *priv);
-static void gfar_set_multi(struct net_device *dev);
-static void gfar_set_hash_for_addr(struct net_device *dev, u8 *addr);
-static void gfar_configure_serdes(struct net_device *dev);
-static int gfar_poll_rx(struct napi_struct *napi, int budget);
-static int gfar_poll_tx(struct napi_struct *napi, int budget);
-static int gfar_poll_rx_sq(struct napi_struct *napi, int budget);
-static int gfar_poll_tx_sq(struct napi_struct *napi, int budget);
-#ifdef CONFIG_NET_POLL_CONTROLLER
-static void gfar_netpoll(struct net_device *dev);
-#endif
-int gfar_clean_rx_ring(struct gfar_priv_rx_q *rx_queue, int rx_work_limit);
-static void gfar_clean_tx_ring(struct gfar_priv_tx_q *tx_queue);
-static void gfar_process_frame(struct net_device *ndev, struct sk_buff *skb);
-static void gfar_halt_nodisable(struct gfar_private *priv);
-static void gfar_clear_exact_match(struct net_device *dev);
-static void gfar_set_mac_for_addr(struct net_device *dev, int num,
- const u8 *addr);
-static int gfar_ioctl(struct net_device *dev, struct ifreq *rq, int cmd);
-
MODULE_AUTHOR("Freescale Semiconductor, Inc");
MODULE_DESCRIPTION("Gianfar Ethernet Driver");
MODULE_LICENSE("GPL");
@@ -162,138 +125,6 @@ static void gfar_init_rxbdp(struct gfar_priv_rx_q *rx_queue, struct rxbd8 *bdp,
bdp->lstatus = cpu_to_be32(lstatus);
}
-static void gfar_init_bds(struct net_device *ndev)
-{
- struct gfar_private *priv = netdev_priv(ndev);
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- struct gfar_priv_tx_q *tx_queue = NULL;
- struct gfar_priv_rx_q *rx_queue = NULL;
- struct txbd8 *txbdp;
- u32 __iomem *rfbptr;
- int i, j;
-
- for (i = 0; i < priv->num_tx_queues; i++) {
- tx_queue = priv->tx_queue[i];
- /* Initialize some variables in our dev structure */
- tx_queue->num_txbdfree = tx_queue->tx_ring_size;
- tx_queue->dirty_tx = tx_queue->tx_bd_base;
- tx_queue->cur_tx = tx_queue->tx_bd_base;
- tx_queue->skb_curtx = 0;
- tx_queue->skb_dirtytx = 0;
-
- /* Initialize Transmit Descriptor Ring */
- txbdp = tx_queue->tx_bd_base;
- for (j = 0; j < tx_queue->tx_ring_size; j++) {
- txbdp->lstatus = 0;
- txbdp->bufPtr = 0;
- txbdp++;
- }
-
- /* Set the last descriptor in the ring to indicate wrap */
- txbdp--;
- txbdp->status = cpu_to_be16(be16_to_cpu(txbdp->status) |
- TXBD_WRAP);
- }
-
- rfbptr = ®s->rfbptr0;
- for (i = 0; i < priv->num_rx_queues; i++) {
- rx_queue = priv->rx_queue[i];
-
- rx_queue->next_to_clean = 0;
- rx_queue->next_to_use = 0;
- rx_queue->next_to_alloc = 0;
-
- /* make sure next_to_clean != next_to_use after this
- * by leaving at least 1 unused descriptor
- */
- gfar_alloc_rx_buffs(rx_queue, gfar_rxbd_unused(rx_queue));
-
- rx_queue->rfbptr = rfbptr;
- rfbptr += 2;
- }
-}
-
-static int gfar_alloc_skb_resources(struct net_device *ndev)
-{
- void *vaddr;
- dma_addr_t addr;
- int i, j;
- struct gfar_private *priv = netdev_priv(ndev);
- struct device *dev = priv->dev;
- struct gfar_priv_tx_q *tx_queue = NULL;
- struct gfar_priv_rx_q *rx_queue = NULL;
-
- priv->total_tx_ring_size = 0;
- for (i = 0; i < priv->num_tx_queues; i++)
- priv->total_tx_ring_size += priv->tx_queue[i]->tx_ring_size;
-
- priv->total_rx_ring_size = 0;
- for (i = 0; i < priv->num_rx_queues; i++)
- priv->total_rx_ring_size += priv->rx_queue[i]->rx_ring_size;
-
- /* Allocate memory for the buffer descriptors */
- vaddr = dma_alloc_coherent(dev,
- (priv->total_tx_ring_size *
- sizeof(struct txbd8)) +
- (priv->total_rx_ring_size *
- sizeof(struct rxbd8)),
- &addr, GFP_KERNEL);
- if (!vaddr)
- return -ENOMEM;
-
- for (i = 0; i < priv->num_tx_queues; i++) {
- tx_queue = priv->tx_queue[i];
- tx_queue->tx_bd_base = vaddr;
- tx_queue->tx_bd_dma_base = addr;
- tx_queue->dev = ndev;
- /* enet DMA only understands physical addresses */
- addr += sizeof(struct txbd8) * tx_queue->tx_ring_size;
- vaddr += sizeof(struct txbd8) * tx_queue->tx_ring_size;
- }
-
- /* Start the rx descriptor ring where the tx ring leaves off */
- for (i = 0; i < priv->num_rx_queues; i++) {
- rx_queue = priv->rx_queue[i];
- rx_queue->rx_bd_base = vaddr;
- rx_queue->rx_bd_dma_base = addr;
- rx_queue->ndev = ndev;
- rx_queue->dev = dev;
- addr += sizeof(struct rxbd8) * rx_queue->rx_ring_size;
- vaddr += sizeof(struct rxbd8) * rx_queue->rx_ring_size;
- }
-
- /* Setup the skbuff rings */
- for (i = 0; i < priv->num_tx_queues; i++) {
- tx_queue = priv->tx_queue[i];
- tx_queue->tx_skbuff =
- kmalloc_array(tx_queue->tx_ring_size,
- sizeof(*tx_queue->tx_skbuff),
- GFP_KERNEL);
- if (!tx_queue->tx_skbuff)
- goto cleanup;
-
- for (j = 0; j < tx_queue->tx_ring_size; j++)
- tx_queue->tx_skbuff[j] = NULL;
- }
-
- for (i = 0; i < priv->num_rx_queues; i++) {
- rx_queue = priv->rx_queue[i];
- rx_queue->rx_buff = kcalloc(rx_queue->rx_ring_size,
- sizeof(*rx_queue->rx_buff),
- GFP_KERNEL);
- if (!rx_queue->rx_buff)
- goto cleanup;
- }
-
- gfar_init_bds(ndev);
-
- return 0;
-
-cleanup:
- free_skb_resources(priv);
- return -ENOMEM;
-}
-
static void gfar_init_tx_rx_base(struct gfar_private *priv)
{
struct gfar __iomem *regs = priv->gfargrp[0].regs;
@@ -444,7 +275,7 @@ static void gfar_configure_coalescing(struct gfar_private *priv,
}
}
-void gfar_configure_coalescing_all(struct gfar_private *priv)
+static void gfar_configure_coalescing_all(struct gfar_private *priv)
{
gfar_configure_coalescing(priv, 0xFF, 0xFF);
}
@@ -477,6 +308,62 @@ static struct net_device_stats *gfar_get_stats(struct net_device *dev)
return &dev->stats;
}
+/* Set the appropriate hash bit for the given addr */
+/* The algorithm works like so:
+ * 1) Take the Destination Address (ie the multicast address), and
+ * do a CRC on it (little endian), and reverse the bits of the
+ * result.
+ * 2) Use the 8 most significant bits as a hash into a 256-entry
+ * table. The table is controlled through 8 32-bit registers:
+ * gaddr0-7. gaddr0's MSB is entry 0, and gaddr7's LSB is
+ * gaddr7. This means that the 3 most significant bits in the
+ * hash index which gaddr register to use, and the 5 other bits
+ * indicate which bit (assuming an IBM numbering scheme, which
+ * for PowerPC (tm) is usually the case) in the register holds
+ * the entry.
+ */
+static void gfar_set_hash_for_addr(struct net_device *dev, u8 *addr)
+{
+ u32 tempval;
+ struct gfar_private *priv = netdev_priv(dev);
+ u32 result = ether_crc(ETH_ALEN, addr);
+ int width = priv->hash_width;
+ u8 whichbit = (result >> (32 - width)) & 0x1f;
+ u8 whichreg = result >> (32 - width + 5);
+ u32 value = (1 << (31-whichbit));
+
+ tempval = gfar_read(priv->hash_regs[whichreg]);
+ tempval |= value;
+ gfar_write(priv->hash_regs[whichreg], tempval);
+}
+
+/* There are multiple MAC Address register pairs on some controllers
+ * This function sets the numth pair to a given address
+ */
+static void gfar_set_mac_for_addr(struct net_device *dev, int num,
+ const u8 *addr)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ u32 tempval;
+ u32 __iomem *macptr = ®s->macstnaddr1;
+
+ macptr += num*2;
+
+ /* For a station address of 0x12345678ABCD in transmission
+ * order (BE), MACnADDR1 is set to 0xCDAB7856 and
+ * MACnADDR2 is set to 0x34120000.
+ */
+ tempval = (addr[5] << 24) | (addr[4] << 16) |
+ (addr[3] << 8) | addr[2];
+
+ gfar_write(macptr, tempval);
+
+ tempval = (addr[1] << 24) | (addr[0] << 16);
+
+ gfar_write(macptr+1, tempval);
+}
+
static int gfar_set_mac_addr(struct net_device *dev, void *p)
{
eth_mac_addr(dev, p);
@@ -486,24 +373,6 @@ static int gfar_set_mac_addr(struct net_device *dev, void *p)
return 0;
}
-static const struct net_device_ops gfar_netdev_ops = {
- .ndo_open = gfar_enet_open,
- .ndo_start_xmit = gfar_start_xmit,
- .ndo_stop = gfar_close,
- .ndo_change_mtu = gfar_change_mtu,
- .ndo_set_features = gfar_set_features,
- .ndo_set_rx_mode = gfar_set_multi,
- .ndo_tx_timeout = gfar_timeout,
- .ndo_do_ioctl = gfar_ioctl,
- .ndo_get_stats = gfar_get_stats,
- .ndo_change_carrier = fixed_phy_change_carrier,
- .ndo_set_mac_address = gfar_set_mac_addr,
- .ndo_validate_addr = eth_validate_addr,
-#ifdef CONFIG_NET_POLL_CONTROLLER
- .ndo_poll_controller = gfar_netpoll,
-#endif
-};
-
static void gfar_ints_disable(struct gfar_private *priv)
{
int i;
@@ -723,10 +592,53 @@ static int gfar_of_group_count(struct device_node *np)
return num;
}
+/* Reads the controller's registers to determine what interface
+ * connects it to the PHY.
+ */
+static phy_interface_t gfar_get_interface(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ u32 ecntrl;
+
+ ecntrl = gfar_read(®s->ecntrl);
+
+ if (ecntrl & ECNTRL_SGMII_MODE)
+ return PHY_INTERFACE_MODE_SGMII;
+
+ if (ecntrl & ECNTRL_TBI_MODE) {
+ if (ecntrl & ECNTRL_REDUCED_MODE)
+ return PHY_INTERFACE_MODE_RTBI;
+ else
+ return PHY_INTERFACE_MODE_TBI;
+ }
+
+ if (ecntrl & ECNTRL_REDUCED_MODE) {
+ if (ecntrl & ECNTRL_REDUCED_MII_MODE) {
+ return PHY_INTERFACE_MODE_RMII;
+ }
+ else {
+ phy_interface_t interface = priv->interface;
+
+ /* This isn't autodetected right now, so it must
+ * be set by the device tree or platform code.
+ */
+ if (interface == PHY_INTERFACE_MODE_RGMII_ID)
+ return PHY_INTERFACE_MODE_RGMII_ID;
+
+ return PHY_INTERFACE_MODE_RGMII;
+ }
+ }
+
+ if (priv->device_flags & FSL_GIANFAR_DEV_HAS_GIGABIT)
+ return PHY_INTERFACE_MODE_GMII;
+
+ return PHY_INTERFACE_MODE_MII;
+}
+
static int gfar_of_init(struct platform_device *ofdev, struct net_device **pdev)
{
const char *model;
- const char *ctype;
const void *mac_addr;
int err = 0, i;
struct net_device *dev = NULL;
@@ -889,13 +801,15 @@ static int gfar_of_init(struct platform_device *ofdev, struct net_device **pdev)
FSL_GIANFAR_DEV_HAS_TIMER |
FSL_GIANFAR_DEV_HAS_RX_FILER;
- err = of_property_read_string(np, "phy-connection-type", &ctype);
-
- /* We only care about rgmii-id. The rest are autodetected */
- if (err == 0 && !strcmp(ctype, "rgmii-id"))
- priv->interface = PHY_INTERFACE_MODE_RGMII_ID;
+ /* Use PHY connection type from the DT node if one is specified there.
+ * rgmii-id really needs to be specified. Other types can be
+ * detected by hardware
+ */
+ err = of_get_phy_mode(np);
+ if (err >= 0)
+ priv->interface = err;
else
- priv->interface = PHY_INTERFACE_MODE_MII;
+ priv->interface = gfar_get_interface(dev);
if (of_find_property(np, "fsl,magic-packet", NULL))
priv->device_flags |= FSL_GIANFAR_DEV_HAS_MAGIC_PACKET;
@@ -931,85 +845,6 @@ static int gfar_of_init(struct platform_device *ofdev, struct net_device **pdev)
return err;
}
-static int gfar_hwtstamp_set(struct net_device *netdev, struct ifreq *ifr)
-{
- struct hwtstamp_config config;
- struct gfar_private *priv = netdev_priv(netdev);
-
- if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
- return -EFAULT;
-
- /* reserved for future extensions */
- if (config.flags)
- return -EINVAL;
-
- switch (config.tx_type) {
- case HWTSTAMP_TX_OFF:
- priv->hwts_tx_en = 0;
- break;
- case HWTSTAMP_TX_ON:
- if (!(priv->device_flags & FSL_GIANFAR_DEV_HAS_TIMER))
- return -ERANGE;
- priv->hwts_tx_en = 1;
- break;
- default:
- return -ERANGE;
- }
-
- switch (config.rx_filter) {
- case HWTSTAMP_FILTER_NONE:
- if (priv->hwts_rx_en) {
- priv->hwts_rx_en = 0;
- reset_gfar(netdev);
- }
- break;
- default:
- if (!(priv->device_flags & FSL_GIANFAR_DEV_HAS_TIMER))
- return -ERANGE;
- if (!priv->hwts_rx_en) {
- priv->hwts_rx_en = 1;
- reset_gfar(netdev);
- }
- config.rx_filter = HWTSTAMP_FILTER_ALL;
- break;
- }
-
- return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ?
- -EFAULT : 0;
-}
-
-static int gfar_hwtstamp_get(struct net_device *netdev, struct ifreq *ifr)
-{
- struct hwtstamp_config config;
- struct gfar_private *priv = netdev_priv(netdev);
-
- config.flags = 0;
- config.tx_type = priv->hwts_tx_en ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
- config.rx_filter = (priv->hwts_rx_en ?
- HWTSTAMP_FILTER_ALL : HWTSTAMP_FILTER_NONE);
-
- return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ?
- -EFAULT : 0;
-}
-
-static int gfar_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
-{
- struct phy_device *phydev = dev->phydev;
-
- if (!netif_running(dev))
- return -EINVAL;
-
- if (cmd == SIOCSHWTSTAMP)
- return gfar_hwtstamp_set(dev, rq);
- if (cmd == SIOCGHWTSTAMP)
- return gfar_hwtstamp_get(dev, rq);
-
- if (!phydev)
- return -ENODEV;
-
- return phy_mii_ioctl(phydev, rq, cmd);
-}
-
static u32 cluster_entry_per_class(struct gfar_private *priv, u32 rqfar,
u32 class)
{
@@ -1133,6 +968,2183 @@ static void gfar_detect_errata(struct gfar_private *priv)
priv->errata);
}
+static void gfar_init_addr_hash_table(struct gfar_private *priv)
+{
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+
+ if (priv->device_flags & FSL_GIANFAR_DEV_HAS_EXTENDED_HASH) {
+ priv->extended_hash = 1;
+ priv->hash_width = 9;
+
+ priv->hash_regs[0] = ®s->igaddr0;
+ priv->hash_regs[1] = ®s->igaddr1;
+ priv->hash_regs[2] = ®s->igaddr2;
+ priv->hash_regs[3] = ®s->igaddr3;
+ priv->hash_regs[4] = ®s->igaddr4;
+ priv->hash_regs[5] = ®s->igaddr5;
+ priv->hash_regs[6] = ®s->igaddr6;
+ priv->hash_regs[7] = ®s->igaddr7;
+ priv->hash_regs[8] = ®s->gaddr0;
+ priv->hash_regs[9] = ®s->gaddr1;
+ priv->hash_regs[10] = ®s->gaddr2;
+ priv->hash_regs[11] = ®s->gaddr3;
+ priv->hash_regs[12] = ®s->gaddr4;
+ priv->hash_regs[13] = ®s->gaddr5;
+ priv->hash_regs[14] = ®s->gaddr6;
+ priv->hash_regs[15] = ®s->gaddr7;
+
+ } else {
+ priv->extended_hash = 0;
+ priv->hash_width = 8;
+
+ priv->hash_regs[0] = ®s->gaddr0;
+ priv->hash_regs[1] = ®s->gaddr1;
+ priv->hash_regs[2] = ®s->gaddr2;
+ priv->hash_regs[3] = ®s->gaddr3;
+ priv->hash_regs[4] = ®s->gaddr4;
+ priv->hash_regs[5] = ®s->gaddr5;
+ priv->hash_regs[6] = ®s->gaddr6;
+ priv->hash_regs[7] = ®s->gaddr7;
+ }
+}
+
+static int __gfar_is_rx_idle(struct gfar_private *priv)
+{
+ u32 res;
+
+ /* Normaly TSEC should not hang on GRS commands, so we should
+ * actually wait for IEVENT_GRSC flag.
+ */
+ if (!gfar_has_errata(priv, GFAR_ERRATA_A002))
+ return 0;
+
+ /* Read the eTSEC register at offset 0xD1C. If bits 7-14 are
+ * the same as bits 23-30, the eTSEC Rx is assumed to be idle
+ * and the Rx can be safely reset.
+ */
+ res = gfar_read((void __iomem *)priv->gfargrp[0].regs + 0xd1c);
+ res &= 0x7f807f80;
+ if ((res & 0xffff) == (res >> 16))
+ return 1;
+
+ return 0;
+}
+
+/* Halt the receive and transmit queues */
+static void gfar_halt_nodisable(struct gfar_private *priv)
+{
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ u32 tempval;
+ unsigned int timeout;
+ int stopped;
+
+ gfar_ints_disable(priv);
+
+ if (gfar_is_dma_stopped(priv))
+ return;
+
+ /* Stop the DMA, and wait for it to stop */
+ tempval = gfar_read(®s->dmactrl);
+ tempval |= (DMACTRL_GRS | DMACTRL_GTS);
+ gfar_write(®s->dmactrl, tempval);
+
+retry:
+ timeout = 1000;
+ while (!(stopped = gfar_is_dma_stopped(priv)) && timeout) {
+ cpu_relax();
+ timeout--;
+ }
+
+ if (!timeout)
+ stopped = gfar_is_dma_stopped(priv);
+
+ if (!stopped && !gfar_is_rx_dma_stopped(priv) &&
+ !__gfar_is_rx_idle(priv))
+ goto retry;
+}
+
+/* Halt the receive and transmit queues */
+static void gfar_halt(struct gfar_private *priv)
+{
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ u32 tempval;
+
+ /* Dissable the Rx/Tx hw queues */
+ gfar_write(®s->rqueue, 0);
+ gfar_write(®s->tqueue, 0);
+
+ mdelay(10);
+
+ gfar_halt_nodisable(priv);
+
+ /* Disable Rx/Tx DMA */
+ tempval = gfar_read(®s->maccfg1);
+ tempval &= ~(MACCFG1_RX_EN | MACCFG1_TX_EN);
+ gfar_write(®s->maccfg1, tempval);
+}
+
+static void free_skb_tx_queue(struct gfar_priv_tx_q *tx_queue)
+{
+ struct txbd8 *txbdp;
+ struct gfar_private *priv = netdev_priv(tx_queue->dev);
+ int i, j;
+
+ txbdp = tx_queue->tx_bd_base;
+
+ for (i = 0; i < tx_queue->tx_ring_size; i++) {
+ if (!tx_queue->tx_skbuff[i])
+ continue;
+
+ dma_unmap_single(priv->dev, be32_to_cpu(txbdp->bufPtr),
+ be16_to_cpu(txbdp->length), DMA_TO_DEVICE);
+ txbdp->lstatus = 0;
+ for (j = 0; j < skb_shinfo(tx_queue->tx_skbuff[i])->nr_frags;
+ j++) {
+ txbdp++;
+ dma_unmap_page(priv->dev, be32_to_cpu(txbdp->bufPtr),
+ be16_to_cpu(txbdp->length),
+ DMA_TO_DEVICE);
+ }
+ txbdp++;
+ dev_kfree_skb_any(tx_queue->tx_skbuff[i]);
+ tx_queue->tx_skbuff[i] = NULL;
+ }
+ kfree(tx_queue->tx_skbuff);
+ tx_queue->tx_skbuff = NULL;
+}
+
+static void free_skb_rx_queue(struct gfar_priv_rx_q *rx_queue)
+{
+ int i;
+
+ struct rxbd8 *rxbdp = rx_queue->rx_bd_base;
+
+ dev_kfree_skb(rx_queue->skb);
+
+ for (i = 0; i < rx_queue->rx_ring_size; i++) {
+ struct gfar_rx_buff *rxb = &rx_queue->rx_buff[i];
+
+ rxbdp->lstatus = 0;
+ rxbdp->bufPtr = 0;
+ rxbdp++;
+
+ if (!rxb->page)
+ continue;
+
+ dma_unmap_page(rx_queue->dev, rxb->dma,
+ PAGE_SIZE, DMA_FROM_DEVICE);
+ __free_page(rxb->page);
+
+ rxb->page = NULL;
+ }
+
+ kfree(rx_queue->rx_buff);
+ rx_queue->rx_buff = NULL;
+}
+
+/* If there are any tx skbs or rx skbs still around, free them.
+ * Then free tx_skbuff and rx_skbuff
+ */
+static void free_skb_resources(struct gfar_private *priv)
+{
+ struct gfar_priv_tx_q *tx_queue = NULL;
+ struct gfar_priv_rx_q *rx_queue = NULL;
+ int i;
+
+ /* Go through all the buffer descriptors and free their data buffers */
+ for (i = 0; i < priv->num_tx_queues; i++) {
+ struct netdev_queue *txq;
+
+ tx_queue = priv->tx_queue[i];
+ txq = netdev_get_tx_queue(tx_queue->dev, tx_queue->qindex);
+ if (tx_queue->tx_skbuff)
+ free_skb_tx_queue(tx_queue);
+ netdev_tx_reset_queue(txq);
+ }
+
+ for (i = 0; i < priv->num_rx_queues; i++) {
+ rx_queue = priv->rx_queue[i];
+ if (rx_queue->rx_buff)
+ free_skb_rx_queue(rx_queue);
+ }
+
+ dma_free_coherent(priv->dev,
+ sizeof(struct txbd8) * priv->total_tx_ring_size +
+ sizeof(struct rxbd8) * priv->total_rx_ring_size,
+ priv->tx_queue[0]->tx_bd_base,
+ priv->tx_queue[0]->tx_bd_dma_base);
+}
+
+void stop_gfar(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+
+ netif_tx_stop_all_queues(dev);
+
+ smp_mb__before_atomic();
+ set_bit(GFAR_DOWN, &priv->state);
+ smp_mb__after_atomic();
+
+ disable_napi(priv);
+
+ /* disable ints and gracefully shut down Rx/Tx DMA */
+ gfar_halt(priv);
+
+ phy_stop(dev->phydev);
+
+ free_skb_resources(priv);
+}
+
+static void gfar_start(struct gfar_private *priv)
+{
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ u32 tempval;
+ int i = 0;
+
+ /* Enable Rx/Tx hw queues */
+ gfar_write(®s->rqueue, priv->rqueue);
+ gfar_write(®s->tqueue, priv->tqueue);
+
+ /* Initialize DMACTRL to have WWR and WOP */
+ tempval = gfar_read(®s->dmactrl);
+ tempval |= DMACTRL_INIT_SETTINGS;
+ gfar_write(®s->dmactrl, tempval);
+
+ /* Make sure we aren't stopped */
+ tempval = gfar_read(®s->dmactrl);
+ tempval &= ~(DMACTRL_GRS | DMACTRL_GTS);
+ gfar_write(®s->dmactrl, tempval);
+
+ for (i = 0; i < priv->num_grps; i++) {
+ regs = priv->gfargrp[i].regs;
+ /* Clear THLT/RHLT, so that the DMA starts polling now */
+ gfar_write(®s->tstat, priv->gfargrp[i].tstat);
+ gfar_write(®s->rstat, priv->gfargrp[i].rstat);
+ }
+
+ /* Enable Rx/Tx DMA */
+ tempval = gfar_read(®s->maccfg1);
+ tempval |= (MACCFG1_RX_EN | MACCFG1_TX_EN);
+ gfar_write(®s->maccfg1, tempval);
+
+ gfar_ints_enable(priv);
+
+ netif_trans_update(priv->ndev); /* prevent tx timeout */
+}
+
+static bool gfar_new_page(struct gfar_priv_rx_q *rxq, struct gfar_rx_buff *rxb)
+{
+ struct page *page;
+ dma_addr_t addr;
+
+ page = dev_alloc_page();
+ if (unlikely(!page))
+ return false;
+
+ addr = dma_map_page(rxq->dev, page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(rxq->dev, addr))) {
+ __free_page(page);
+
+ return false;
+ }
+
+ rxb->dma = addr;
+ rxb->page = page;
+ rxb->page_offset = 0;
+
+ return true;
+}
+
+static void gfar_rx_alloc_err(struct gfar_priv_rx_q *rx_queue)
+{
+ struct gfar_private *priv = netdev_priv(rx_queue->ndev);
+ struct gfar_extra_stats *estats = &priv->extra_stats;
+
+ netdev_err(rx_queue->ndev, "Can't alloc RX buffers\n");
+ atomic64_inc(&estats->rx_alloc_err);
+}
+
+static void gfar_alloc_rx_buffs(struct gfar_priv_rx_q *rx_queue,
+ int alloc_cnt)
+{
+ struct rxbd8 *bdp;
+ struct gfar_rx_buff *rxb;
+ int i;
+
+ i = rx_queue->next_to_use;
+ bdp = &rx_queue->rx_bd_base[i];
+ rxb = &rx_queue->rx_buff[i];
+
+ while (alloc_cnt--) {
+ /* try reuse page */
+ if (unlikely(!rxb->page)) {
+ if (unlikely(!gfar_new_page(rx_queue, rxb))) {
+ gfar_rx_alloc_err(rx_queue);
+ break;
+ }
+ }
+
+ /* Setup the new RxBD */
+ gfar_init_rxbdp(rx_queue, bdp,
+ rxb->dma + rxb->page_offset + RXBUF_ALIGNMENT);
+
+ /* Update to the next pointer */
+ bdp++;
+ rxb++;
+
+ if (unlikely(++i == rx_queue->rx_ring_size)) {
+ i = 0;
+ bdp = rx_queue->rx_bd_base;
+ rxb = rx_queue->rx_buff;
+ }
+ }
+
+ rx_queue->next_to_use = i;
+ rx_queue->next_to_alloc = i;
+}
+
+static void gfar_init_bds(struct net_device *ndev)
+{
+ struct gfar_private *priv = netdev_priv(ndev);
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ struct gfar_priv_tx_q *tx_queue = NULL;
+ struct gfar_priv_rx_q *rx_queue = NULL;
+ struct txbd8 *txbdp;
+ u32 __iomem *rfbptr;
+ int i, j;
+
+ for (i = 0; i < priv->num_tx_queues; i++) {
+ tx_queue = priv->tx_queue[i];
+ /* Initialize some variables in our dev structure */
+ tx_queue->num_txbdfree = tx_queue->tx_ring_size;
+ tx_queue->dirty_tx = tx_queue->tx_bd_base;
+ tx_queue->cur_tx = tx_queue->tx_bd_base;
+ tx_queue->skb_curtx = 0;
+ tx_queue->skb_dirtytx = 0;
+
+ /* Initialize Transmit Descriptor Ring */
+ txbdp = tx_queue->tx_bd_base;
+ for (j = 0; j < tx_queue->tx_ring_size; j++) {
+ txbdp->lstatus = 0;
+ txbdp->bufPtr = 0;
+ txbdp++;
+ }
+
+ /* Set the last descriptor in the ring to indicate wrap */
+ txbdp--;
+ txbdp->status = cpu_to_be16(be16_to_cpu(txbdp->status) |
+ TXBD_WRAP);
+ }
+
+ rfbptr = ®s->rfbptr0;
+ for (i = 0; i < priv->num_rx_queues; i++) {
+ rx_queue = priv->rx_queue[i];
+
+ rx_queue->next_to_clean = 0;
+ rx_queue->next_to_use = 0;
+ rx_queue->next_to_alloc = 0;
+
+ /* make sure next_to_clean != next_to_use after this
+ * by leaving at least 1 unused descriptor
+ */
+ gfar_alloc_rx_buffs(rx_queue, gfar_rxbd_unused(rx_queue));
+
+ rx_queue->rfbptr = rfbptr;
+ rfbptr += 2;
+ }
+}
+
+static int gfar_alloc_skb_resources(struct net_device *ndev)
+{
+ void *vaddr;
+ dma_addr_t addr;
+ int i, j;
+ struct gfar_private *priv = netdev_priv(ndev);
+ struct device *dev = priv->dev;
+ struct gfar_priv_tx_q *tx_queue = NULL;
+ struct gfar_priv_rx_q *rx_queue = NULL;
+
+ priv->total_tx_ring_size = 0;
+ for (i = 0; i < priv->num_tx_queues; i++)
+ priv->total_tx_ring_size += priv->tx_queue[i]->tx_ring_size;
+
+ priv->total_rx_ring_size = 0;
+ for (i = 0; i < priv->num_rx_queues; i++)
+ priv->total_rx_ring_size += priv->rx_queue[i]->rx_ring_size;
+
+ /* Allocate memory for the buffer descriptors */
+ vaddr = dma_alloc_coherent(dev,
+ (priv->total_tx_ring_size *
+ sizeof(struct txbd8)) +
+ (priv->total_rx_ring_size *
+ sizeof(struct rxbd8)),
+ &addr, GFP_KERNEL);
+ if (!vaddr)
+ return -ENOMEM;
+
+ for (i = 0; i < priv->num_tx_queues; i++) {
+ tx_queue = priv->tx_queue[i];
+ tx_queue->tx_bd_base = vaddr;
+ tx_queue->tx_bd_dma_base = addr;
+ tx_queue->dev = ndev;
+ /* enet DMA only understands physical addresses */
+ addr += sizeof(struct txbd8) * tx_queue->tx_ring_size;
+ vaddr += sizeof(struct txbd8) * tx_queue->tx_ring_size;
+ }
+
+ /* Start the rx descriptor ring where the tx ring leaves off */
+ for (i = 0; i < priv->num_rx_queues; i++) {
+ rx_queue = priv->rx_queue[i];
+ rx_queue->rx_bd_base = vaddr;
+ rx_queue->rx_bd_dma_base = addr;
+ rx_queue->ndev = ndev;
+ rx_queue->dev = dev;
+ addr += sizeof(struct rxbd8) * rx_queue->rx_ring_size;
+ vaddr += sizeof(struct rxbd8) * rx_queue->rx_ring_size;
+ }
+
+ /* Setup the skbuff rings */
+ for (i = 0; i < priv->num_tx_queues; i++) {
+ tx_queue = priv->tx_queue[i];
+ tx_queue->tx_skbuff =
+ kmalloc_array(tx_queue->tx_ring_size,
+ sizeof(*tx_queue->tx_skbuff),
+ GFP_KERNEL);
+ if (!tx_queue->tx_skbuff)
+ goto cleanup;
+
+ for (j = 0; j < tx_queue->tx_ring_size; j++)
+ tx_queue->tx_skbuff[j] = NULL;
+ }
+
+ for (i = 0; i < priv->num_rx_queues; i++) {
+ rx_queue = priv->rx_queue[i];
+ rx_queue->rx_buff = kcalloc(rx_queue->rx_ring_size,
+ sizeof(*rx_queue->rx_buff),
+ GFP_KERNEL);
+ if (!rx_queue->rx_buff)
+ goto cleanup;
+ }
+
+ gfar_init_bds(ndev);
+
+ return 0;
+
+cleanup:
+ free_skb_resources(priv);
+ return -ENOMEM;
+}
+
+/* Bring the controller up and running */
+int startup_gfar(struct net_device *ndev)
+{
+ struct gfar_private *priv = netdev_priv(ndev);
+ int err;
+
+ gfar_mac_reset(priv);
+
+ err = gfar_alloc_skb_resources(ndev);
+ if (err)
+ return err;
+
+ gfar_init_tx_rx_base(priv);
+
+ smp_mb__before_atomic();
+ clear_bit(GFAR_DOWN, &priv->state);
+ smp_mb__after_atomic();
+
+ /* Start Rx/Tx DMA and enable the interrupts */
+ gfar_start(priv);
+
+ /* force link state update after mac reset */
+ priv->oldlink = 0;
+ priv->oldspeed = 0;
+ priv->oldduplex = -1;
+
+ phy_start(ndev->phydev);
+
+ enable_napi(priv);
+
+ netif_tx_wake_all_queues(ndev);
+
+ return 0;
+}
+
+static u32 gfar_get_flowctrl_cfg(struct gfar_private *priv)
+{
+ struct net_device *ndev = priv->ndev;
+ struct phy_device *phydev = ndev->phydev;
+ u32 val = 0;
+
+ if (!phydev->duplex)
+ return val;
+
+ if (!priv->pause_aneg_en) {
+ if (priv->tx_pause_en)
+ val |= MACCFG1_TX_FLOW;
+ if (priv->rx_pause_en)
+ val |= MACCFG1_RX_FLOW;
+ } else {
+ u16 lcl_adv, rmt_adv;
+ u8 flowctrl;
+ /* get link partner capabilities */
+ rmt_adv = 0;
+ if (phydev->pause)
+ rmt_adv = LPA_PAUSE_CAP;
+ if (phydev->asym_pause)
+ rmt_adv |= LPA_PAUSE_ASYM;
+
+ lcl_adv = linkmode_adv_to_lcl_adv_t(phydev->advertising);
+ flowctrl = mii_resolve_flowctrl_fdx(lcl_adv, rmt_adv);
+ if (flowctrl & FLOW_CTRL_TX)
+ val |= MACCFG1_TX_FLOW;
+ if (flowctrl & FLOW_CTRL_RX)
+ val |= MACCFG1_RX_FLOW;
+ }
+
+ return val;
+}
+
+static noinline void gfar_update_link_state(struct gfar_private *priv)
+{
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ struct net_device *ndev = priv->ndev;
+ struct phy_device *phydev = ndev->phydev;
+ struct gfar_priv_rx_q *rx_queue = NULL;
+ int i;
+
+ if (unlikely(test_bit(GFAR_RESETTING, &priv->state)))
+ return;
+
+ if (phydev->link) {
+ u32 tempval1 = gfar_read(®s->maccfg1);
+ u32 tempval = gfar_read(®s->maccfg2);
+ u32 ecntrl = gfar_read(®s->ecntrl);
+ u32 tx_flow_oldval = (tempval1 & MACCFG1_TX_FLOW);
+
+ if (phydev->duplex != priv->oldduplex) {
+ if (!(phydev->duplex))
+ tempval &= ~(MACCFG2_FULL_DUPLEX);
+ else
+ tempval |= MACCFG2_FULL_DUPLEX;
+
+ priv->oldduplex = phydev->duplex;
+ }
+
+ if (phydev->speed != priv->oldspeed) {
+ switch (phydev->speed) {
+ case 1000:
+ tempval =
+ ((tempval & ~(MACCFG2_IF)) | MACCFG2_GMII);
+
+ ecntrl &= ~(ECNTRL_R100);
+ break;
+ case 100:
+ case 10:
+ tempval =
+ ((tempval & ~(MACCFG2_IF)) | MACCFG2_MII);
+
+ /* Reduced mode distinguishes
+ * between 10 and 100
+ */
+ if (phydev->speed == SPEED_100)
+ ecntrl |= ECNTRL_R100;
+ else
+ ecntrl &= ~(ECNTRL_R100);
+ break;
+ default:
+ netif_warn(priv, link, priv->ndev,
+ "Ack! Speed (%d) is not 10/100/1000!\n",
+ phydev->speed);
+ break;
+ }
+
+ priv->oldspeed = phydev->speed;
+ }
+
+ tempval1 &= ~(MACCFG1_TX_FLOW | MACCFG1_RX_FLOW);
+ tempval1 |= gfar_get_flowctrl_cfg(priv);
+
+ /* Turn last free buffer recording on */
+ if ((tempval1 & MACCFG1_TX_FLOW) && !tx_flow_oldval) {
+ for (i = 0; i < priv->num_rx_queues; i++) {
+ u32 bdp_dma;
+
+ rx_queue = priv->rx_queue[i];
+ bdp_dma = gfar_rxbd_dma_lastfree(rx_queue);
+ gfar_write(rx_queue->rfbptr, bdp_dma);
+ }
+
+ priv->tx_actual_en = 1;
+ }
+
+ if (unlikely(!(tempval1 & MACCFG1_TX_FLOW) && tx_flow_oldval))
+ priv->tx_actual_en = 0;
+
+ gfar_write(®s->maccfg1, tempval1);
+ gfar_write(®s->maccfg2, tempval);
+ gfar_write(®s->ecntrl, ecntrl);
+
+ if (!priv->oldlink)
+ priv->oldlink = 1;
+
+ } else if (priv->oldlink) {
+ priv->oldlink = 0;
+ priv->oldspeed = 0;
+ priv->oldduplex = -1;
+ }
+
+ if (netif_msg_link(priv))
+ phy_print_status(phydev);
+}
+
+/* Called every time the controller might need to be made
+ * aware of new link state. The PHY code conveys this
+ * information through variables in the phydev structure, and this
+ * function converts those variables into the appropriate
+ * register values, and can bring down the device if needed.
+ */
+static void adjust_link(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ struct phy_device *phydev = dev->phydev;
+
+ if (unlikely(phydev->link != priv->oldlink ||
+ (phydev->link && (phydev->duplex != priv->oldduplex ||
+ phydev->speed != priv->oldspeed))))
+ gfar_update_link_state(priv);
+}
+
+/* Initialize TBI PHY interface for communicating with the
+ * SERDES lynx PHY on the chip. We communicate with this PHY
+ * through the MDIO bus on each controller, treating it as a
+ * "normal" PHY at the address found in the TBIPA register. We assume
+ * that the TBIPA register is valid. Either the MDIO bus code will set
+ * it to a value that doesn't conflict with other PHYs on the bus, or the
+ * value doesn't matter, as there are no other PHYs on the bus.
+ */
+static void gfar_configure_serdes(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ struct phy_device *tbiphy;
+
+ if (!priv->tbi_node) {
+ dev_warn(&dev->dev, "error: SGMII mode requires that the "
+ "device tree specify a tbi-handle\n");
+ return;
+ }
+
+ tbiphy = of_phy_find_device(priv->tbi_node);
+ if (!tbiphy) {
+ dev_err(&dev->dev, "error: Could not get TBI device\n");
+ return;
+ }
+
+ /* If the link is already up, we must already be ok, and don't need to
+ * configure and reset the TBI<->SerDes link. Maybe U-Boot configured
+ * everything for us? Resetting it takes the link down and requires
+ * several seconds for it to come back.
+ */
+ if (phy_read(tbiphy, MII_BMSR) & BMSR_LSTATUS) {
+ put_device(&tbiphy->mdio.dev);
+ return;
+ }
+
+ /* Single clk mode, mii mode off(for serdes communication) */
+ phy_write(tbiphy, MII_TBICON, TBICON_CLK_SELECT);
+
+ phy_write(tbiphy, MII_ADVERTISE,
+ ADVERTISE_1000XFULL | ADVERTISE_1000XPAUSE |
+ ADVERTISE_1000XPSE_ASYM);
+
+ phy_write(tbiphy, MII_BMCR,
+ BMCR_ANENABLE | BMCR_ANRESTART | BMCR_FULLDPLX |
+ BMCR_SPEED1000);
+
+ put_device(&tbiphy->mdio.dev);
+}
+
+/* Initializes driver's PHY state, and attaches to the PHY.
+ * Returns 0 on success.
+ */
+static int init_phy(struct net_device *dev)
+{
+ __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
+ struct gfar_private *priv = netdev_priv(dev);
+ phy_interface_t interface = priv->interface;
+ struct phy_device *phydev;
+ struct ethtool_eee edata;
+
+ linkmode_set_bit_array(phy_10_100_features_array,
+ ARRAY_SIZE(phy_10_100_features_array),
+ mask);
+ linkmode_set_bit(ETHTOOL_LINK_MODE_Autoneg_BIT, mask);
+ linkmode_set_bit(ETHTOOL_LINK_MODE_MII_BIT, mask);
+ if (priv->device_flags & FSL_GIANFAR_DEV_HAS_GIGABIT)
+ linkmode_set_bit(ETHTOOL_LINK_MODE_1000baseT_Full_BIT, mask);
+
+ priv->oldlink = 0;
+ priv->oldspeed = 0;
+ priv->oldduplex = -1;
+
+ phydev = of_phy_connect(dev, priv->phy_node, &adjust_link, 0,
+ interface);
+ if (!phydev) {
+ dev_err(&dev->dev, "could not attach to PHY\n");
+ return -ENODEV;
+ }
+
+ if (interface == PHY_INTERFACE_MODE_SGMII)
+ gfar_configure_serdes(dev);
+
+ /* Remove any features not supported by the controller */
+ linkmode_and(phydev->supported, phydev->supported, mask);
+ linkmode_copy(phydev->advertising, phydev->supported);
+
+ /* Add support for flow control */
+ phy_support_asym_pause(phydev);
+
+ /* disable EEE autoneg, EEE not supported by eTSEC */
+ memset(&edata, 0, sizeof(struct ethtool_eee));
+ phy_ethtool_set_eee(phydev, &edata);
+
+ return 0;
+}
+
+static inline struct txfcb *gfar_add_fcb(struct sk_buff *skb)
+{
+ struct txfcb *fcb = skb_push(skb, GMAC_FCB_LEN);
+
+ memset(fcb, 0, GMAC_FCB_LEN);
+
+ return fcb;
+}
+
+static inline void gfar_tx_checksum(struct sk_buff *skb, struct txfcb *fcb,
+ int fcb_length)
+{
+ /* If we're here, it's a IP packet with a TCP or UDP
+ * payload. We set it to checksum, using a pseudo-header
+ * we provide
+ */
+ u8 flags = TXFCB_DEFAULT;
+
+ /* Tell the controller what the protocol is
+ * And provide the already calculated phcs
+ */
+ if (ip_hdr(skb)->protocol == IPPROTO_UDP) {
+ flags |= TXFCB_UDP;
+ fcb->phcs = (__force __be16)(udp_hdr(skb)->check);
+ } else
+ fcb->phcs = (__force __be16)(tcp_hdr(skb)->check);
+
+ /* l3os is the distance between the start of the
+ * frame (skb->data) and the start of the IP hdr.
+ * l4os is the distance between the start of the
+ * l3 hdr and the l4 hdr
+ */
+ fcb->l3os = (u8)(skb_network_offset(skb) - fcb_length);
+ fcb->l4os = skb_network_header_len(skb);
+
+ fcb->flags = flags;
+}
+
+static inline void gfar_tx_vlan(struct sk_buff *skb, struct txfcb *fcb)
+{
+ fcb->flags |= TXFCB_VLN;
+ fcb->vlctl = cpu_to_be16(skb_vlan_tag_get(skb));
+}
+
+static inline struct txbd8 *skip_txbd(struct txbd8 *bdp, int stride,
+ struct txbd8 *base, int ring_size)
+{
+ struct txbd8 *new_bd = bdp + stride;
+
+ return (new_bd >= (base + ring_size)) ? (new_bd - ring_size) : new_bd;
+}
+
+static inline struct txbd8 *next_txbd(struct txbd8 *bdp, struct txbd8 *base,
+ int ring_size)
+{
+ return skip_txbd(bdp, 1, base, ring_size);
+}
+
+/* eTSEC12: csum generation not supported for some fcb offsets */
+static inline bool gfar_csum_errata_12(struct gfar_private *priv,
+ unsigned long fcb_addr)
+{
+ return (gfar_has_errata(priv, GFAR_ERRATA_12) &&
+ (fcb_addr % 0x20) > 0x18);
+}
+
+/* eTSEC76: csum generation for frames larger than 2500 may
+ * cause excess delays before start of transmission
+ */
+static inline bool gfar_csum_errata_76(struct gfar_private *priv,
+ unsigned int len)
+{
+ return (gfar_has_errata(priv, GFAR_ERRATA_76) &&
+ (len > 2500));
+}
+
+/* This is called by the kernel when a frame is ready for transmission.
+ * It is pointed to by the dev->hard_start_xmit function pointer
+ */
+static netdev_tx_t gfar_start_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ struct gfar_priv_tx_q *tx_queue = NULL;
+ struct netdev_queue *txq;
+ struct gfar __iomem *regs = NULL;
+ struct txfcb *fcb = NULL;
+ struct txbd8 *txbdp, *txbdp_start, *base, *txbdp_tstamp = NULL;
+ u32 lstatus;
+ skb_frag_t *frag;
+ int i, rq = 0;
+ int do_tstamp, do_csum, do_vlan;
+ u32 bufaddr;
+ unsigned int nr_frags, nr_txbds, bytes_sent, fcb_len = 0;
+
+ rq = skb->queue_mapping;
+ tx_queue = priv->tx_queue[rq];
+ txq = netdev_get_tx_queue(dev, rq);
+ base = tx_queue->tx_bd_base;
+ regs = tx_queue->grp->regs;
+
+ do_csum = (CHECKSUM_PARTIAL == skb->ip_summed);
+ do_vlan = skb_vlan_tag_present(skb);
+ do_tstamp = (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
+ priv->hwts_tx_en;
+
+ if (do_csum || do_vlan)
+ fcb_len = GMAC_FCB_LEN;
+
+ /* check if time stamp should be generated */
+ if (unlikely(do_tstamp))
+ fcb_len = GMAC_FCB_LEN + GMAC_TXPAL_LEN;
+
+ /* make space for additional header when fcb is needed */
+ if (fcb_len && unlikely(skb_headroom(skb) < fcb_len)) {
+ struct sk_buff *skb_new;
+
+ skb_new = skb_realloc_headroom(skb, fcb_len);
+ if (!skb_new) {
+ dev->stats.tx_errors++;
+ dev_kfree_skb_any(skb);
+ return NETDEV_TX_OK;
+ }
+
+ if (skb->sk)
+ skb_set_owner_w(skb_new, skb->sk);
+ dev_consume_skb_any(skb);
+ skb = skb_new;
+ }
+
+ /* total number of fragments in the SKB */
+ nr_frags = skb_shinfo(skb)->nr_frags;
+
+ /* calculate the required number of TxBDs for this skb */
+ if (unlikely(do_tstamp))
+ nr_txbds = nr_frags + 2;
+ else
+ nr_txbds = nr_frags + 1;
+
+ /* check if there is space to queue this packet */
+ if (nr_txbds > tx_queue->num_txbdfree) {
+ /* no space, stop the queue */
+ netif_tx_stop_queue(txq);
+ dev->stats.tx_fifo_errors++;
+ return NETDEV_TX_BUSY;
+ }
+
+ /* Update transmit stats */
+ bytes_sent = skb->len;
+ tx_queue->stats.tx_bytes += bytes_sent;
+ /* keep Tx bytes on wire for BQL accounting */
+ GFAR_CB(skb)->bytes_sent = bytes_sent;
+ tx_queue->stats.tx_packets++;
+
+ txbdp = txbdp_start = tx_queue->cur_tx;
+ lstatus = be32_to_cpu(txbdp->lstatus);
+
+ /* Add TxPAL between FCB and frame if required */
+ if (unlikely(do_tstamp)) {
+ skb_push(skb, GMAC_TXPAL_LEN);
+ memset(skb->data, 0, GMAC_TXPAL_LEN);
+ }
+
+ /* Add TxFCB if required */
+ if (fcb_len) {
+ fcb = gfar_add_fcb(skb);
+ lstatus |= BD_LFLAG(TXBD_TOE);
+ }
+
+ /* Set up checksumming */
+ if (do_csum) {
+ gfar_tx_checksum(skb, fcb, fcb_len);
+
+ if (unlikely(gfar_csum_errata_12(priv, (unsigned long)fcb)) ||
+ unlikely(gfar_csum_errata_76(priv, skb->len))) {
+ __skb_pull(skb, GMAC_FCB_LEN);
+ skb_checksum_help(skb);
+ if (do_vlan || do_tstamp) {
+ /* put back a new fcb for vlan/tstamp TOE */
+ fcb = gfar_add_fcb(skb);
+ } else {
+ /* Tx TOE not used */
+ lstatus &= ~(BD_LFLAG(TXBD_TOE));
+ fcb = NULL;
+ }
+ }
+ }
+
+ if (do_vlan)
+ gfar_tx_vlan(skb, fcb);
+
+ bufaddr = dma_map_single(priv->dev, skb->data, skb_headlen(skb),
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(priv->dev, bufaddr)))
+ goto dma_map_err;
+
+ txbdp_start->bufPtr = cpu_to_be32(bufaddr);
+
+ /* Time stamp insertion requires one additional TxBD */
+ if (unlikely(do_tstamp))
+ txbdp_tstamp = txbdp = next_txbd(txbdp, base,
+ tx_queue->tx_ring_size);
+
+ if (likely(!nr_frags)) {
+ if (likely(!do_tstamp))
+ lstatus |= BD_LFLAG(TXBD_LAST | TXBD_INTERRUPT);
+ } else {
+ u32 lstatus_start = lstatus;
+
+ /* Place the fragment addresses and lengths into the TxBDs */
+ frag = &skb_shinfo(skb)->frags[0];
+ for (i = 0; i < nr_frags; i++, frag++) {
+ unsigned int size;
+
+ /* Point at the next BD, wrapping as needed */
+ txbdp = next_txbd(txbdp, base, tx_queue->tx_ring_size);
+
+ size = skb_frag_size(frag);
+
+ lstatus = be32_to_cpu(txbdp->lstatus) | size |
+ BD_LFLAG(TXBD_READY);
+
+ /* Handle the last BD specially */
+ if (i == nr_frags - 1)
+ lstatus |= BD_LFLAG(TXBD_LAST | TXBD_INTERRUPT);
+
+ bufaddr = skb_frag_dma_map(priv->dev, frag, 0,
+ size, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(priv->dev, bufaddr)))
+ goto dma_map_err;
+
+ /* set the TxBD length and buffer pointer */
+ txbdp->bufPtr = cpu_to_be32(bufaddr);
+ txbdp->lstatus = cpu_to_be32(lstatus);
+ }
+
+ lstatus = lstatus_start;
+ }
+
+ /* If time stamping is requested one additional TxBD must be set up. The
+ * first TxBD points to the FCB and must have a data length of
+ * GMAC_FCB_LEN. The second TxBD points to the actual frame data with
+ * the full frame length.
+ */
+ if (unlikely(do_tstamp)) {
+ u32 lstatus_ts = be32_to_cpu(txbdp_tstamp->lstatus);
+
+ bufaddr = be32_to_cpu(txbdp_start->bufPtr);
+ bufaddr += fcb_len;
+
+ lstatus_ts |= BD_LFLAG(TXBD_READY) |
+ (skb_headlen(skb) - fcb_len);
+ if (!nr_frags)
+ lstatus_ts |= BD_LFLAG(TXBD_LAST | TXBD_INTERRUPT);
+
+ txbdp_tstamp->bufPtr = cpu_to_be32(bufaddr);
+ txbdp_tstamp->lstatus = cpu_to_be32(lstatus_ts);
+ lstatus |= BD_LFLAG(TXBD_CRC | TXBD_READY) | GMAC_FCB_LEN;
+
+ /* Setup tx hardware time stamping */
+ skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+ fcb->ptp = 1;
+ } else {
+ lstatus |= BD_LFLAG(TXBD_CRC | TXBD_READY) | skb_headlen(skb);
+ }
+
+ netdev_tx_sent_queue(txq, bytes_sent);
+
+ gfar_wmb();
+
+ txbdp_start->lstatus = cpu_to_be32(lstatus);
+
+ gfar_wmb(); /* force lstatus write before tx_skbuff */
+
+ tx_queue->tx_skbuff[tx_queue->skb_curtx] = skb;
+
+ /* Update the current skb pointer to the next entry we will use
+ * (wrapping if necessary)
+ */
+ tx_queue->skb_curtx = (tx_queue->skb_curtx + 1) &
+ TX_RING_MOD_MASK(tx_queue->tx_ring_size);
+
+ tx_queue->cur_tx = next_txbd(txbdp, base, tx_queue->tx_ring_size);
+
+ /* We can work in parallel with gfar_clean_tx_ring(), except
+ * when modifying num_txbdfree. Note that we didn't grab the lock
+ * when we were reading the num_txbdfree and checking for available
+ * space, that's because outside of this function it can only grow.
+ */
+ spin_lock_bh(&tx_queue->txlock);
+ /* reduce TxBD free count */
+ tx_queue->num_txbdfree -= (nr_txbds);
+ spin_unlock_bh(&tx_queue->txlock);
+
+ /* If the next BD still needs to be cleaned up, then the bds
+ * are full. We need to tell the kernel to stop sending us stuff.
+ */
+ if (!tx_queue->num_txbdfree) {
+ netif_tx_stop_queue(txq);
+
+ dev->stats.tx_fifo_errors++;
+ }
+
+ /* Tell the DMA to go go go */
+ gfar_write(®s->tstat, TSTAT_CLEAR_THALT >> tx_queue->qindex);
+
+ return NETDEV_TX_OK;
+
+dma_map_err:
+ txbdp = next_txbd(txbdp_start, base, tx_queue->tx_ring_size);
+ if (do_tstamp)
+ txbdp = next_txbd(txbdp, base, tx_queue->tx_ring_size);
+ for (i = 0; i < nr_frags; i++) {
+ lstatus = be32_to_cpu(txbdp->lstatus);
+ if (!(lstatus & BD_LFLAG(TXBD_READY)))
+ break;
+
+ lstatus &= ~BD_LFLAG(TXBD_READY);
+ txbdp->lstatus = cpu_to_be32(lstatus);
+ bufaddr = be32_to_cpu(txbdp->bufPtr);
+ dma_unmap_page(priv->dev, bufaddr, be16_to_cpu(txbdp->length),
+ DMA_TO_DEVICE);
+ txbdp = next_txbd(txbdp, base, tx_queue->tx_ring_size);
+ }
+ gfar_wmb();
+ dev_kfree_skb_any(skb);
+ return NETDEV_TX_OK;
+}
+
+/* Changes the mac address if the controller is not running. */
+static int gfar_set_mac_address(struct net_device *dev)
+{
+ gfar_set_mac_for_addr(dev, 0, dev->dev_addr);
+
+ return 0;
+}
+
+static int gfar_change_mtu(struct net_device *dev, int new_mtu)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+
+ while (test_and_set_bit_lock(GFAR_RESETTING, &priv->state))
+ cpu_relax();
+
+ if (dev->flags & IFF_UP)
+ stop_gfar(dev);
+
+ dev->mtu = new_mtu;
+
+ if (dev->flags & IFF_UP)
+ startup_gfar(dev);
+
+ clear_bit_unlock(GFAR_RESETTING, &priv->state);
+
+ return 0;
+}
+
+void reset_gfar(struct net_device *ndev)
+{
+ struct gfar_private *priv = netdev_priv(ndev);
+
+ while (test_and_set_bit_lock(GFAR_RESETTING, &priv->state))
+ cpu_relax();
+
+ stop_gfar(ndev);
+ startup_gfar(ndev);
+
+ clear_bit_unlock(GFAR_RESETTING, &priv->state);
+}
+
+/* gfar_reset_task gets scheduled when a packet has not been
+ * transmitted after a set amount of time.
+ * For now, assume that clearing out all the structures, and
+ * starting over will fix the problem.
+ */
+static void gfar_reset_task(struct work_struct *work)
+{
+ struct gfar_private *priv = container_of(work, struct gfar_private,
+ reset_task);
+ reset_gfar(priv->ndev);
+}
+
+static void gfar_timeout(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+
+ dev->stats.tx_errors++;
+ schedule_work(&priv->reset_task);
+}
+
+static int gfar_hwtstamp_set(struct net_device *netdev, struct ifreq *ifr)
+{
+ struct hwtstamp_config config;
+ struct gfar_private *priv = netdev_priv(netdev);
+
+ if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
+ return -EFAULT;
+
+ /* reserved for future extensions */
+ if (config.flags)
+ return -EINVAL;
+
+ switch (config.tx_type) {
+ case HWTSTAMP_TX_OFF:
+ priv->hwts_tx_en = 0;
+ break;
+ case HWTSTAMP_TX_ON:
+ if (!(priv->device_flags & FSL_GIANFAR_DEV_HAS_TIMER))
+ return -ERANGE;
+ priv->hwts_tx_en = 1;
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ switch (config.rx_filter) {
+ case HWTSTAMP_FILTER_NONE:
+ if (priv->hwts_rx_en) {
+ priv->hwts_rx_en = 0;
+ reset_gfar(netdev);
+ }
+ break;
+ default:
+ if (!(priv->device_flags & FSL_GIANFAR_DEV_HAS_TIMER))
+ return -ERANGE;
+ if (!priv->hwts_rx_en) {
+ priv->hwts_rx_en = 1;
+ reset_gfar(netdev);
+ }
+ config.rx_filter = HWTSTAMP_FILTER_ALL;
+ break;
+ }
+
+ return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ?
+ -EFAULT : 0;
+}
+
+static int gfar_hwtstamp_get(struct net_device *netdev, struct ifreq *ifr)
+{
+ struct hwtstamp_config config;
+ struct gfar_private *priv = netdev_priv(netdev);
+
+ config.flags = 0;
+ config.tx_type = priv->hwts_tx_en ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
+ config.rx_filter = (priv->hwts_rx_en ?
+ HWTSTAMP_FILTER_ALL : HWTSTAMP_FILTER_NONE);
+
+ return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ?
+ -EFAULT : 0;
+}
+
+static int gfar_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
+{
+ struct phy_device *phydev = dev->phydev;
+
+ if (!netif_running(dev))
+ return -EINVAL;
+
+ if (cmd == SIOCSHWTSTAMP)
+ return gfar_hwtstamp_set(dev, rq);
+ if (cmd == SIOCGHWTSTAMP)
+ return gfar_hwtstamp_get(dev, rq);
+
+ if (!phydev)
+ return -ENODEV;
+
+ return phy_mii_ioctl(phydev, rq, cmd);
+}
+
+/* Interrupt Handler for Transmit complete */
+static void gfar_clean_tx_ring(struct gfar_priv_tx_q *tx_queue)
+{
+ struct net_device *dev = tx_queue->dev;
+ struct netdev_queue *txq;
+ struct gfar_private *priv = netdev_priv(dev);
+ struct txbd8 *bdp, *next = NULL;
+ struct txbd8 *lbdp = NULL;
+ struct txbd8 *base = tx_queue->tx_bd_base;
+ struct sk_buff *skb;
+ int skb_dirtytx;
+ int tx_ring_size = tx_queue->tx_ring_size;
+ int frags = 0, nr_txbds = 0;
+ int i;
+ int howmany = 0;
+ int tqi = tx_queue->qindex;
+ unsigned int bytes_sent = 0;
+ u32 lstatus;
+ size_t buflen;
+
+ txq = netdev_get_tx_queue(dev, tqi);
+ bdp = tx_queue->dirty_tx;
+ skb_dirtytx = tx_queue->skb_dirtytx;
+
+ while ((skb = tx_queue->tx_skbuff[skb_dirtytx])) {
+
+ frags = skb_shinfo(skb)->nr_frags;
+
+ /* When time stamping, one additional TxBD must be freed.
+ * Also, we need to dma_unmap_single() the TxPAL.
+ */
+ if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS))
+ nr_txbds = frags + 2;
+ else
+ nr_txbds = frags + 1;
+
+ lbdp = skip_txbd(bdp, nr_txbds - 1, base, tx_ring_size);
+
+ lstatus = be32_to_cpu(lbdp->lstatus);
+
+ /* Only clean completed frames */
+ if ((lstatus & BD_LFLAG(TXBD_READY)) &&
+ (lstatus & BD_LENGTH_MASK))
+ break;
+
+ if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS)) {
+ next = next_txbd(bdp, base, tx_ring_size);
+ buflen = be16_to_cpu(next->length) +
+ GMAC_FCB_LEN + GMAC_TXPAL_LEN;
+ } else
+ buflen = be16_to_cpu(bdp->length);
+
+ dma_unmap_single(priv->dev, be32_to_cpu(bdp->bufPtr),
+ buflen, DMA_TO_DEVICE);
+
+ if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS)) {
+ struct skb_shared_hwtstamps shhwtstamps;
+ u64 *ns = (u64 *)(((uintptr_t)skb->data + 0x10) &
+ ~0x7UL);
+
+ memset(&shhwtstamps, 0, sizeof(shhwtstamps));
+ shhwtstamps.hwtstamp = ns_to_ktime(be64_to_cpu(*ns));
+ skb_pull(skb, GMAC_FCB_LEN + GMAC_TXPAL_LEN);
+ skb_tstamp_tx(skb, &shhwtstamps);
+ gfar_clear_txbd_status(bdp);
+ bdp = next;
+ }
+
+ gfar_clear_txbd_status(bdp);
+ bdp = next_txbd(bdp, base, tx_ring_size);
+
+ for (i = 0; i < frags; i++) {
+ dma_unmap_page(priv->dev, be32_to_cpu(bdp->bufPtr),
+ be16_to_cpu(bdp->length),
+ DMA_TO_DEVICE);
+ gfar_clear_txbd_status(bdp);
+ bdp = next_txbd(bdp, base, tx_ring_size);
+ }
+
+ bytes_sent += GFAR_CB(skb)->bytes_sent;
+
+ dev_kfree_skb_any(skb);
+
+ tx_queue->tx_skbuff[skb_dirtytx] = NULL;
+
+ skb_dirtytx = (skb_dirtytx + 1) &
+ TX_RING_MOD_MASK(tx_ring_size);
+
+ howmany++;
+ spin_lock(&tx_queue->txlock);
+ tx_queue->num_txbdfree += nr_txbds;
+ spin_unlock(&tx_queue->txlock);
+ }
+
+ /* If we freed a buffer, we can restart transmission, if necessary */
+ if (tx_queue->num_txbdfree &&
+ netif_tx_queue_stopped(txq) &&
+ !(test_bit(GFAR_DOWN, &priv->state)))
+ netif_wake_subqueue(priv->ndev, tqi);
+
+ /* Update dirty indicators */
+ tx_queue->skb_dirtytx = skb_dirtytx;
+ tx_queue->dirty_tx = bdp;
+
+ netdev_tx_completed_queue(txq, howmany, bytes_sent);
+}
+
+static void count_errors(u32 lstatus, struct net_device *ndev)
+{
+ struct gfar_private *priv = netdev_priv(ndev);
+ struct net_device_stats *stats = &ndev->stats;
+ struct gfar_extra_stats *estats = &priv->extra_stats;
+
+ /* If the packet was truncated, none of the other errors matter */
+ if (lstatus & BD_LFLAG(RXBD_TRUNCATED)) {
+ stats->rx_length_errors++;
+
+ atomic64_inc(&estats->rx_trunc);
+
+ return;
+ }
+ /* Count the errors, if there were any */
+ if (lstatus & BD_LFLAG(RXBD_LARGE | RXBD_SHORT)) {
+ stats->rx_length_errors++;
+
+ if (lstatus & BD_LFLAG(RXBD_LARGE))
+ atomic64_inc(&estats->rx_large);
+ else
+ atomic64_inc(&estats->rx_short);
+ }
+ if (lstatus & BD_LFLAG(RXBD_NONOCTET)) {
+ stats->rx_frame_errors++;
+ atomic64_inc(&estats->rx_nonoctet);
+ }
+ if (lstatus & BD_LFLAG(RXBD_CRCERR)) {
+ atomic64_inc(&estats->rx_crcerr);
+ stats->rx_crc_errors++;
+ }
+ if (lstatus & BD_LFLAG(RXBD_OVERRUN)) {
+ atomic64_inc(&estats->rx_overrun);
+ stats->rx_over_errors++;
+ }
+}
+
+static irqreturn_t gfar_receive(int irq, void *grp_id)
+{
+ struct gfar_priv_grp *grp = (struct gfar_priv_grp *)grp_id;
+ unsigned long flags;
+ u32 imask, ievent;
+
+ ievent = gfar_read(&grp->regs->ievent);
+
+ if (unlikely(ievent & IEVENT_FGPI)) {
+ gfar_write(&grp->regs->ievent, IEVENT_FGPI);
+ return IRQ_HANDLED;
+ }
+
+ if (likely(napi_schedule_prep(&grp->napi_rx))) {
+ spin_lock_irqsave(&grp->grplock, flags);
+ imask = gfar_read(&grp->regs->imask);
+ imask &= IMASK_RX_DISABLED;
+ gfar_write(&grp->regs->imask, imask);
+ spin_unlock_irqrestore(&grp->grplock, flags);
+ __napi_schedule(&grp->napi_rx);
+ } else {
+ /* Clear IEVENT, so interrupts aren't called again
+ * because of the packets that have already arrived.
+ */
+ gfar_write(&grp->regs->ievent, IEVENT_RX_MASK);
+ }
+
+ return IRQ_HANDLED;
+}
+
+/* Interrupt Handler for Transmit complete */
+static irqreturn_t gfar_transmit(int irq, void *grp_id)
+{
+ struct gfar_priv_grp *grp = (struct gfar_priv_grp *)grp_id;
+ unsigned long flags;
+ u32 imask;
+
+ if (likely(napi_schedule_prep(&grp->napi_tx))) {
+ spin_lock_irqsave(&grp->grplock, flags);
+ imask = gfar_read(&grp->regs->imask);
+ imask &= IMASK_TX_DISABLED;
+ gfar_write(&grp->regs->imask, imask);
+ spin_unlock_irqrestore(&grp->grplock, flags);
+ __napi_schedule(&grp->napi_tx);
+ } else {
+ /* Clear IEVENT, so interrupts aren't called again
+ * because of the packets that have already arrived.
+ */
+ gfar_write(&grp->regs->ievent, IEVENT_TX_MASK);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static bool gfar_add_rx_frag(struct gfar_rx_buff *rxb, u32 lstatus,
+ struct sk_buff *skb, bool first)
+{
+ int size = lstatus & BD_LENGTH_MASK;
+ struct page *page = rxb->page;
+
+ if (likely(first)) {
+ skb_put(skb, size);
+ } else {
+ /* the last fragments' length contains the full frame length */
+ if (lstatus & BD_LFLAG(RXBD_LAST))
+ size -= skb->len;
+
+ skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
+ rxb->page_offset + RXBUF_ALIGNMENT,
+ size, GFAR_RXB_TRUESIZE);
+ }
+
+ /* try reuse page */
+ if (unlikely(page_count(page) != 1 || page_is_pfmemalloc(page)))
+ return false;
+
+ /* change offset to the other half */
+ rxb->page_offset ^= GFAR_RXB_TRUESIZE;
+
+ page_ref_inc(page);
+
+ return true;
+}
+
+static void gfar_reuse_rx_page(struct gfar_priv_rx_q *rxq,
+ struct gfar_rx_buff *old_rxb)
+{
+ struct gfar_rx_buff *new_rxb;
+ u16 nta = rxq->next_to_alloc;
+
+ new_rxb = &rxq->rx_buff[nta];
+
+ /* find next buf that can reuse a page */
+ nta++;
+ rxq->next_to_alloc = (nta < rxq->rx_ring_size) ? nta : 0;
+
+ /* copy page reference */
+ *new_rxb = *old_rxb;
+
+ /* sync for use by the device */
+ dma_sync_single_range_for_device(rxq->dev, old_rxb->dma,
+ old_rxb->page_offset,
+ GFAR_RXB_TRUESIZE, DMA_FROM_DEVICE);
+}
+
+static struct sk_buff *gfar_get_next_rxbuff(struct gfar_priv_rx_q *rx_queue,
+ u32 lstatus, struct sk_buff *skb)
+{
+ struct gfar_rx_buff *rxb = &rx_queue->rx_buff[rx_queue->next_to_clean];
+ struct page *page = rxb->page;
+ bool first = false;
+
+ if (likely(!skb)) {
+ void *buff_addr = page_address(page) + rxb->page_offset;
+
+ skb = build_skb(buff_addr, GFAR_SKBFRAG_SIZE);
+ if (unlikely(!skb)) {
+ gfar_rx_alloc_err(rx_queue);
+ return NULL;
+ }
+ skb_reserve(skb, RXBUF_ALIGNMENT);
+ first = true;
+ }
+
+ dma_sync_single_range_for_cpu(rx_queue->dev, rxb->dma, rxb->page_offset,
+ GFAR_RXB_TRUESIZE, DMA_FROM_DEVICE);
+
+ if (gfar_add_rx_frag(rxb, lstatus, skb, first)) {
+ /* reuse the free half of the page */
+ gfar_reuse_rx_page(rx_queue, rxb);
+ } else {
+ /* page cannot be reused, unmap it */
+ dma_unmap_page(rx_queue->dev, rxb->dma,
+ PAGE_SIZE, DMA_FROM_DEVICE);
+ }
+
+ /* clear rxb content */
+ rxb->page = NULL;
+
+ return skb;
+}
+
+static inline void gfar_rx_checksum(struct sk_buff *skb, struct rxfcb *fcb)
+{
+ /* If valid headers were found, and valid sums
+ * were verified, then we tell the kernel that no
+ * checksumming is necessary. Otherwise, it is [FIXME]
+ */
+ if ((be16_to_cpu(fcb->flags) & RXFCB_CSUM_MASK) ==
+ (RXFCB_CIP | RXFCB_CTU))
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ else
+ skb_checksum_none_assert(skb);
+}
+
+/* gfar_process_frame() -- handle one incoming packet if skb isn't NULL. */
+static void gfar_process_frame(struct net_device *ndev, struct sk_buff *skb)
+{
+ struct gfar_private *priv = netdev_priv(ndev);
+ struct rxfcb *fcb = NULL;
+
+ /* fcb is at the beginning if exists */
+ fcb = (struct rxfcb *)skb->data;
+
+ /* Remove the FCB from the skb
+ * Remove the padded bytes, if there are any
+ */
+ if (priv->uses_rxfcb)
+ skb_pull(skb, GMAC_FCB_LEN);
+
+ /* Get receive timestamp from the skb */
+ if (priv->hwts_rx_en) {
+ struct skb_shared_hwtstamps *shhwtstamps = skb_hwtstamps(skb);
+ u64 *ns = (u64 *) skb->data;
+
+ memset(shhwtstamps, 0, sizeof(*shhwtstamps));
+ shhwtstamps->hwtstamp = ns_to_ktime(be64_to_cpu(*ns));
+ }
+
+ if (priv->padding)
+ skb_pull(skb, priv->padding);
+
+ /* Trim off the FCS */
+ pskb_trim(skb, skb->len - ETH_FCS_LEN);
+
+ if (ndev->features & NETIF_F_RXCSUM)
+ gfar_rx_checksum(skb, fcb);
+
+ /* There's need to check for NETIF_F_HW_VLAN_CTAG_RX here.
+ * Even if vlan rx accel is disabled, on some chips
+ * RXFCB_VLN is pseudo randomly set.
+ */
+ if (ndev->features & NETIF_F_HW_VLAN_CTAG_RX &&
+ be16_to_cpu(fcb->flags) & RXFCB_VLN)
+ __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
+ be16_to_cpu(fcb->vlctl));
+}
+
+/* gfar_clean_rx_ring() -- Processes each frame in the rx ring
+ * until the budget/quota has been reached. Returns the number
+ * of frames handled
+ */
+static int gfar_clean_rx_ring(struct gfar_priv_rx_q *rx_queue,
+ int rx_work_limit)
+{
+ struct net_device *ndev = rx_queue->ndev;
+ struct gfar_private *priv = netdev_priv(ndev);
+ struct rxbd8 *bdp;
+ int i, howmany = 0;
+ struct sk_buff *skb = rx_queue->skb;
+ int cleaned_cnt = gfar_rxbd_unused(rx_queue);
+ unsigned int total_bytes = 0, total_pkts = 0;
+
+ /* Get the first full descriptor */
+ i = rx_queue->next_to_clean;
+
+ while (rx_work_limit--) {
+ u32 lstatus;
+
+ if (cleaned_cnt >= GFAR_RX_BUFF_ALLOC) {
+ gfar_alloc_rx_buffs(rx_queue, cleaned_cnt);
+ cleaned_cnt = 0;
+ }
+
+ bdp = &rx_queue->rx_bd_base[i];
+ lstatus = be32_to_cpu(bdp->lstatus);
+ if (lstatus & BD_LFLAG(RXBD_EMPTY))
+ break;
+
+ /* order rx buffer descriptor reads */
+ rmb();
+
+ /* fetch next to clean buffer from the ring */
+ skb = gfar_get_next_rxbuff(rx_queue, lstatus, skb);
+ if (unlikely(!skb))
+ break;
+
+ cleaned_cnt++;
+ howmany++;
+
+ if (unlikely(++i == rx_queue->rx_ring_size))
+ i = 0;
+
+ rx_queue->next_to_clean = i;
+
+ /* fetch next buffer if not the last in frame */
+ if (!(lstatus & BD_LFLAG(RXBD_LAST)))
+ continue;
+
+ if (unlikely(lstatus & BD_LFLAG(RXBD_ERR))) {
+ count_errors(lstatus, ndev);
+
+ /* discard faulty buffer */
+ dev_kfree_skb(skb);
+ skb = NULL;
+ rx_queue->stats.rx_dropped++;
+ continue;
+ }
+
+ gfar_process_frame(ndev, skb);
+
+ /* Increment the number of packets */
+ total_pkts++;
+ total_bytes += skb->len;
+
+ skb_record_rx_queue(skb, rx_queue->qindex);
+
+ skb->protocol = eth_type_trans(skb, ndev);
+
+ /* Send the packet up the stack */
+ napi_gro_receive(&rx_queue->grp->napi_rx, skb);
+
+ skb = NULL;
+ }
+
+ /* Store incomplete frames for completion */
+ rx_queue->skb = skb;
+
+ rx_queue->stats.rx_packets += total_pkts;
+ rx_queue->stats.rx_bytes += total_bytes;
+
+ if (cleaned_cnt)
+ gfar_alloc_rx_buffs(rx_queue, cleaned_cnt);
+
+ /* Update Last Free RxBD pointer for LFC */
+ if (unlikely(priv->tx_actual_en)) {
+ u32 bdp_dma = gfar_rxbd_dma_lastfree(rx_queue);
+
+ gfar_write(rx_queue->rfbptr, bdp_dma);
+ }
+
+ return howmany;
+}
+
+static int gfar_poll_rx_sq(struct napi_struct *napi, int budget)
+{
+ struct gfar_priv_grp *gfargrp =
+ container_of(napi, struct gfar_priv_grp, napi_rx);
+ struct gfar __iomem *regs = gfargrp->regs;
+ struct gfar_priv_rx_q *rx_queue = gfargrp->rx_queue;
+ int work_done = 0;
+
+ /* Clear IEVENT, so interrupts aren't called again
+ * because of the packets that have already arrived
+ */
+ gfar_write(®s->ievent, IEVENT_RX_MASK);
+
+ work_done = gfar_clean_rx_ring(rx_queue, budget);
+
+ if (work_done < budget) {
+ u32 imask;
+ napi_complete_done(napi, work_done);
+ /* Clear the halt bit in RSTAT */
+ gfar_write(®s->rstat, gfargrp->rstat);
+
+ spin_lock_irq(&gfargrp->grplock);
+ imask = gfar_read(®s->imask);
+ imask |= IMASK_RX_DEFAULT;
+ gfar_write(®s->imask, imask);
+ spin_unlock_irq(&gfargrp->grplock);
+ }
+
+ return work_done;
+}
+
+static int gfar_poll_tx_sq(struct napi_struct *napi, int budget)
+{
+ struct gfar_priv_grp *gfargrp =
+ container_of(napi, struct gfar_priv_grp, napi_tx);
+ struct gfar __iomem *regs = gfargrp->regs;
+ struct gfar_priv_tx_q *tx_queue = gfargrp->tx_queue;
+ u32 imask;
+
+ /* Clear IEVENT, so interrupts aren't called again
+ * because of the packets that have already arrived
+ */
+ gfar_write(®s->ievent, IEVENT_TX_MASK);
+
+ /* run Tx cleanup to completion */
+ if (tx_queue->tx_skbuff[tx_queue->skb_dirtytx])
+ gfar_clean_tx_ring(tx_queue);
+
+ napi_complete(napi);
+
+ spin_lock_irq(&gfargrp->grplock);
+ imask = gfar_read(®s->imask);
+ imask |= IMASK_TX_DEFAULT;
+ gfar_write(®s->imask, imask);
+ spin_unlock_irq(&gfargrp->grplock);
+
+ return 0;
+}
+
+static int gfar_poll_rx(struct napi_struct *napi, int budget)
+{
+ struct gfar_priv_grp *gfargrp =
+ container_of(napi, struct gfar_priv_grp, napi_rx);
+ struct gfar_private *priv = gfargrp->priv;
+ struct gfar __iomem *regs = gfargrp->regs;
+ struct gfar_priv_rx_q *rx_queue = NULL;
+ int work_done = 0, work_done_per_q = 0;
+ int i, budget_per_q = 0;
+ unsigned long rstat_rxf;
+ int num_act_queues;
+
+ /* Clear IEVENT, so interrupts aren't called again
+ * because of the packets that have already arrived
+ */
+ gfar_write(®s->ievent, IEVENT_RX_MASK);
+
+ rstat_rxf = gfar_read(®s->rstat) & RSTAT_RXF_MASK;
+
+ num_act_queues = bitmap_weight(&rstat_rxf, MAX_RX_QS);
+ if (num_act_queues)
+ budget_per_q = budget/num_act_queues;
+
+ for_each_set_bit(i, &gfargrp->rx_bit_map, priv->num_rx_queues) {
+ /* skip queue if not active */
+ if (!(rstat_rxf & (RSTAT_CLEAR_RXF0 >> i)))
+ continue;
+
+ rx_queue = priv->rx_queue[i];
+ work_done_per_q =
+ gfar_clean_rx_ring(rx_queue, budget_per_q);
+ work_done += work_done_per_q;
+
+ /* finished processing this queue */
+ if (work_done_per_q < budget_per_q) {
+ /* clear active queue hw indication */
+ gfar_write(®s->rstat,
+ RSTAT_CLEAR_RXF0 >> i);
+ num_act_queues--;
+
+ if (!num_act_queues)
+ break;
+ }
+ }
+
+ if (!num_act_queues) {
+ u32 imask;
+ napi_complete_done(napi, work_done);
+
+ /* Clear the halt bit in RSTAT */
+ gfar_write(®s->rstat, gfargrp->rstat);
+
+ spin_lock_irq(&gfargrp->grplock);
+ imask = gfar_read(®s->imask);
+ imask |= IMASK_RX_DEFAULT;
+ gfar_write(®s->imask, imask);
+ spin_unlock_irq(&gfargrp->grplock);
+ }
+
+ return work_done;
+}
+
+static int gfar_poll_tx(struct napi_struct *napi, int budget)
+{
+ struct gfar_priv_grp *gfargrp =
+ container_of(napi, struct gfar_priv_grp, napi_tx);
+ struct gfar_private *priv = gfargrp->priv;
+ struct gfar __iomem *regs = gfargrp->regs;
+ struct gfar_priv_tx_q *tx_queue = NULL;
+ int has_tx_work = 0;
+ int i;
+
+ /* Clear IEVENT, so interrupts aren't called again
+ * because of the packets that have already arrived
+ */
+ gfar_write(®s->ievent, IEVENT_TX_MASK);
+
+ for_each_set_bit(i, &gfargrp->tx_bit_map, priv->num_tx_queues) {
+ tx_queue = priv->tx_queue[i];
+ /* run Tx cleanup to completion */
+ if (tx_queue->tx_skbuff[tx_queue->skb_dirtytx]) {
+ gfar_clean_tx_ring(tx_queue);
+ has_tx_work = 1;
+ }
+ }
+
+ if (!has_tx_work) {
+ u32 imask;
+ napi_complete(napi);
+
+ spin_lock_irq(&gfargrp->grplock);
+ imask = gfar_read(®s->imask);
+ imask |= IMASK_TX_DEFAULT;
+ gfar_write(®s->imask, imask);
+ spin_unlock_irq(&gfargrp->grplock);
+ }
+
+ return 0;
+}
+
+/* GFAR error interrupt handler */
+static irqreturn_t gfar_error(int irq, void *grp_id)
+{
+ struct gfar_priv_grp *gfargrp = grp_id;
+ struct gfar __iomem *regs = gfargrp->regs;
+ struct gfar_private *priv= gfargrp->priv;
+ struct net_device *dev = priv->ndev;
+
+ /* Save ievent for future reference */
+ u32 events = gfar_read(®s->ievent);
+
+ /* Clear IEVENT */
+ gfar_write(®s->ievent, events & IEVENT_ERR_MASK);
+
+ /* Magic Packet is not an error. */
+ if ((priv->device_flags & FSL_GIANFAR_DEV_HAS_MAGIC_PACKET) &&
+ (events & IEVENT_MAG))
+ events &= ~IEVENT_MAG;
+
+ /* Hmm... */
+ if (netif_msg_rx_err(priv) || netif_msg_tx_err(priv))
+ netdev_dbg(dev,
+ "error interrupt (ievent=0x%08x imask=0x%08x)\n",
+ events, gfar_read(®s->imask));
+
+ /* Update the error counters */
+ if (events & IEVENT_TXE) {
+ dev->stats.tx_errors++;
+
+ if (events & IEVENT_LC)
+ dev->stats.tx_window_errors++;
+ if (events & IEVENT_CRL)
+ dev->stats.tx_aborted_errors++;
+ if (events & IEVENT_XFUN) {
+ netif_dbg(priv, tx_err, dev,
+ "TX FIFO underrun, packet dropped\n");
+ dev->stats.tx_dropped++;
+ atomic64_inc(&priv->extra_stats.tx_underrun);
+
+ schedule_work(&priv->reset_task);
+ }
+ netif_dbg(priv, tx_err, dev, "Transmit Error\n");
+ }
+ if (events & IEVENT_BSY) {
+ dev->stats.rx_over_errors++;
+ atomic64_inc(&priv->extra_stats.rx_bsy);
+
+ netif_dbg(priv, rx_err, dev, "busy error (rstat: %x)\n",
+ gfar_read(®s->rstat));
+ }
+ if (events & IEVENT_BABR) {
+ dev->stats.rx_errors++;
+ atomic64_inc(&priv->extra_stats.rx_babr);
+
+ netif_dbg(priv, rx_err, dev, "babbling RX error\n");
+ }
+ if (events & IEVENT_EBERR) {
+ atomic64_inc(&priv->extra_stats.eberr);
+ netif_dbg(priv, rx_err, dev, "bus error\n");
+ }
+ if (events & IEVENT_RXC)
+ netif_dbg(priv, rx_status, dev, "control frame\n");
+
+ if (events & IEVENT_BABT) {
+ atomic64_inc(&priv->extra_stats.tx_babt);
+ netif_dbg(priv, tx_err, dev, "babbling TX error\n");
+ }
+ return IRQ_HANDLED;
+}
+
+/* The interrupt handler for devices with one interrupt */
+static irqreturn_t gfar_interrupt(int irq, void *grp_id)
+{
+ struct gfar_priv_grp *gfargrp = grp_id;
+
+ /* Save ievent for future reference */
+ u32 events = gfar_read(&gfargrp->regs->ievent);
+
+ /* Check for reception */
+ if (events & IEVENT_RX_MASK)
+ gfar_receive(irq, grp_id);
+
+ /* Check for transmit completion */
+ if (events & IEVENT_TX_MASK)
+ gfar_transmit(irq, grp_id);
+
+ /* Check for errors */
+ if (events & IEVENT_ERR_MASK)
+ gfar_error(irq, grp_id);
+
+ return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_NET_POLL_CONTROLLER
+/* Polling 'interrupt' - used by things like netconsole to send skbs
+ * without having to re-enable interrupts. It's not called while
+ * the interrupt routine is executing.
+ */
+static void gfar_netpoll(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ int i;
+
+ /* If the device has multiple interrupts, run tx/rx */
+ if (priv->device_flags & FSL_GIANFAR_DEV_HAS_MULTI_INTR) {
+ for (i = 0; i < priv->num_grps; i++) {
+ struct gfar_priv_grp *grp = &priv->gfargrp[i];
+
+ disable_irq(gfar_irq(grp, TX)->irq);
+ disable_irq(gfar_irq(grp, RX)->irq);
+ disable_irq(gfar_irq(grp, ER)->irq);
+ gfar_interrupt(gfar_irq(grp, TX)->irq, grp);
+ enable_irq(gfar_irq(grp, ER)->irq);
+ enable_irq(gfar_irq(grp, RX)->irq);
+ enable_irq(gfar_irq(grp, TX)->irq);
+ }
+ } else {
+ for (i = 0; i < priv->num_grps; i++) {
+ struct gfar_priv_grp *grp = &priv->gfargrp[i];
+
+ disable_irq(gfar_irq(grp, TX)->irq);
+ gfar_interrupt(gfar_irq(grp, TX)->irq, grp);
+ enable_irq(gfar_irq(grp, TX)->irq);
+ }
+ }
+}
+#endif
+
+static void free_grp_irqs(struct gfar_priv_grp *grp)
+{
+ free_irq(gfar_irq(grp, TX)->irq, grp);
+ free_irq(gfar_irq(grp, RX)->irq, grp);
+ free_irq(gfar_irq(grp, ER)->irq, grp);
+}
+
+static int register_grp_irqs(struct gfar_priv_grp *grp)
+{
+ struct gfar_private *priv = grp->priv;
+ struct net_device *dev = priv->ndev;
+ int err;
+
+ /* If the device has multiple interrupts, register for
+ * them. Otherwise, only register for the one
+ */
+ if (priv->device_flags & FSL_GIANFAR_DEV_HAS_MULTI_INTR) {
+ /* Install our interrupt handlers for Error,
+ * Transmit, and Receive
+ */
+ err = request_irq(gfar_irq(grp, ER)->irq, gfar_error, 0,
+ gfar_irq(grp, ER)->name, grp);
+ if (err < 0) {
+ netif_err(priv, intr, dev, "Can't get IRQ %d\n",
+ gfar_irq(grp, ER)->irq);
+
+ goto err_irq_fail;
+ }
+ enable_irq_wake(gfar_irq(grp, ER)->irq);
+
+ err = request_irq(gfar_irq(grp, TX)->irq, gfar_transmit, 0,
+ gfar_irq(grp, TX)->name, grp);
+ if (err < 0) {
+ netif_err(priv, intr, dev, "Can't get IRQ %d\n",
+ gfar_irq(grp, TX)->irq);
+ goto tx_irq_fail;
+ }
+ err = request_irq(gfar_irq(grp, RX)->irq, gfar_receive, 0,
+ gfar_irq(grp, RX)->name, grp);
+ if (err < 0) {
+ netif_err(priv, intr, dev, "Can't get IRQ %d\n",
+ gfar_irq(grp, RX)->irq);
+ goto rx_irq_fail;
+ }
+ enable_irq_wake(gfar_irq(grp, RX)->irq);
+
+ } else {
+ err = request_irq(gfar_irq(grp, TX)->irq, gfar_interrupt, 0,
+ gfar_irq(grp, TX)->name, grp);
+ if (err < 0) {
+ netif_err(priv, intr, dev, "Can't get IRQ %d\n",
+ gfar_irq(grp, TX)->irq);
+ goto err_irq_fail;
+ }
+ enable_irq_wake(gfar_irq(grp, TX)->irq);
+ }
+
+ return 0;
+
+rx_irq_fail:
+ free_irq(gfar_irq(grp, TX)->irq, grp);
+tx_irq_fail:
+ free_irq(gfar_irq(grp, ER)->irq, grp);
+err_irq_fail:
+ return err;
+
+}
+
+static void gfar_free_irq(struct gfar_private *priv)
+{
+ int i;
+
+ /* Free the IRQs */
+ if (priv->device_flags & FSL_GIANFAR_DEV_HAS_MULTI_INTR) {
+ for (i = 0; i < priv->num_grps; i++)
+ free_grp_irqs(&priv->gfargrp[i]);
+ } else {
+ for (i = 0; i < priv->num_grps; i++)
+ free_irq(gfar_irq(&priv->gfargrp[i], TX)->irq,
+ &priv->gfargrp[i]);
+ }
+}
+
+static int gfar_request_irq(struct gfar_private *priv)
+{
+ int err, i, j;
+
+ for (i = 0; i < priv->num_grps; i++) {
+ err = register_grp_irqs(&priv->gfargrp[i]);
+ if (err) {
+ for (j = 0; j < i; j++)
+ free_grp_irqs(&priv->gfargrp[j]);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+/* Called when something needs to use the ethernet device
+ * Returns 0 for success.
+ */
+static int gfar_enet_open(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+ int err;
+
+ err = init_phy(dev);
+ if (err)
+ return err;
+
+ err = gfar_request_irq(priv);
+ if (err)
+ return err;
+
+ err = startup_gfar(dev);
+ if (err)
+ return err;
+
+ return err;
+}
+
+/* Stops the kernel queue, and halts the controller */
+static int gfar_close(struct net_device *dev)
+{
+ struct gfar_private *priv = netdev_priv(dev);
+
+ cancel_work_sync(&priv->reset_task);
+ stop_gfar(dev);
+
+ /* Disconnect from the PHY */
+ phy_disconnect(dev->phydev);
+
+ gfar_free_irq(priv);
+
+ return 0;
+}
+
+/* Clears each of the exact match registers to zero, so they
+ * don't interfere with normal reception
+ */
+static void gfar_clear_exact_match(struct net_device *dev)
+{
+ int idx;
+ static const u8 zero_arr[ETH_ALEN] = {0, 0, 0, 0, 0, 0};
+
+ for (idx = 1; idx < GFAR_EM_NUM + 1; idx++)
+ gfar_set_mac_for_addr(dev, idx, zero_arr);
+}
+
+/* Update the hash table based on the current list of multicast
+ * addresses we subscribe to. Also, change the promiscuity of
+ * the device based on the flags (this function is called
+ * whenever dev->flags is changed
+ */
+static void gfar_set_multi(struct net_device *dev)
+{
+ struct netdev_hw_addr *ha;
+ struct gfar_private *priv = netdev_priv(dev);
+ struct gfar __iomem *regs = priv->gfargrp[0].regs;
+ u32 tempval;
+
+ if (dev->flags & IFF_PROMISC) {
+ /* Set RCTRL to PROM */
+ tempval = gfar_read(®s->rctrl);
+ tempval |= RCTRL_PROM;
+ gfar_write(®s->rctrl, tempval);
+ } else {
+ /* Set RCTRL to not PROM */
+ tempval = gfar_read(®s->rctrl);
+ tempval &= ~(RCTRL_PROM);
+ gfar_write(®s->rctrl, tempval);
+ }
+
+ if (dev->flags & IFF_ALLMULTI) {
+ /* Set the hash to rx all multicast frames */
+ gfar_write(®s->igaddr0, 0xffffffff);
+ gfar_write(®s->igaddr1, 0xffffffff);
+ gfar_write(®s->igaddr2, 0xffffffff);
+ gfar_write(®s->igaddr3, 0xffffffff);
+ gfar_write(®s->igaddr4, 0xffffffff);
+ gfar_write(®s->igaddr5, 0xffffffff);
+ gfar_write(®s->igaddr6, 0xffffffff);
+ gfar_write(®s->igaddr7, 0xffffffff);
+ gfar_write(®s->gaddr0, 0xffffffff);
+ gfar_write(®s->gaddr1, 0xffffffff);
+ gfar_write(®s->gaddr2, 0xffffffff);
+ gfar_write(®s->gaddr3, 0xffffffff);
+ gfar_write(®s->gaddr4, 0xffffffff);
+ gfar_write(®s->gaddr5, 0xffffffff);
+ gfar_write(®s->gaddr6, 0xffffffff);
+ gfar_write(®s->gaddr7, 0xffffffff);
+ } else {
+ int em_num;
+ int idx;
+
+ /* zero out the hash */
+ gfar_write(®s->igaddr0, 0x0);
+ gfar_write(®s->igaddr1, 0x0);
+ gfar_write(®s->igaddr2, 0x0);
+ gfar_write(®s->igaddr3, 0x0);
+ gfar_write(®s->igaddr4, 0x0);
+ gfar_write(®s->igaddr5, 0x0);
+ gfar_write(®s->igaddr6, 0x0);
+ gfar_write(®s->igaddr7, 0x0);
+ gfar_write(®s->gaddr0, 0x0);
+ gfar_write(®s->gaddr1, 0x0);
+ gfar_write(®s->gaddr2, 0x0);
+ gfar_write(®s->gaddr3, 0x0);
+ gfar_write(®s->gaddr4, 0x0);
+ gfar_write(®s->gaddr5, 0x0);
+ gfar_write(®s->gaddr6, 0x0);
+ gfar_write(®s->gaddr7, 0x0);
+
+ /* If we have extended hash tables, we need to
+ * clear the exact match registers to prepare for
+ * setting them
+ */
+ if (priv->extended_hash) {
+ em_num = GFAR_EM_NUM + 1;
+ gfar_clear_exact_match(dev);
+ idx = 1;
+ } else {
+ idx = 0;
+ em_num = 0;
+ }
+
+ if (netdev_mc_empty(dev))
+ return;
+
+ /* Parse the list, and set the appropriate bits */
+ netdev_for_each_mc_addr(ha, dev) {
+ if (idx < em_num) {
+ gfar_set_mac_for_addr(dev, idx, ha->addr);
+ idx++;
+ } else
+ gfar_set_hash_for_addr(dev, ha->addr);
+ }
+ }
+}
+
void gfar_mac_reset(struct gfar_private *priv)
{
struct gfar __iomem *regs = priv->gfargrp[0].regs;
@@ -1262,45 +3274,23 @@ static void gfar_hw_init(struct gfar_private *priv)
gfar_write_isrg(priv);
}
-static void gfar_init_addr_hash_table(struct gfar_private *priv)
-{
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
-
- if (priv->device_flags & FSL_GIANFAR_DEV_HAS_EXTENDED_HASH) {
- priv->extended_hash = 1;
- priv->hash_width = 9;
-
- priv->hash_regs[0] = ®s->igaddr0;
- priv->hash_regs[1] = ®s->igaddr1;
- priv->hash_regs[2] = ®s->igaddr2;
- priv->hash_regs[3] = ®s->igaddr3;
- priv->hash_regs[4] = ®s->igaddr4;
- priv->hash_regs[5] = ®s->igaddr5;
- priv->hash_regs[6] = ®s->igaddr6;
- priv->hash_regs[7] = ®s->igaddr7;
- priv->hash_regs[8] = ®s->gaddr0;
- priv->hash_regs[9] = ®s->gaddr1;
- priv->hash_regs[10] = ®s->gaddr2;
- priv->hash_regs[11] = ®s->gaddr3;
- priv->hash_regs[12] = ®s->gaddr4;
- priv->hash_regs[13] = ®s->gaddr5;
- priv->hash_regs[14] = ®s->gaddr6;
- priv->hash_regs[15] = ®s->gaddr7;
-
- } else {
- priv->extended_hash = 0;
- priv->hash_width = 8;
-
- priv->hash_regs[0] = ®s->gaddr0;
- priv->hash_regs[1] = ®s->gaddr1;
- priv->hash_regs[2] = ®s->gaddr2;
- priv->hash_regs[3] = ®s->gaddr3;
- priv->hash_regs[4] = ®s->gaddr4;
- priv->hash_regs[5] = ®s->gaddr5;
- priv->hash_regs[6] = ®s->gaddr6;
- priv->hash_regs[7] = ®s->gaddr7;
- }
-}
+static const struct net_device_ops gfar_netdev_ops = {
+ .ndo_open = gfar_enet_open,
+ .ndo_start_xmit = gfar_start_xmit,
+ .ndo_stop = gfar_close,
+ .ndo_change_mtu = gfar_change_mtu,
+ .ndo_set_features = gfar_set_features,
+ .ndo_set_rx_mode = gfar_set_multi,
+ .ndo_tx_timeout = gfar_timeout,
+ .ndo_do_ioctl = gfar_ioctl,
+ .ndo_get_stats = gfar_get_stats,
+ .ndo_change_carrier = fixed_phy_change_carrier,
+ .ndo_set_mac_address = gfar_set_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = gfar_netpoll,
+#endif
+};
/* Set up the ethernet device structure, private data,
* and anything else we need before we start
@@ -1731,2038 +3721,6 @@ static const struct dev_pm_ops gfar_pm_ops = {
#endif
-/* Reads the controller's registers to determine what interface
- * connects it to the PHY.
- */
-static phy_interface_t gfar_get_interface(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- u32 ecntrl;
-
- ecntrl = gfar_read(®s->ecntrl);
-
- if (ecntrl & ECNTRL_SGMII_MODE)
- return PHY_INTERFACE_MODE_SGMII;
-
- if (ecntrl & ECNTRL_TBI_MODE) {
- if (ecntrl & ECNTRL_REDUCED_MODE)
- return PHY_INTERFACE_MODE_RTBI;
- else
- return PHY_INTERFACE_MODE_TBI;
- }
-
- if (ecntrl & ECNTRL_REDUCED_MODE) {
- if (ecntrl & ECNTRL_REDUCED_MII_MODE) {
- return PHY_INTERFACE_MODE_RMII;
- }
- else {
- phy_interface_t interface = priv->interface;
-
- /* This isn't autodetected right now, so it must
- * be set by the device tree or platform code.
- */
- if (interface == PHY_INTERFACE_MODE_RGMII_ID)
- return PHY_INTERFACE_MODE_RGMII_ID;
-
- return PHY_INTERFACE_MODE_RGMII;
- }
- }
-
- if (priv->device_flags & FSL_GIANFAR_DEV_HAS_GIGABIT)
- return PHY_INTERFACE_MODE_GMII;
-
- return PHY_INTERFACE_MODE_MII;
-}
-
-
-/* Initializes driver's PHY state, and attaches to the PHY.
- * Returns 0 on success.
- */
-static int init_phy(struct net_device *dev)
-{
- __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
- struct gfar_private *priv = netdev_priv(dev);
- phy_interface_t interface;
- struct phy_device *phydev;
- struct ethtool_eee edata;
-
- linkmode_set_bit_array(phy_10_100_features_array,
- ARRAY_SIZE(phy_10_100_features_array),
- mask);
- linkmode_set_bit(ETHTOOL_LINK_MODE_Autoneg_BIT, mask);
- linkmode_set_bit(ETHTOOL_LINK_MODE_MII_BIT, mask);
- if (priv->device_flags & FSL_GIANFAR_DEV_HAS_GIGABIT)
- linkmode_set_bit(ETHTOOL_LINK_MODE_1000baseT_Full_BIT, mask);
-
- priv->oldlink = 0;
- priv->oldspeed = 0;
- priv->oldduplex = -1;
-
- interface = gfar_get_interface(dev);
-
- phydev = of_phy_connect(dev, priv->phy_node, &adjust_link, 0,
- interface);
- if (!phydev) {
- dev_err(&dev->dev, "could not attach to PHY\n");
- return -ENODEV;
- }
-
- if (interface == PHY_INTERFACE_MODE_SGMII)
- gfar_configure_serdes(dev);
-
- /* Remove any features not supported by the controller */
- linkmode_and(phydev->supported, phydev->supported, mask);
- linkmode_copy(phydev->advertising, phydev->supported);
-
- /* Add support for flow control */
- phy_support_asym_pause(phydev);
-
- /* disable EEE autoneg, EEE not supported by eTSEC */
- memset(&edata, 0, sizeof(struct ethtool_eee));
- phy_ethtool_set_eee(phydev, &edata);
-
- return 0;
-}
-
-/* Initialize TBI PHY interface for communicating with the
- * SERDES lynx PHY on the chip. We communicate with this PHY
- * through the MDIO bus on each controller, treating it as a
- * "normal" PHY at the address found in the TBIPA register. We assume
- * that the TBIPA register is valid. Either the MDIO bus code will set
- * it to a value that doesn't conflict with other PHYs on the bus, or the
- * value doesn't matter, as there are no other PHYs on the bus.
- */
-static void gfar_configure_serdes(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
- struct phy_device *tbiphy;
-
- if (!priv->tbi_node) {
- dev_warn(&dev->dev, "error: SGMII mode requires that the "
- "device tree specify a tbi-handle\n");
- return;
- }
-
- tbiphy = of_phy_find_device(priv->tbi_node);
- if (!tbiphy) {
- dev_err(&dev->dev, "error: Could not get TBI device\n");
- return;
- }
-
- /* If the link is already up, we must already be ok, and don't need to
- * configure and reset the TBI<->SerDes link. Maybe U-Boot configured
- * everything for us? Resetting it takes the link down and requires
- * several seconds for it to come back.
- */
- if (phy_read(tbiphy, MII_BMSR) & BMSR_LSTATUS) {
- put_device(&tbiphy->mdio.dev);
- return;
- }
-
- /* Single clk mode, mii mode off(for serdes communication) */
- phy_write(tbiphy, MII_TBICON, TBICON_CLK_SELECT);
-
- phy_write(tbiphy, MII_ADVERTISE,
- ADVERTISE_1000XFULL | ADVERTISE_1000XPAUSE |
- ADVERTISE_1000XPSE_ASYM);
-
- phy_write(tbiphy, MII_BMCR,
- BMCR_ANENABLE | BMCR_ANRESTART | BMCR_FULLDPLX |
- BMCR_SPEED1000);
-
- put_device(&tbiphy->mdio.dev);
-}
-
-static int __gfar_is_rx_idle(struct gfar_private *priv)
-{
- u32 res;
-
- /* Normaly TSEC should not hang on GRS commands, so we should
- * actually wait for IEVENT_GRSC flag.
- */
- if (!gfar_has_errata(priv, GFAR_ERRATA_A002))
- return 0;
-
- /* Read the eTSEC register at offset 0xD1C. If bits 7-14 are
- * the same as bits 23-30, the eTSEC Rx is assumed to be idle
- * and the Rx can be safely reset.
- */
- res = gfar_read((void __iomem *)priv->gfargrp[0].regs + 0xd1c);
- res &= 0x7f807f80;
- if ((res & 0xffff) == (res >> 16))
- return 1;
-
- return 0;
-}
-
-/* Halt the receive and transmit queues */
-static void gfar_halt_nodisable(struct gfar_private *priv)
-{
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- u32 tempval;
- unsigned int timeout;
- int stopped;
-
- gfar_ints_disable(priv);
-
- if (gfar_is_dma_stopped(priv))
- return;
-
- /* Stop the DMA, and wait for it to stop */
- tempval = gfar_read(®s->dmactrl);
- tempval |= (DMACTRL_GRS | DMACTRL_GTS);
- gfar_write(®s->dmactrl, tempval);
-
-retry:
- timeout = 1000;
- while (!(stopped = gfar_is_dma_stopped(priv)) && timeout) {
- cpu_relax();
- timeout--;
- }
-
- if (!timeout)
- stopped = gfar_is_dma_stopped(priv);
-
- if (!stopped && !gfar_is_rx_dma_stopped(priv) &&
- !__gfar_is_rx_idle(priv))
- goto retry;
-}
-
-/* Halt the receive and transmit queues */
-void gfar_halt(struct gfar_private *priv)
-{
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- u32 tempval;
-
- /* Dissable the Rx/Tx hw queues */
- gfar_write(®s->rqueue, 0);
- gfar_write(®s->tqueue, 0);
-
- mdelay(10);
-
- gfar_halt_nodisable(priv);
-
- /* Disable Rx/Tx DMA */
- tempval = gfar_read(®s->maccfg1);
- tempval &= ~(MACCFG1_RX_EN | MACCFG1_TX_EN);
- gfar_write(®s->maccfg1, tempval);
-}
-
-void stop_gfar(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
-
- netif_tx_stop_all_queues(dev);
-
- smp_mb__before_atomic();
- set_bit(GFAR_DOWN, &priv->state);
- smp_mb__after_atomic();
-
- disable_napi(priv);
-
- /* disable ints and gracefully shut down Rx/Tx DMA */
- gfar_halt(priv);
-
- phy_stop(dev->phydev);
-
- free_skb_resources(priv);
-}
-
-static void free_skb_tx_queue(struct gfar_priv_tx_q *tx_queue)
-{
- struct txbd8 *txbdp;
- struct gfar_private *priv = netdev_priv(tx_queue->dev);
- int i, j;
-
- txbdp = tx_queue->tx_bd_base;
-
- for (i = 0; i < tx_queue->tx_ring_size; i++) {
- if (!tx_queue->tx_skbuff[i])
- continue;
-
- dma_unmap_single(priv->dev, be32_to_cpu(txbdp->bufPtr),
- be16_to_cpu(txbdp->length), DMA_TO_DEVICE);
- txbdp->lstatus = 0;
- for (j = 0; j < skb_shinfo(tx_queue->tx_skbuff[i])->nr_frags;
- j++) {
- txbdp++;
- dma_unmap_page(priv->dev, be32_to_cpu(txbdp->bufPtr),
- be16_to_cpu(txbdp->length),
- DMA_TO_DEVICE);
- }
- txbdp++;
- dev_kfree_skb_any(tx_queue->tx_skbuff[i]);
- tx_queue->tx_skbuff[i] = NULL;
- }
- kfree(tx_queue->tx_skbuff);
- tx_queue->tx_skbuff = NULL;
-}
-
-static void free_skb_rx_queue(struct gfar_priv_rx_q *rx_queue)
-{
- int i;
-
- struct rxbd8 *rxbdp = rx_queue->rx_bd_base;
-
- if (rx_queue->skb)
- dev_kfree_skb(rx_queue->skb);
-
- for (i = 0; i < rx_queue->rx_ring_size; i++) {
- struct gfar_rx_buff *rxb = &rx_queue->rx_buff[i];
-
- rxbdp->lstatus = 0;
- rxbdp->bufPtr = 0;
- rxbdp++;
-
- if (!rxb->page)
- continue;
-
- dma_unmap_page(rx_queue->dev, rxb->dma,
- PAGE_SIZE, DMA_FROM_DEVICE);
- __free_page(rxb->page);
-
- rxb->page = NULL;
- }
-
- kfree(rx_queue->rx_buff);
- rx_queue->rx_buff = NULL;
-}
-
-/* If there are any tx skbs or rx skbs still around, free them.
- * Then free tx_skbuff and rx_skbuff
- */
-static void free_skb_resources(struct gfar_private *priv)
-{
- struct gfar_priv_tx_q *tx_queue = NULL;
- struct gfar_priv_rx_q *rx_queue = NULL;
- int i;
-
- /* Go through all the buffer descriptors and free their data buffers */
- for (i = 0; i < priv->num_tx_queues; i++) {
- struct netdev_queue *txq;
-
- tx_queue = priv->tx_queue[i];
- txq = netdev_get_tx_queue(tx_queue->dev, tx_queue->qindex);
- if (tx_queue->tx_skbuff)
- free_skb_tx_queue(tx_queue);
- netdev_tx_reset_queue(txq);
- }
-
- for (i = 0; i < priv->num_rx_queues; i++) {
- rx_queue = priv->rx_queue[i];
- if (rx_queue->rx_buff)
- free_skb_rx_queue(rx_queue);
- }
-
- dma_free_coherent(priv->dev,
- sizeof(struct txbd8) * priv->total_tx_ring_size +
- sizeof(struct rxbd8) * priv->total_rx_ring_size,
- priv->tx_queue[0]->tx_bd_base,
- priv->tx_queue[0]->tx_bd_dma_base);
-}
-
-void gfar_start(struct gfar_private *priv)
-{
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- u32 tempval;
- int i = 0;
-
- /* Enable Rx/Tx hw queues */
- gfar_write(®s->rqueue, priv->rqueue);
- gfar_write(®s->tqueue, priv->tqueue);
-
- /* Initialize DMACTRL to have WWR and WOP */
- tempval = gfar_read(®s->dmactrl);
- tempval |= DMACTRL_INIT_SETTINGS;
- gfar_write(®s->dmactrl, tempval);
-
- /* Make sure we aren't stopped */
- tempval = gfar_read(®s->dmactrl);
- tempval &= ~(DMACTRL_GRS | DMACTRL_GTS);
- gfar_write(®s->dmactrl, tempval);
-
- for (i = 0; i < priv->num_grps; i++) {
- regs = priv->gfargrp[i].regs;
- /* Clear THLT/RHLT, so that the DMA starts polling now */
- gfar_write(®s->tstat, priv->gfargrp[i].tstat);
- gfar_write(®s->rstat, priv->gfargrp[i].rstat);
- }
-
- /* Enable Rx/Tx DMA */
- tempval = gfar_read(®s->maccfg1);
- tempval |= (MACCFG1_RX_EN | MACCFG1_TX_EN);
- gfar_write(®s->maccfg1, tempval);
-
- gfar_ints_enable(priv);
-
- netif_trans_update(priv->ndev); /* prevent tx timeout */
-}
-
-static void free_grp_irqs(struct gfar_priv_grp *grp)
-{
- free_irq(gfar_irq(grp, TX)->irq, grp);
- free_irq(gfar_irq(grp, RX)->irq, grp);
- free_irq(gfar_irq(grp, ER)->irq, grp);
-}
-
-static int register_grp_irqs(struct gfar_priv_grp *grp)
-{
- struct gfar_private *priv = grp->priv;
- struct net_device *dev = priv->ndev;
- int err;
-
- /* If the device has multiple interrupts, register for
- * them. Otherwise, only register for the one
- */
- if (priv->device_flags & FSL_GIANFAR_DEV_HAS_MULTI_INTR) {
- /* Install our interrupt handlers for Error,
- * Transmit, and Receive
- */
- err = request_irq(gfar_irq(grp, ER)->irq, gfar_error, 0,
- gfar_irq(grp, ER)->name, grp);
- if (err < 0) {
- netif_err(priv, intr, dev, "Can't get IRQ %d\n",
- gfar_irq(grp, ER)->irq);
-
- goto err_irq_fail;
- }
- enable_irq_wake(gfar_irq(grp, ER)->irq);
-
- err = request_irq(gfar_irq(grp, TX)->irq, gfar_transmit, 0,
- gfar_irq(grp, TX)->name, grp);
- if (err < 0) {
- netif_err(priv, intr, dev, "Can't get IRQ %d\n",
- gfar_irq(grp, TX)->irq);
- goto tx_irq_fail;
- }
- err = request_irq(gfar_irq(grp, RX)->irq, gfar_receive, 0,
- gfar_irq(grp, RX)->name, grp);
- if (err < 0) {
- netif_err(priv, intr, dev, "Can't get IRQ %d\n",
- gfar_irq(grp, RX)->irq);
- goto rx_irq_fail;
- }
- enable_irq_wake(gfar_irq(grp, RX)->irq);
-
- } else {
- err = request_irq(gfar_irq(grp, TX)->irq, gfar_interrupt, 0,
- gfar_irq(grp, TX)->name, grp);
- if (err < 0) {
- netif_err(priv, intr, dev, "Can't get IRQ %d\n",
- gfar_irq(grp, TX)->irq);
- goto err_irq_fail;
- }
- enable_irq_wake(gfar_irq(grp, TX)->irq);
- }
-
- return 0;
-
-rx_irq_fail:
- free_irq(gfar_irq(grp, TX)->irq, grp);
-tx_irq_fail:
- free_irq(gfar_irq(grp, ER)->irq, grp);
-err_irq_fail:
- return err;
-
-}
-
-static void gfar_free_irq(struct gfar_private *priv)
-{
- int i;
-
- /* Free the IRQs */
- if (priv->device_flags & FSL_GIANFAR_DEV_HAS_MULTI_INTR) {
- for (i = 0; i < priv->num_grps; i++)
- free_grp_irqs(&priv->gfargrp[i]);
- } else {
- for (i = 0; i < priv->num_grps; i++)
- free_irq(gfar_irq(&priv->gfargrp[i], TX)->irq,
- &priv->gfargrp[i]);
- }
-}
-
-static int gfar_request_irq(struct gfar_private *priv)
-{
- int err, i, j;
-
- for (i = 0; i < priv->num_grps; i++) {
- err = register_grp_irqs(&priv->gfargrp[i]);
- if (err) {
- for (j = 0; j < i; j++)
- free_grp_irqs(&priv->gfargrp[j]);
- return err;
- }
- }
-
- return 0;
-}
-
-/* Bring the controller up and running */
-int startup_gfar(struct net_device *ndev)
-{
- struct gfar_private *priv = netdev_priv(ndev);
- int err;
-
- gfar_mac_reset(priv);
-
- err = gfar_alloc_skb_resources(ndev);
- if (err)
- return err;
-
- gfar_init_tx_rx_base(priv);
-
- smp_mb__before_atomic();
- clear_bit(GFAR_DOWN, &priv->state);
- smp_mb__after_atomic();
-
- /* Start Rx/Tx DMA and enable the interrupts */
- gfar_start(priv);
-
- /* force link state update after mac reset */
- priv->oldlink = 0;
- priv->oldspeed = 0;
- priv->oldduplex = -1;
-
- phy_start(ndev->phydev);
-
- enable_napi(priv);
-
- netif_tx_wake_all_queues(ndev);
-
- return 0;
-}
-
-/* Called when something needs to use the ethernet device
- * Returns 0 for success.
- */
-static int gfar_enet_open(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
- int err;
-
- err = init_phy(dev);
- if (err)
- return err;
-
- err = gfar_request_irq(priv);
- if (err)
- return err;
-
- err = startup_gfar(dev);
- if (err)
- return err;
-
- return err;
-}
-
-static inline struct txfcb *gfar_add_fcb(struct sk_buff *skb)
-{
- struct txfcb *fcb = skb_push(skb, GMAC_FCB_LEN);
-
- memset(fcb, 0, GMAC_FCB_LEN);
-
- return fcb;
-}
-
-static inline void gfar_tx_checksum(struct sk_buff *skb, struct txfcb *fcb,
- int fcb_length)
-{
- /* If we're here, it's a IP packet with a TCP or UDP
- * payload. We set it to checksum, using a pseudo-header
- * we provide
- */
- u8 flags = TXFCB_DEFAULT;
-
- /* Tell the controller what the protocol is
- * And provide the already calculated phcs
- */
- if (ip_hdr(skb)->protocol == IPPROTO_UDP) {
- flags |= TXFCB_UDP;
- fcb->phcs = (__force __be16)(udp_hdr(skb)->check);
- } else
- fcb->phcs = (__force __be16)(tcp_hdr(skb)->check);
-
- /* l3os is the distance between the start of the
- * frame (skb->data) and the start of the IP hdr.
- * l4os is the distance between the start of the
- * l3 hdr and the l4 hdr
- */
- fcb->l3os = (u8)(skb_network_offset(skb) - fcb_length);
- fcb->l4os = skb_network_header_len(skb);
-
- fcb->flags = flags;
-}
-
-static inline void gfar_tx_vlan(struct sk_buff *skb, struct txfcb *fcb)
-{
- fcb->flags |= TXFCB_VLN;
- fcb->vlctl = cpu_to_be16(skb_vlan_tag_get(skb));
-}
-
-static inline struct txbd8 *skip_txbd(struct txbd8 *bdp, int stride,
- struct txbd8 *base, int ring_size)
-{
- struct txbd8 *new_bd = bdp + stride;
-
- return (new_bd >= (base + ring_size)) ? (new_bd - ring_size) : new_bd;
-}
-
-static inline struct txbd8 *next_txbd(struct txbd8 *bdp, struct txbd8 *base,
- int ring_size)
-{
- return skip_txbd(bdp, 1, base, ring_size);
-}
-
-/* eTSEC12: csum generation not supported for some fcb offsets */
-static inline bool gfar_csum_errata_12(struct gfar_private *priv,
- unsigned long fcb_addr)
-{
- return (gfar_has_errata(priv, GFAR_ERRATA_12) &&
- (fcb_addr % 0x20) > 0x18);
-}
-
-/* eTSEC76: csum generation for frames larger than 2500 may
- * cause excess delays before start of transmission
- */
-static inline bool gfar_csum_errata_76(struct gfar_private *priv,
- unsigned int len)
-{
- return (gfar_has_errata(priv, GFAR_ERRATA_76) &&
- (len > 2500));
-}
-
-/* This is called by the kernel when a frame is ready for transmission.
- * It is pointed to by the dev->hard_start_xmit function pointer
- */
-static netdev_tx_t gfar_start_xmit(struct sk_buff *skb, struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
- struct gfar_priv_tx_q *tx_queue = NULL;
- struct netdev_queue *txq;
- struct gfar __iomem *regs = NULL;
- struct txfcb *fcb = NULL;
- struct txbd8 *txbdp, *txbdp_start, *base, *txbdp_tstamp = NULL;
- u32 lstatus;
- skb_frag_t *frag;
- int i, rq = 0;
- int do_tstamp, do_csum, do_vlan;
- u32 bufaddr;
- unsigned int nr_frags, nr_txbds, bytes_sent, fcb_len = 0;
-
- rq = skb->queue_mapping;
- tx_queue = priv->tx_queue[rq];
- txq = netdev_get_tx_queue(dev, rq);
- base = tx_queue->tx_bd_base;
- regs = tx_queue->grp->regs;
-
- do_csum = (CHECKSUM_PARTIAL == skb->ip_summed);
- do_vlan = skb_vlan_tag_present(skb);
- do_tstamp = (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
- priv->hwts_tx_en;
-
- if (do_csum || do_vlan)
- fcb_len = GMAC_FCB_LEN;
-
- /* check if time stamp should be generated */
- if (unlikely(do_tstamp))
- fcb_len = GMAC_FCB_LEN + GMAC_TXPAL_LEN;
-
- /* make space for additional header when fcb is needed */
- if (fcb_len && unlikely(skb_headroom(skb) < fcb_len)) {
- struct sk_buff *skb_new;
-
- skb_new = skb_realloc_headroom(skb, fcb_len);
- if (!skb_new) {
- dev->stats.tx_errors++;
- dev_kfree_skb_any(skb);
- return NETDEV_TX_OK;
- }
-
- if (skb->sk)
- skb_set_owner_w(skb_new, skb->sk);
- dev_consume_skb_any(skb);
- skb = skb_new;
- }
-
- /* total number of fragments in the SKB */
- nr_frags = skb_shinfo(skb)->nr_frags;
-
- /* calculate the required number of TxBDs for this skb */
- if (unlikely(do_tstamp))
- nr_txbds = nr_frags + 2;
- else
- nr_txbds = nr_frags + 1;
-
- /* check if there is space to queue this packet */
- if (nr_txbds > tx_queue->num_txbdfree) {
- /* no space, stop the queue */
- netif_tx_stop_queue(txq);
- dev->stats.tx_fifo_errors++;
- return NETDEV_TX_BUSY;
- }
-
- /* Update transmit stats */
- bytes_sent = skb->len;
- tx_queue->stats.tx_bytes += bytes_sent;
- /* keep Tx bytes on wire for BQL accounting */
- GFAR_CB(skb)->bytes_sent = bytes_sent;
- tx_queue->stats.tx_packets++;
-
- txbdp = txbdp_start = tx_queue->cur_tx;
- lstatus = be32_to_cpu(txbdp->lstatus);
-
- /* Add TxPAL between FCB and frame if required */
- if (unlikely(do_tstamp)) {
- skb_push(skb, GMAC_TXPAL_LEN);
- memset(skb->data, 0, GMAC_TXPAL_LEN);
- }
-
- /* Add TxFCB if required */
- if (fcb_len) {
- fcb = gfar_add_fcb(skb);
- lstatus |= BD_LFLAG(TXBD_TOE);
- }
-
- /* Set up checksumming */
- if (do_csum) {
- gfar_tx_checksum(skb, fcb, fcb_len);
-
- if (unlikely(gfar_csum_errata_12(priv, (unsigned long)fcb)) ||
- unlikely(gfar_csum_errata_76(priv, skb->len))) {
- __skb_pull(skb, GMAC_FCB_LEN);
- skb_checksum_help(skb);
- if (do_vlan || do_tstamp) {
- /* put back a new fcb for vlan/tstamp TOE */
- fcb = gfar_add_fcb(skb);
- } else {
- /* Tx TOE not used */
- lstatus &= ~(BD_LFLAG(TXBD_TOE));
- fcb = NULL;
- }
- }
- }
-
- if (do_vlan)
- gfar_tx_vlan(skb, fcb);
-
- bufaddr = dma_map_single(priv->dev, skb->data, skb_headlen(skb),
- DMA_TO_DEVICE);
- if (unlikely(dma_mapping_error(priv->dev, bufaddr)))
- goto dma_map_err;
-
- txbdp_start->bufPtr = cpu_to_be32(bufaddr);
-
- /* Time stamp insertion requires one additional TxBD */
- if (unlikely(do_tstamp))
- txbdp_tstamp = txbdp = next_txbd(txbdp, base,
- tx_queue->tx_ring_size);
-
- if (likely(!nr_frags)) {
- if (likely(!do_tstamp))
- lstatus |= BD_LFLAG(TXBD_LAST | TXBD_INTERRUPT);
- } else {
- u32 lstatus_start = lstatus;
-
- /* Place the fragment addresses and lengths into the TxBDs */
- frag = &skb_shinfo(skb)->frags[0];
- for (i = 0; i < nr_frags; i++, frag++) {
- unsigned int size;
-
- /* Point at the next BD, wrapping as needed */
- txbdp = next_txbd(txbdp, base, tx_queue->tx_ring_size);
-
- size = skb_frag_size(frag);
-
- lstatus = be32_to_cpu(txbdp->lstatus) | size |
- BD_LFLAG(TXBD_READY);
-
- /* Handle the last BD specially */
- if (i == nr_frags - 1)
- lstatus |= BD_LFLAG(TXBD_LAST | TXBD_INTERRUPT);
-
- bufaddr = skb_frag_dma_map(priv->dev, frag, 0,
- size, DMA_TO_DEVICE);
- if (unlikely(dma_mapping_error(priv->dev, bufaddr)))
- goto dma_map_err;
-
- /* set the TxBD length and buffer pointer */
- txbdp->bufPtr = cpu_to_be32(bufaddr);
- txbdp->lstatus = cpu_to_be32(lstatus);
- }
-
- lstatus = lstatus_start;
- }
-
- /* If time stamping is requested one additional TxBD must be set up. The
- * first TxBD points to the FCB and must have a data length of
- * GMAC_FCB_LEN. The second TxBD points to the actual frame data with
- * the full frame length.
- */
- if (unlikely(do_tstamp)) {
- u32 lstatus_ts = be32_to_cpu(txbdp_tstamp->lstatus);
-
- bufaddr = be32_to_cpu(txbdp_start->bufPtr);
- bufaddr += fcb_len;
-
- lstatus_ts |= BD_LFLAG(TXBD_READY) |
- (skb_headlen(skb) - fcb_len);
- if (!nr_frags)
- lstatus_ts |= BD_LFLAG(TXBD_LAST | TXBD_INTERRUPT);
-
- txbdp_tstamp->bufPtr = cpu_to_be32(bufaddr);
- txbdp_tstamp->lstatus = cpu_to_be32(lstatus_ts);
- lstatus |= BD_LFLAG(TXBD_CRC | TXBD_READY) | GMAC_FCB_LEN;
-
- /* Setup tx hardware time stamping */
- skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
- fcb->ptp = 1;
- } else {
- lstatus |= BD_LFLAG(TXBD_CRC | TXBD_READY) | skb_headlen(skb);
- }
-
- netdev_tx_sent_queue(txq, bytes_sent);
-
- gfar_wmb();
-
- txbdp_start->lstatus = cpu_to_be32(lstatus);
-
- gfar_wmb(); /* force lstatus write before tx_skbuff */
-
- tx_queue->tx_skbuff[tx_queue->skb_curtx] = skb;
-
- /* Update the current skb pointer to the next entry we will use
- * (wrapping if necessary)
- */
- tx_queue->skb_curtx = (tx_queue->skb_curtx + 1) &
- TX_RING_MOD_MASK(tx_queue->tx_ring_size);
-
- tx_queue->cur_tx = next_txbd(txbdp, base, tx_queue->tx_ring_size);
-
- /* We can work in parallel with gfar_clean_tx_ring(), except
- * when modifying num_txbdfree. Note that we didn't grab the lock
- * when we were reading the num_txbdfree and checking for available
- * space, that's because outside of this function it can only grow.
- */
- spin_lock_bh(&tx_queue->txlock);
- /* reduce TxBD free count */
- tx_queue->num_txbdfree -= (nr_txbds);
- spin_unlock_bh(&tx_queue->txlock);
-
- /* If the next BD still needs to be cleaned up, then the bds
- * are full. We need to tell the kernel to stop sending us stuff.
- */
- if (!tx_queue->num_txbdfree) {
- netif_tx_stop_queue(txq);
-
- dev->stats.tx_fifo_errors++;
- }
-
- /* Tell the DMA to go go go */
- gfar_write(®s->tstat, TSTAT_CLEAR_THALT >> tx_queue->qindex);
-
- return NETDEV_TX_OK;
-
-dma_map_err:
- txbdp = next_txbd(txbdp_start, base, tx_queue->tx_ring_size);
- if (do_tstamp)
- txbdp = next_txbd(txbdp, base, tx_queue->tx_ring_size);
- for (i = 0; i < nr_frags; i++) {
- lstatus = be32_to_cpu(txbdp->lstatus);
- if (!(lstatus & BD_LFLAG(TXBD_READY)))
- break;
-
- lstatus &= ~BD_LFLAG(TXBD_READY);
- txbdp->lstatus = cpu_to_be32(lstatus);
- bufaddr = be32_to_cpu(txbdp->bufPtr);
- dma_unmap_page(priv->dev, bufaddr, be16_to_cpu(txbdp->length),
- DMA_TO_DEVICE);
- txbdp = next_txbd(txbdp, base, tx_queue->tx_ring_size);
- }
- gfar_wmb();
- dev_kfree_skb_any(skb);
- return NETDEV_TX_OK;
-}
-
-/* Stops the kernel queue, and halts the controller */
-static int gfar_close(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
-
- cancel_work_sync(&priv->reset_task);
- stop_gfar(dev);
-
- /* Disconnect from the PHY */
- phy_disconnect(dev->phydev);
-
- gfar_free_irq(priv);
-
- return 0;
-}
-
-/* Changes the mac address if the controller is not running. */
-static int gfar_set_mac_address(struct net_device *dev)
-{
- gfar_set_mac_for_addr(dev, 0, dev->dev_addr);
-
- return 0;
-}
-
-static int gfar_change_mtu(struct net_device *dev, int new_mtu)
-{
- struct gfar_private *priv = netdev_priv(dev);
-
- while (test_and_set_bit_lock(GFAR_RESETTING, &priv->state))
- cpu_relax();
-
- if (dev->flags & IFF_UP)
- stop_gfar(dev);
-
- dev->mtu = new_mtu;
-
- if (dev->flags & IFF_UP)
- startup_gfar(dev);
-
- clear_bit_unlock(GFAR_RESETTING, &priv->state);
-
- return 0;
-}
-
-void reset_gfar(struct net_device *ndev)
-{
- struct gfar_private *priv = netdev_priv(ndev);
-
- while (test_and_set_bit_lock(GFAR_RESETTING, &priv->state))
- cpu_relax();
-
- stop_gfar(ndev);
- startup_gfar(ndev);
-
- clear_bit_unlock(GFAR_RESETTING, &priv->state);
-}
-
-/* gfar_reset_task gets scheduled when a packet has not been
- * transmitted after a set amount of time.
- * For now, assume that clearing out all the structures, and
- * starting over will fix the problem.
- */
-static void gfar_reset_task(struct work_struct *work)
-{
- struct gfar_private *priv = container_of(work, struct gfar_private,
- reset_task);
- reset_gfar(priv->ndev);
-}
-
-static void gfar_timeout(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
-
- dev->stats.tx_errors++;
- schedule_work(&priv->reset_task);
-}
-
-/* Interrupt Handler for Transmit complete */
-static void gfar_clean_tx_ring(struct gfar_priv_tx_q *tx_queue)
-{
- struct net_device *dev = tx_queue->dev;
- struct netdev_queue *txq;
- struct gfar_private *priv = netdev_priv(dev);
- struct txbd8 *bdp, *next = NULL;
- struct txbd8 *lbdp = NULL;
- struct txbd8 *base = tx_queue->tx_bd_base;
- struct sk_buff *skb;
- int skb_dirtytx;
- int tx_ring_size = tx_queue->tx_ring_size;
- int frags = 0, nr_txbds = 0;
- int i;
- int howmany = 0;
- int tqi = tx_queue->qindex;
- unsigned int bytes_sent = 0;
- u32 lstatus;
- size_t buflen;
-
- txq = netdev_get_tx_queue(dev, tqi);
- bdp = tx_queue->dirty_tx;
- skb_dirtytx = tx_queue->skb_dirtytx;
-
- while ((skb = tx_queue->tx_skbuff[skb_dirtytx])) {
-
- frags = skb_shinfo(skb)->nr_frags;
-
- /* When time stamping, one additional TxBD must be freed.
- * Also, we need to dma_unmap_single() the TxPAL.
- */
- if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS))
- nr_txbds = frags + 2;
- else
- nr_txbds = frags + 1;
-
- lbdp = skip_txbd(bdp, nr_txbds - 1, base, tx_ring_size);
-
- lstatus = be32_to_cpu(lbdp->lstatus);
-
- /* Only clean completed frames */
- if ((lstatus & BD_LFLAG(TXBD_READY)) &&
- (lstatus & BD_LENGTH_MASK))
- break;
-
- if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS)) {
- next = next_txbd(bdp, base, tx_ring_size);
- buflen = be16_to_cpu(next->length) +
- GMAC_FCB_LEN + GMAC_TXPAL_LEN;
- } else
- buflen = be16_to_cpu(bdp->length);
-
- dma_unmap_single(priv->dev, be32_to_cpu(bdp->bufPtr),
- buflen, DMA_TO_DEVICE);
-
- if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS)) {
- struct skb_shared_hwtstamps shhwtstamps;
- u64 *ns = (u64 *)(((uintptr_t)skb->data + 0x10) &
- ~0x7UL);
-
- memset(&shhwtstamps, 0, sizeof(shhwtstamps));
- shhwtstamps.hwtstamp = ns_to_ktime(be64_to_cpu(*ns));
- skb_pull(skb, GMAC_FCB_LEN + GMAC_TXPAL_LEN);
- skb_tstamp_tx(skb, &shhwtstamps);
- gfar_clear_txbd_status(bdp);
- bdp = next;
- }
-
- gfar_clear_txbd_status(bdp);
- bdp = next_txbd(bdp, base, tx_ring_size);
-
- for (i = 0; i < frags; i++) {
- dma_unmap_page(priv->dev, be32_to_cpu(bdp->bufPtr),
- be16_to_cpu(bdp->length),
- DMA_TO_DEVICE);
- gfar_clear_txbd_status(bdp);
- bdp = next_txbd(bdp, base, tx_ring_size);
- }
-
- bytes_sent += GFAR_CB(skb)->bytes_sent;
-
- dev_kfree_skb_any(skb);
-
- tx_queue->tx_skbuff[skb_dirtytx] = NULL;
-
- skb_dirtytx = (skb_dirtytx + 1) &
- TX_RING_MOD_MASK(tx_ring_size);
-
- howmany++;
- spin_lock(&tx_queue->txlock);
- tx_queue->num_txbdfree += nr_txbds;
- spin_unlock(&tx_queue->txlock);
- }
-
- /* If we freed a buffer, we can restart transmission, if necessary */
- if (tx_queue->num_txbdfree &&
- netif_tx_queue_stopped(txq) &&
- !(test_bit(GFAR_DOWN, &priv->state)))
- netif_wake_subqueue(priv->ndev, tqi);
-
- /* Update dirty indicators */
- tx_queue->skb_dirtytx = skb_dirtytx;
- tx_queue->dirty_tx = bdp;
-
- netdev_tx_completed_queue(txq, howmany, bytes_sent);
-}
-
-static bool gfar_new_page(struct gfar_priv_rx_q *rxq, struct gfar_rx_buff *rxb)
-{
- struct page *page;
- dma_addr_t addr;
-
- page = dev_alloc_page();
- if (unlikely(!page))
- return false;
-
- addr = dma_map_page(rxq->dev, page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
- if (unlikely(dma_mapping_error(rxq->dev, addr))) {
- __free_page(page);
-
- return false;
- }
-
- rxb->dma = addr;
- rxb->page = page;
- rxb->page_offset = 0;
-
- return true;
-}
-
-static void gfar_rx_alloc_err(struct gfar_priv_rx_q *rx_queue)
-{
- struct gfar_private *priv = netdev_priv(rx_queue->ndev);
- struct gfar_extra_stats *estats = &priv->extra_stats;
-
- netdev_err(rx_queue->ndev, "Can't alloc RX buffers\n");
- atomic64_inc(&estats->rx_alloc_err);
-}
-
-static void gfar_alloc_rx_buffs(struct gfar_priv_rx_q *rx_queue,
- int alloc_cnt)
-{
- struct rxbd8 *bdp;
- struct gfar_rx_buff *rxb;
- int i;
-
- i = rx_queue->next_to_use;
- bdp = &rx_queue->rx_bd_base[i];
- rxb = &rx_queue->rx_buff[i];
-
- while (alloc_cnt--) {
- /* try reuse page */
- if (unlikely(!rxb->page)) {
- if (unlikely(!gfar_new_page(rx_queue, rxb))) {
- gfar_rx_alloc_err(rx_queue);
- break;
- }
- }
-
- /* Setup the new RxBD */
- gfar_init_rxbdp(rx_queue, bdp,
- rxb->dma + rxb->page_offset + RXBUF_ALIGNMENT);
-
- /* Update to the next pointer */
- bdp++;
- rxb++;
-
- if (unlikely(++i == rx_queue->rx_ring_size)) {
- i = 0;
- bdp = rx_queue->rx_bd_base;
- rxb = rx_queue->rx_buff;
- }
- }
-
- rx_queue->next_to_use = i;
- rx_queue->next_to_alloc = i;
-}
-
-static void count_errors(u32 lstatus, struct net_device *ndev)
-{
- struct gfar_private *priv = netdev_priv(ndev);
- struct net_device_stats *stats = &ndev->stats;
- struct gfar_extra_stats *estats = &priv->extra_stats;
-
- /* If the packet was truncated, none of the other errors matter */
- if (lstatus & BD_LFLAG(RXBD_TRUNCATED)) {
- stats->rx_length_errors++;
-
- atomic64_inc(&estats->rx_trunc);
-
- return;
- }
- /* Count the errors, if there were any */
- if (lstatus & BD_LFLAG(RXBD_LARGE | RXBD_SHORT)) {
- stats->rx_length_errors++;
-
- if (lstatus & BD_LFLAG(RXBD_LARGE))
- atomic64_inc(&estats->rx_large);
- else
- atomic64_inc(&estats->rx_short);
- }
- if (lstatus & BD_LFLAG(RXBD_NONOCTET)) {
- stats->rx_frame_errors++;
- atomic64_inc(&estats->rx_nonoctet);
- }
- if (lstatus & BD_LFLAG(RXBD_CRCERR)) {
- atomic64_inc(&estats->rx_crcerr);
- stats->rx_crc_errors++;
- }
- if (lstatus & BD_LFLAG(RXBD_OVERRUN)) {
- atomic64_inc(&estats->rx_overrun);
- stats->rx_over_errors++;
- }
-}
-
-irqreturn_t gfar_receive(int irq, void *grp_id)
-{
- struct gfar_priv_grp *grp = (struct gfar_priv_grp *)grp_id;
- unsigned long flags;
- u32 imask, ievent;
-
- ievent = gfar_read(&grp->regs->ievent);
-
- if (unlikely(ievent & IEVENT_FGPI)) {
- gfar_write(&grp->regs->ievent, IEVENT_FGPI);
- return IRQ_HANDLED;
- }
-
- if (likely(napi_schedule_prep(&grp->napi_rx))) {
- spin_lock_irqsave(&grp->grplock, flags);
- imask = gfar_read(&grp->regs->imask);
- imask &= IMASK_RX_DISABLED;
- gfar_write(&grp->regs->imask, imask);
- spin_unlock_irqrestore(&grp->grplock, flags);
- __napi_schedule(&grp->napi_rx);
- } else {
- /* Clear IEVENT, so interrupts aren't called again
- * because of the packets that have already arrived.
- */
- gfar_write(&grp->regs->ievent, IEVENT_RX_MASK);
- }
-
- return IRQ_HANDLED;
-}
-
-/* Interrupt Handler for Transmit complete */
-static irqreturn_t gfar_transmit(int irq, void *grp_id)
-{
- struct gfar_priv_grp *grp = (struct gfar_priv_grp *)grp_id;
- unsigned long flags;
- u32 imask;
-
- if (likely(napi_schedule_prep(&grp->napi_tx))) {
- spin_lock_irqsave(&grp->grplock, flags);
- imask = gfar_read(&grp->regs->imask);
- imask &= IMASK_TX_DISABLED;
- gfar_write(&grp->regs->imask, imask);
- spin_unlock_irqrestore(&grp->grplock, flags);
- __napi_schedule(&grp->napi_tx);
- } else {
- /* Clear IEVENT, so interrupts aren't called again
- * because of the packets that have already arrived.
- */
- gfar_write(&grp->regs->ievent, IEVENT_TX_MASK);
- }
-
- return IRQ_HANDLED;
-}
-
-static bool gfar_add_rx_frag(struct gfar_rx_buff *rxb, u32 lstatus,
- struct sk_buff *skb, bool first)
-{
- int size = lstatus & BD_LENGTH_MASK;
- struct page *page = rxb->page;
-
- if (likely(first)) {
- skb_put(skb, size);
- } else {
- /* the last fragments' length contains the full frame length */
- if (lstatus & BD_LFLAG(RXBD_LAST))
- size -= skb->len;
-
- skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
- rxb->page_offset + RXBUF_ALIGNMENT,
- size, GFAR_RXB_TRUESIZE);
- }
-
- /* try reuse page */
- if (unlikely(page_count(page) != 1 || page_is_pfmemalloc(page)))
- return false;
-
- /* change offset to the other half */
- rxb->page_offset ^= GFAR_RXB_TRUESIZE;
-
- page_ref_inc(page);
-
- return true;
-}
-
-static void gfar_reuse_rx_page(struct gfar_priv_rx_q *rxq,
- struct gfar_rx_buff *old_rxb)
-{
- struct gfar_rx_buff *new_rxb;
- u16 nta = rxq->next_to_alloc;
-
- new_rxb = &rxq->rx_buff[nta];
-
- /* find next buf that can reuse a page */
- nta++;
- rxq->next_to_alloc = (nta < rxq->rx_ring_size) ? nta : 0;
-
- /* copy page reference */
- *new_rxb = *old_rxb;
-
- /* sync for use by the device */
- dma_sync_single_range_for_device(rxq->dev, old_rxb->dma,
- old_rxb->page_offset,
- GFAR_RXB_TRUESIZE, DMA_FROM_DEVICE);
-}
-
-static struct sk_buff *gfar_get_next_rxbuff(struct gfar_priv_rx_q *rx_queue,
- u32 lstatus, struct sk_buff *skb)
-{
- struct gfar_rx_buff *rxb = &rx_queue->rx_buff[rx_queue->next_to_clean];
- struct page *page = rxb->page;
- bool first = false;
-
- if (likely(!skb)) {
- void *buff_addr = page_address(page) + rxb->page_offset;
-
- skb = build_skb(buff_addr, GFAR_SKBFRAG_SIZE);
- if (unlikely(!skb)) {
- gfar_rx_alloc_err(rx_queue);
- return NULL;
- }
- skb_reserve(skb, RXBUF_ALIGNMENT);
- first = true;
- }
-
- dma_sync_single_range_for_cpu(rx_queue->dev, rxb->dma, rxb->page_offset,
- GFAR_RXB_TRUESIZE, DMA_FROM_DEVICE);
-
- if (gfar_add_rx_frag(rxb, lstatus, skb, first)) {
- /* reuse the free half of the page */
- gfar_reuse_rx_page(rx_queue, rxb);
- } else {
- /* page cannot be reused, unmap it */
- dma_unmap_page(rx_queue->dev, rxb->dma,
- PAGE_SIZE, DMA_FROM_DEVICE);
- }
-
- /* clear rxb content */
- rxb->page = NULL;
-
- return skb;
-}
-
-static inline void gfar_rx_checksum(struct sk_buff *skb, struct rxfcb *fcb)
-{
- /* If valid headers were found, and valid sums
- * were verified, then we tell the kernel that no
- * checksumming is necessary. Otherwise, it is [FIXME]
- */
- if ((be16_to_cpu(fcb->flags) & RXFCB_CSUM_MASK) ==
- (RXFCB_CIP | RXFCB_CTU))
- skb->ip_summed = CHECKSUM_UNNECESSARY;
- else
- skb_checksum_none_assert(skb);
-}
-
-/* gfar_process_frame() -- handle one incoming packet if skb isn't NULL. */
-static void gfar_process_frame(struct net_device *ndev, struct sk_buff *skb)
-{
- struct gfar_private *priv = netdev_priv(ndev);
- struct rxfcb *fcb = NULL;
-
- /* fcb is at the beginning if exists */
- fcb = (struct rxfcb *)skb->data;
-
- /* Remove the FCB from the skb
- * Remove the padded bytes, if there are any
- */
- if (priv->uses_rxfcb)
- skb_pull(skb, GMAC_FCB_LEN);
-
- /* Get receive timestamp from the skb */
- if (priv->hwts_rx_en) {
- struct skb_shared_hwtstamps *shhwtstamps = skb_hwtstamps(skb);
- u64 *ns = (u64 *) skb->data;
-
- memset(shhwtstamps, 0, sizeof(*shhwtstamps));
- shhwtstamps->hwtstamp = ns_to_ktime(be64_to_cpu(*ns));
- }
-
- if (priv->padding)
- skb_pull(skb, priv->padding);
-
- /* Trim off the FCS */
- pskb_trim(skb, skb->len - ETH_FCS_LEN);
-
- if (ndev->features & NETIF_F_RXCSUM)
- gfar_rx_checksum(skb, fcb);
-
- /* There's need to check for NETIF_F_HW_VLAN_CTAG_RX here.
- * Even if vlan rx accel is disabled, on some chips
- * RXFCB_VLN is pseudo randomly set.
- */
- if (ndev->features & NETIF_F_HW_VLAN_CTAG_RX &&
- be16_to_cpu(fcb->flags) & RXFCB_VLN)
- __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
- be16_to_cpu(fcb->vlctl));
-}
-
-/* gfar_clean_rx_ring() -- Processes each frame in the rx ring
- * until the budget/quota has been reached. Returns the number
- * of frames handled
- */
-int gfar_clean_rx_ring(struct gfar_priv_rx_q *rx_queue, int rx_work_limit)
-{
- struct net_device *ndev = rx_queue->ndev;
- struct gfar_private *priv = netdev_priv(ndev);
- struct rxbd8 *bdp;
- int i, howmany = 0;
- struct sk_buff *skb = rx_queue->skb;
- int cleaned_cnt = gfar_rxbd_unused(rx_queue);
- unsigned int total_bytes = 0, total_pkts = 0;
-
- /* Get the first full descriptor */
- i = rx_queue->next_to_clean;
-
- while (rx_work_limit--) {
- u32 lstatus;
-
- if (cleaned_cnt >= GFAR_RX_BUFF_ALLOC) {
- gfar_alloc_rx_buffs(rx_queue, cleaned_cnt);
- cleaned_cnt = 0;
- }
-
- bdp = &rx_queue->rx_bd_base[i];
- lstatus = be32_to_cpu(bdp->lstatus);
- if (lstatus & BD_LFLAG(RXBD_EMPTY))
- break;
-
- /* order rx buffer descriptor reads */
- rmb();
-
- /* fetch next to clean buffer from the ring */
- skb = gfar_get_next_rxbuff(rx_queue, lstatus, skb);
- if (unlikely(!skb))
- break;
-
- cleaned_cnt++;
- howmany++;
-
- if (unlikely(++i == rx_queue->rx_ring_size))
- i = 0;
-
- rx_queue->next_to_clean = i;
-
- /* fetch next buffer if not the last in frame */
- if (!(lstatus & BD_LFLAG(RXBD_LAST)))
- continue;
-
- if (unlikely(lstatus & BD_LFLAG(RXBD_ERR))) {
- count_errors(lstatus, ndev);
-
- /* discard faulty buffer */
- dev_kfree_skb(skb);
- skb = NULL;
- rx_queue->stats.rx_dropped++;
- continue;
- }
-
- gfar_process_frame(ndev, skb);
-
- /* Increment the number of packets */
- total_pkts++;
- total_bytes += skb->len;
-
- skb_record_rx_queue(skb, rx_queue->qindex);
-
- skb->protocol = eth_type_trans(skb, ndev);
-
- /* Send the packet up the stack */
- napi_gro_receive(&rx_queue->grp->napi_rx, skb);
-
- skb = NULL;
- }
-
- /* Store incomplete frames for completion */
- rx_queue->skb = skb;
-
- rx_queue->stats.rx_packets += total_pkts;
- rx_queue->stats.rx_bytes += total_bytes;
-
- if (cleaned_cnt)
- gfar_alloc_rx_buffs(rx_queue, cleaned_cnt);
-
- /* Update Last Free RxBD pointer for LFC */
- if (unlikely(priv->tx_actual_en)) {
- u32 bdp_dma = gfar_rxbd_dma_lastfree(rx_queue);
-
- gfar_write(rx_queue->rfbptr, bdp_dma);
- }
-
- return howmany;
-}
-
-static int gfar_poll_rx_sq(struct napi_struct *napi, int budget)
-{
- struct gfar_priv_grp *gfargrp =
- container_of(napi, struct gfar_priv_grp, napi_rx);
- struct gfar __iomem *regs = gfargrp->regs;
- struct gfar_priv_rx_q *rx_queue = gfargrp->rx_queue;
- int work_done = 0;
-
- /* Clear IEVENT, so interrupts aren't called again
- * because of the packets that have already arrived
- */
- gfar_write(®s->ievent, IEVENT_RX_MASK);
-
- work_done = gfar_clean_rx_ring(rx_queue, budget);
-
- if (work_done < budget) {
- u32 imask;
- napi_complete_done(napi, work_done);
- /* Clear the halt bit in RSTAT */
- gfar_write(®s->rstat, gfargrp->rstat);
-
- spin_lock_irq(&gfargrp->grplock);
- imask = gfar_read(®s->imask);
- imask |= IMASK_RX_DEFAULT;
- gfar_write(®s->imask, imask);
- spin_unlock_irq(&gfargrp->grplock);
- }
-
- return work_done;
-}
-
-static int gfar_poll_tx_sq(struct napi_struct *napi, int budget)
-{
- struct gfar_priv_grp *gfargrp =
- container_of(napi, struct gfar_priv_grp, napi_tx);
- struct gfar __iomem *regs = gfargrp->regs;
- struct gfar_priv_tx_q *tx_queue = gfargrp->tx_queue;
- u32 imask;
-
- /* Clear IEVENT, so interrupts aren't called again
- * because of the packets that have already arrived
- */
- gfar_write(®s->ievent, IEVENT_TX_MASK);
-
- /* run Tx cleanup to completion */
- if (tx_queue->tx_skbuff[tx_queue->skb_dirtytx])
- gfar_clean_tx_ring(tx_queue);
-
- napi_complete(napi);
-
- spin_lock_irq(&gfargrp->grplock);
- imask = gfar_read(®s->imask);
- imask |= IMASK_TX_DEFAULT;
- gfar_write(®s->imask, imask);
- spin_unlock_irq(&gfargrp->grplock);
-
- return 0;
-}
-
-static int gfar_poll_rx(struct napi_struct *napi, int budget)
-{
- struct gfar_priv_grp *gfargrp =
- container_of(napi, struct gfar_priv_grp, napi_rx);
- struct gfar_private *priv = gfargrp->priv;
- struct gfar __iomem *regs = gfargrp->regs;
- struct gfar_priv_rx_q *rx_queue = NULL;
- int work_done = 0, work_done_per_q = 0;
- int i, budget_per_q = 0;
- unsigned long rstat_rxf;
- int num_act_queues;
-
- /* Clear IEVENT, so interrupts aren't called again
- * because of the packets that have already arrived
- */
- gfar_write(®s->ievent, IEVENT_RX_MASK);
-
- rstat_rxf = gfar_read(®s->rstat) & RSTAT_RXF_MASK;
-
- num_act_queues = bitmap_weight(&rstat_rxf, MAX_RX_QS);
- if (num_act_queues)
- budget_per_q = budget/num_act_queues;
-
- for_each_set_bit(i, &gfargrp->rx_bit_map, priv->num_rx_queues) {
- /* skip queue if not active */
- if (!(rstat_rxf & (RSTAT_CLEAR_RXF0 >> i)))
- continue;
-
- rx_queue = priv->rx_queue[i];
- work_done_per_q =
- gfar_clean_rx_ring(rx_queue, budget_per_q);
- work_done += work_done_per_q;
-
- /* finished processing this queue */
- if (work_done_per_q < budget_per_q) {
- /* clear active queue hw indication */
- gfar_write(®s->rstat,
- RSTAT_CLEAR_RXF0 >> i);
- num_act_queues--;
-
- if (!num_act_queues)
- break;
- }
- }
-
- if (!num_act_queues) {
- u32 imask;
- napi_complete_done(napi, work_done);
-
- /* Clear the halt bit in RSTAT */
- gfar_write(®s->rstat, gfargrp->rstat);
-
- spin_lock_irq(&gfargrp->grplock);
- imask = gfar_read(®s->imask);
- imask |= IMASK_RX_DEFAULT;
- gfar_write(®s->imask, imask);
- spin_unlock_irq(&gfargrp->grplock);
- }
-
- return work_done;
-}
-
-static int gfar_poll_tx(struct napi_struct *napi, int budget)
-{
- struct gfar_priv_grp *gfargrp =
- container_of(napi, struct gfar_priv_grp, napi_tx);
- struct gfar_private *priv = gfargrp->priv;
- struct gfar __iomem *regs = gfargrp->regs;
- struct gfar_priv_tx_q *tx_queue = NULL;
- int has_tx_work = 0;
- int i;
-
- /* Clear IEVENT, so interrupts aren't called again
- * because of the packets that have already arrived
- */
- gfar_write(®s->ievent, IEVENT_TX_MASK);
-
- for_each_set_bit(i, &gfargrp->tx_bit_map, priv->num_tx_queues) {
- tx_queue = priv->tx_queue[i];
- /* run Tx cleanup to completion */
- if (tx_queue->tx_skbuff[tx_queue->skb_dirtytx]) {
- gfar_clean_tx_ring(tx_queue);
- has_tx_work = 1;
- }
- }
-
- if (!has_tx_work) {
- u32 imask;
- napi_complete(napi);
-
- spin_lock_irq(&gfargrp->grplock);
- imask = gfar_read(®s->imask);
- imask |= IMASK_TX_DEFAULT;
- gfar_write(®s->imask, imask);
- spin_unlock_irq(&gfargrp->grplock);
- }
-
- return 0;
-}
-
-
-#ifdef CONFIG_NET_POLL_CONTROLLER
-/* Polling 'interrupt' - used by things like netconsole to send skbs
- * without having to re-enable interrupts. It's not called while
- * the interrupt routine is executing.
- */
-static void gfar_netpoll(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
- int i;
-
- /* If the device has multiple interrupts, run tx/rx */
- if (priv->device_flags & FSL_GIANFAR_DEV_HAS_MULTI_INTR) {
- for (i = 0; i < priv->num_grps; i++) {
- struct gfar_priv_grp *grp = &priv->gfargrp[i];
-
- disable_irq(gfar_irq(grp, TX)->irq);
- disable_irq(gfar_irq(grp, RX)->irq);
- disable_irq(gfar_irq(grp, ER)->irq);
- gfar_interrupt(gfar_irq(grp, TX)->irq, grp);
- enable_irq(gfar_irq(grp, ER)->irq);
- enable_irq(gfar_irq(grp, RX)->irq);
- enable_irq(gfar_irq(grp, TX)->irq);
- }
- } else {
- for (i = 0; i < priv->num_grps; i++) {
- struct gfar_priv_grp *grp = &priv->gfargrp[i];
-
- disable_irq(gfar_irq(grp, TX)->irq);
- gfar_interrupt(gfar_irq(grp, TX)->irq, grp);
- enable_irq(gfar_irq(grp, TX)->irq);
- }
- }
-}
-#endif
-
-/* The interrupt handler for devices with one interrupt */
-static irqreturn_t gfar_interrupt(int irq, void *grp_id)
-{
- struct gfar_priv_grp *gfargrp = grp_id;
-
- /* Save ievent for future reference */
- u32 events = gfar_read(&gfargrp->regs->ievent);
-
- /* Check for reception */
- if (events & IEVENT_RX_MASK)
- gfar_receive(irq, grp_id);
-
- /* Check for transmit completion */
- if (events & IEVENT_TX_MASK)
- gfar_transmit(irq, grp_id);
-
- /* Check for errors */
- if (events & IEVENT_ERR_MASK)
- gfar_error(irq, grp_id);
-
- return IRQ_HANDLED;
-}
-
-/* Called every time the controller might need to be made
- * aware of new link state. The PHY code conveys this
- * information through variables in the phydev structure, and this
- * function converts those variables into the appropriate
- * register values, and can bring down the device if needed.
- */
-static void adjust_link(struct net_device *dev)
-{
- struct gfar_private *priv = netdev_priv(dev);
- struct phy_device *phydev = dev->phydev;
-
- if (unlikely(phydev->link != priv->oldlink ||
- (phydev->link && (phydev->duplex != priv->oldduplex ||
- phydev->speed != priv->oldspeed))))
- gfar_update_link_state(priv);
-}
-
-/* Update the hash table based on the current list of multicast
- * addresses we subscribe to. Also, change the promiscuity of
- * the device based on the flags (this function is called
- * whenever dev->flags is changed
- */
-static void gfar_set_multi(struct net_device *dev)
-{
- struct netdev_hw_addr *ha;
- struct gfar_private *priv = netdev_priv(dev);
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- u32 tempval;
-
- if (dev->flags & IFF_PROMISC) {
- /* Set RCTRL to PROM */
- tempval = gfar_read(®s->rctrl);
- tempval |= RCTRL_PROM;
- gfar_write(®s->rctrl, tempval);
- } else {
- /* Set RCTRL to not PROM */
- tempval = gfar_read(®s->rctrl);
- tempval &= ~(RCTRL_PROM);
- gfar_write(®s->rctrl, tempval);
- }
-
- if (dev->flags & IFF_ALLMULTI) {
- /* Set the hash to rx all multicast frames */
- gfar_write(®s->igaddr0, 0xffffffff);
- gfar_write(®s->igaddr1, 0xffffffff);
- gfar_write(®s->igaddr2, 0xffffffff);
- gfar_write(®s->igaddr3, 0xffffffff);
- gfar_write(®s->igaddr4, 0xffffffff);
- gfar_write(®s->igaddr5, 0xffffffff);
- gfar_write(®s->igaddr6, 0xffffffff);
- gfar_write(®s->igaddr7, 0xffffffff);
- gfar_write(®s->gaddr0, 0xffffffff);
- gfar_write(®s->gaddr1, 0xffffffff);
- gfar_write(®s->gaddr2, 0xffffffff);
- gfar_write(®s->gaddr3, 0xffffffff);
- gfar_write(®s->gaddr4, 0xffffffff);
- gfar_write(®s->gaddr5, 0xffffffff);
- gfar_write(®s->gaddr6, 0xffffffff);
- gfar_write(®s->gaddr7, 0xffffffff);
- } else {
- int em_num;
- int idx;
-
- /* zero out the hash */
- gfar_write(®s->igaddr0, 0x0);
- gfar_write(®s->igaddr1, 0x0);
- gfar_write(®s->igaddr2, 0x0);
- gfar_write(®s->igaddr3, 0x0);
- gfar_write(®s->igaddr4, 0x0);
- gfar_write(®s->igaddr5, 0x0);
- gfar_write(®s->igaddr6, 0x0);
- gfar_write(®s->igaddr7, 0x0);
- gfar_write(®s->gaddr0, 0x0);
- gfar_write(®s->gaddr1, 0x0);
- gfar_write(®s->gaddr2, 0x0);
- gfar_write(®s->gaddr3, 0x0);
- gfar_write(®s->gaddr4, 0x0);
- gfar_write(®s->gaddr5, 0x0);
- gfar_write(®s->gaddr6, 0x0);
- gfar_write(®s->gaddr7, 0x0);
-
- /* If we have extended hash tables, we need to
- * clear the exact match registers to prepare for
- * setting them
- */
- if (priv->extended_hash) {
- em_num = GFAR_EM_NUM + 1;
- gfar_clear_exact_match(dev);
- idx = 1;
- } else {
- idx = 0;
- em_num = 0;
- }
-
- if (netdev_mc_empty(dev))
- return;
-
- /* Parse the list, and set the appropriate bits */
- netdev_for_each_mc_addr(ha, dev) {
- if (idx < em_num) {
- gfar_set_mac_for_addr(dev, idx, ha->addr);
- idx++;
- } else
- gfar_set_hash_for_addr(dev, ha->addr);
- }
- }
-}
-
-
-/* Clears each of the exact match registers to zero, so they
- * don't interfere with normal reception
- */
-static void gfar_clear_exact_match(struct net_device *dev)
-{
- int idx;
- static const u8 zero_arr[ETH_ALEN] = {0, 0, 0, 0, 0, 0};
-
- for (idx = 1; idx < GFAR_EM_NUM + 1; idx++)
- gfar_set_mac_for_addr(dev, idx, zero_arr);
-}
-
-/* Set the appropriate hash bit for the given addr */
-/* The algorithm works like so:
- * 1) Take the Destination Address (ie the multicast address), and
- * do a CRC on it (little endian), and reverse the bits of the
- * result.
- * 2) Use the 8 most significant bits as a hash into a 256-entry
- * table. The table is controlled through 8 32-bit registers:
- * gaddr0-7. gaddr0's MSB is entry 0, and gaddr7's LSB is
- * gaddr7. This means that the 3 most significant bits in the
- * hash index which gaddr register to use, and the 5 other bits
- * indicate which bit (assuming an IBM numbering scheme, which
- * for PowerPC (tm) is usually the case) in the register holds
- * the entry.
- */
-static void gfar_set_hash_for_addr(struct net_device *dev, u8 *addr)
-{
- u32 tempval;
- struct gfar_private *priv = netdev_priv(dev);
- u32 result = ether_crc(ETH_ALEN, addr);
- int width = priv->hash_width;
- u8 whichbit = (result >> (32 - width)) & 0x1f;
- u8 whichreg = result >> (32 - width + 5);
- u32 value = (1 << (31-whichbit));
-
- tempval = gfar_read(priv->hash_regs[whichreg]);
- tempval |= value;
- gfar_write(priv->hash_regs[whichreg], tempval);
-}
-
-
-/* There are multiple MAC Address register pairs on some controllers
- * This function sets the numth pair to a given address
- */
-static void gfar_set_mac_for_addr(struct net_device *dev, int num,
- const u8 *addr)
-{
- struct gfar_private *priv = netdev_priv(dev);
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- u32 tempval;
- u32 __iomem *macptr = ®s->macstnaddr1;
-
- macptr += num*2;
-
- /* For a station address of 0x12345678ABCD in transmission
- * order (BE), MACnADDR1 is set to 0xCDAB7856 and
- * MACnADDR2 is set to 0x34120000.
- */
- tempval = (addr[5] << 24) | (addr[4] << 16) |
- (addr[3] << 8) | addr[2];
-
- gfar_write(macptr, tempval);
-
- tempval = (addr[1] << 24) | (addr[0] << 16);
-
- gfar_write(macptr+1, tempval);
-}
-
-/* GFAR error interrupt handler */
-static irqreturn_t gfar_error(int irq, void *grp_id)
-{
- struct gfar_priv_grp *gfargrp = grp_id;
- struct gfar __iomem *regs = gfargrp->regs;
- struct gfar_private *priv= gfargrp->priv;
- struct net_device *dev = priv->ndev;
-
- /* Save ievent for future reference */
- u32 events = gfar_read(®s->ievent);
-
- /* Clear IEVENT */
- gfar_write(®s->ievent, events & IEVENT_ERR_MASK);
-
- /* Magic Packet is not an error. */
- if ((priv->device_flags & FSL_GIANFAR_DEV_HAS_MAGIC_PACKET) &&
- (events & IEVENT_MAG))
- events &= ~IEVENT_MAG;
-
- /* Hmm... */
- if (netif_msg_rx_err(priv) || netif_msg_tx_err(priv))
- netdev_dbg(dev,
- "error interrupt (ievent=0x%08x imask=0x%08x)\n",
- events, gfar_read(®s->imask));
-
- /* Update the error counters */
- if (events & IEVENT_TXE) {
- dev->stats.tx_errors++;
-
- if (events & IEVENT_LC)
- dev->stats.tx_window_errors++;
- if (events & IEVENT_CRL)
- dev->stats.tx_aborted_errors++;
- if (events & IEVENT_XFUN) {
- netif_dbg(priv, tx_err, dev,
- "TX FIFO underrun, packet dropped\n");
- dev->stats.tx_dropped++;
- atomic64_inc(&priv->extra_stats.tx_underrun);
-
- schedule_work(&priv->reset_task);
- }
- netif_dbg(priv, tx_err, dev, "Transmit Error\n");
- }
- if (events & IEVENT_BSY) {
- dev->stats.rx_over_errors++;
- atomic64_inc(&priv->extra_stats.rx_bsy);
-
- netif_dbg(priv, rx_err, dev, "busy error (rstat: %x)\n",
- gfar_read(®s->rstat));
- }
- if (events & IEVENT_BABR) {
- dev->stats.rx_errors++;
- atomic64_inc(&priv->extra_stats.rx_babr);
-
- netif_dbg(priv, rx_err, dev, "babbling RX error\n");
- }
- if (events & IEVENT_EBERR) {
- atomic64_inc(&priv->extra_stats.eberr);
- netif_dbg(priv, rx_err, dev, "bus error\n");
- }
- if (events & IEVENT_RXC)
- netif_dbg(priv, rx_status, dev, "control frame\n");
-
- if (events & IEVENT_BABT) {
- atomic64_inc(&priv->extra_stats.tx_babt);
- netif_dbg(priv, tx_err, dev, "babbling TX error\n");
- }
- return IRQ_HANDLED;
-}
-
-static u32 gfar_get_flowctrl_cfg(struct gfar_private *priv)
-{
- struct net_device *ndev = priv->ndev;
- struct phy_device *phydev = ndev->phydev;
- u32 val = 0;
-
- if (!phydev->duplex)
- return val;
-
- if (!priv->pause_aneg_en) {
- if (priv->tx_pause_en)
- val |= MACCFG1_TX_FLOW;
- if (priv->rx_pause_en)
- val |= MACCFG1_RX_FLOW;
- } else {
- u16 lcl_adv, rmt_adv;
- u8 flowctrl;
- /* get link partner capabilities */
- rmt_adv = 0;
- if (phydev->pause)
- rmt_adv = LPA_PAUSE_CAP;
- if (phydev->asym_pause)
- rmt_adv |= LPA_PAUSE_ASYM;
-
- lcl_adv = linkmode_adv_to_lcl_adv_t(phydev->advertising);
- flowctrl = mii_resolve_flowctrl_fdx(lcl_adv, rmt_adv);
- if (flowctrl & FLOW_CTRL_TX)
- val |= MACCFG1_TX_FLOW;
- if (flowctrl & FLOW_CTRL_RX)
- val |= MACCFG1_RX_FLOW;
- }
-
- return val;
-}
-
-static noinline void gfar_update_link_state(struct gfar_private *priv)
-{
- struct gfar __iomem *regs = priv->gfargrp[0].regs;
- struct net_device *ndev = priv->ndev;
- struct phy_device *phydev = ndev->phydev;
- struct gfar_priv_rx_q *rx_queue = NULL;
- int i;
-
- if (unlikely(test_bit(GFAR_RESETTING, &priv->state)))
- return;
-
- if (phydev->link) {
- u32 tempval1 = gfar_read(®s->maccfg1);
- u32 tempval = gfar_read(®s->maccfg2);
- u32 ecntrl = gfar_read(®s->ecntrl);
- u32 tx_flow_oldval = (tempval1 & MACCFG1_TX_FLOW);
-
- if (phydev->duplex != priv->oldduplex) {
- if (!(phydev->duplex))
- tempval &= ~(MACCFG2_FULL_DUPLEX);
- else
- tempval |= MACCFG2_FULL_DUPLEX;
-
- priv->oldduplex = phydev->duplex;
- }
-
- if (phydev->speed != priv->oldspeed) {
- switch (phydev->speed) {
- case 1000:
- tempval =
- ((tempval & ~(MACCFG2_IF)) | MACCFG2_GMII);
-
- ecntrl &= ~(ECNTRL_R100);
- break;
- case 100:
- case 10:
- tempval =
- ((tempval & ~(MACCFG2_IF)) | MACCFG2_MII);
-
- /* Reduced mode distinguishes
- * between 10 and 100
- */
- if (phydev->speed == SPEED_100)
- ecntrl |= ECNTRL_R100;
- else
- ecntrl &= ~(ECNTRL_R100);
- break;
- default:
- netif_warn(priv, link, priv->ndev,
- "Ack! Speed (%d) is not 10/100/1000!\n",
- phydev->speed);
- break;
- }
-
- priv->oldspeed = phydev->speed;
- }
-
- tempval1 &= ~(MACCFG1_TX_FLOW | MACCFG1_RX_FLOW);
- tempval1 |= gfar_get_flowctrl_cfg(priv);
-
- /* Turn last free buffer recording on */
- if ((tempval1 & MACCFG1_TX_FLOW) && !tx_flow_oldval) {
- for (i = 0; i < priv->num_rx_queues; i++) {
- u32 bdp_dma;
-
- rx_queue = priv->rx_queue[i];
- bdp_dma = gfar_rxbd_dma_lastfree(rx_queue);
- gfar_write(rx_queue->rfbptr, bdp_dma);
- }
-
- priv->tx_actual_en = 1;
- }
-
- if (unlikely(!(tempval1 & MACCFG1_TX_FLOW) && tx_flow_oldval))
- priv->tx_actual_en = 0;
-
- gfar_write(®s->maccfg1, tempval1);
- gfar_write(®s->maccfg2, tempval);
- gfar_write(®s->ecntrl, ecntrl);
-
- if (!priv->oldlink)
- priv->oldlink = 1;
-
- } else if (priv->oldlink) {
- priv->oldlink = 0;
- priv->oldspeed = 0;
- priv->oldduplex = -1;
- }
-
- if (netif_msg_link(priv))
- phy_print_status(phydev);
-}
-
static const struct of_device_id gfar_match[] =
{
{
diff --git a/drivers/net/ethernet/freescale/gianfar.h b/drivers/net/ethernet/freescale/gianfar.h
index f2af963..f472a6d 100644
--- a/drivers/net/ethernet/freescale/gianfar.h
+++ b/drivers/net/ethernet/freescale/gianfar.h
@@ -67,8 +67,6 @@ struct ethtool_rx_list {
/* Number of bytes to align the rx bufs to */
#define RXBUF_ALIGNMENT 64
-#define PHY_INIT_TIMEOUT 100000
-
#define DRV_NAME "gfar-enet"
extern const char gfar_driver_version[];
@@ -88,10 +86,6 @@ extern const char gfar_driver_version[];
#define GFAR_RX_MAX_RING_SIZE 256
#define GFAR_TX_MAX_RING_SIZE 256
-#define GFAR_MAX_FIFO_THRESHOLD 511
-#define GFAR_MAX_FIFO_STARVE 511
-#define GFAR_MAX_FIFO_STARVE_OFF 511
-
#define FBTHR_SHIFT 24
#define DEFAULT_RX_LFC_THR 16
#define DEFAULT_LFC_PTVVAL 4
@@ -109,9 +103,6 @@ extern const char gfar_driver_version[];
#define DEFAULT_FIFO_TX_THR 0x100
#define DEFAULT_FIFO_TX_STARVE 0x40
#define DEFAULT_FIFO_TX_STARVE_OFF 0x80
-#define DEFAULT_BD_STASH 1
-#define DEFAULT_STASH_LENGTH 96
-#define DEFAULT_STASH_INDEX 0
/* The number of Exact Match registers */
#define GFAR_EM_NUM 15
@@ -139,15 +130,6 @@ extern const char gfar_driver_version[];
#define DEFAULT_RX_COALESCE 0
#define DEFAULT_RXCOUNT 0
-#define GFAR_SUPPORTED (SUPPORTED_10baseT_Half \
- | SUPPORTED_10baseT_Full \
- | SUPPORTED_100baseT_Half \
- | SUPPORTED_100baseT_Full \
- | SUPPORTED_Autoneg \
- | SUPPORTED_MII)
-
-#define GFAR_SUPPORTED_GBIT SUPPORTED_1000baseT_Full
-
/* TBI register addresses */
#define MII_TBICON 0x11
@@ -185,8 +167,6 @@ extern const char gfar_driver_version[];
#define ECNTRL_REDUCED_MII_MODE 0x00000004
#define ECNTRL_SGMII_MODE 0x00000002
-#define MRBLR_INIT_SETTINGS DEFAULT_RX_BUFFER_SIZE
-
#define MINFLR_INIT_SETTINGS 0x00000040
/* Tqueue control */
@@ -266,12 +246,6 @@ extern const char gfar_driver_version[];
#define DEFAULT_TXIC mk_ic_value(DEFAULT_TXCOUNT, DEFAULT_TXTIME)
#define DEFAULT_RXIC mk_ic_value(DEFAULT_RXCOUNT, DEFAULT_RXTIME)
-#define skip_bd(bdp, stride, base, ring_size) ({ \
- typeof(bdp) new_bd = (bdp) + (stride); \
- (new_bd >= (base) + (ring_size)) ? (new_bd - (ring_size)) : new_bd; })
-
-#define next_bd(bdp, base, ring_size) skip_bd(bdp, 1, base, ring_size)
-
#define RCTRL_TS_ENABLE 0x01000000
#define RCTRL_PAL_MASK 0x001f0000
#define RCTRL_LFC 0x00004000
@@ -385,11 +359,6 @@ extern const char gfar_driver_version[];
#define IMASK_RX_DISABLED ((~(IMASK_RX_DEFAULT)) & IMASK_DEFAULT)
#define IMASK_TX_DISABLED ((~(IMASK_TX_DEFAULT)) & IMASK_DEFAULT)
-/* Fifo management */
-#define FIFO_TX_THR_MASK 0x01ff
-#define FIFO_TX_STARVE_MASK 0x01ff
-#define FIFO_TX_STARVE_OFF_MASK 0x01ff
-
/* Attribute fields */
/* This enables rx snooping for buffers and descriptors */
@@ -1326,16 +1295,9 @@ static inline u32 gfar_rxbd_dma_lastfree(struct gfar_priv_rx_q *rxq)
return bdp_dma;
}
-irqreturn_t gfar_receive(int irq, void *dev_id);
int startup_gfar(struct net_device *dev);
void stop_gfar(struct net_device *dev);
-void reset_gfar(struct net_device *dev);
void gfar_mac_reset(struct gfar_private *priv);
-void gfar_halt(struct gfar_private *priv);
-void gfar_start(struct gfar_private *priv);
-void gfar_phy_test(struct mii_bus *bus, struct phy_device *phydev, int enable,
- u32 regnum, u32 read);
-void gfar_configure_coalescing_all(struct gfar_private *priv);
int gfar_set_features(struct net_device *dev, netdev_features_t features);
extern const struct ethtool_ops gfar_ethtool_ops;
@@ -1348,13 +1310,6 @@ extern const struct ethtool_ops gfar_ethtool_ops;
#define RQFCR_PID_PORT_MASK 0xFFFF0000
#define RQFCR_PID_MAC_MASK 0xFF000000
-struct gfar_mask_entry {
- unsigned int mask; /* The mask value which is valid form start to end */
- unsigned int start;
- unsigned int end;
- unsigned int block; /* Same block values indicate depended entries */
-};
-
/* Represents a receive filer table entry */
struct gfar_filer_entry {
u32 ctrl;
diff --git a/drivers/net/ethernet/freescale/gianfar_ethtool.c b/drivers/net/ethernet/freescale/gianfar_ethtool.c
index 3433b46..3c8e4e2 100644
--- a/drivers/net/ethernet/freescale/gianfar_ethtool.c
+++ b/drivers/net/ethernet/freescale/gianfar_ethtool.c
@@ -45,19 +45,6 @@
#define GFAR_MAX_COAL_USECS 0xffff
#define GFAR_MAX_COAL_FRAMES 0xff
-static void gfar_fill_stats(struct net_device *dev, struct ethtool_stats *dummy,
- u64 *buf);
-static void gfar_gstrings(struct net_device *dev, u32 stringset, u8 * buf);
-static int gfar_gcoalesce(struct net_device *dev,
- struct ethtool_coalesce *cvals);
-static int gfar_scoalesce(struct net_device *dev,
- struct ethtool_coalesce *cvals);
-static void gfar_gringparam(struct net_device *dev,
- struct ethtool_ringparam *rvals);
-static int gfar_sringparam(struct net_device *dev,
- struct ethtool_ringparam *rvals);
-static void gfar_gdrvinfo(struct net_device *dev,
- struct ethtool_drvinfo *drvinfo);
static const char stat_gstrings[][ETH_GSTRING_LEN] = {
/* extra stats */
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index 4972987..aca95f6 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -50,7 +50,7 @@ static void gve_get_stats(struct net_device *dev, struct rtnl_link_stats64 *s)
u64_stats_fetch_begin(&priv->tx[ring].statss);
s->tx_packets += priv->tx[ring].pkt_done;
s->tx_bytes += priv->tx[ring].bytes_done;
- } while (u64_stats_fetch_retry(&priv->rx[ring].statss,
+ } while (u64_stats_fetch_retry(&priv->tx[ring].statss,
start));
}
}
diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c
index c1eba42..3a14bbc 100644
--- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c
+++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c
@@ -24,7 +24,7 @@
#include "hns_dsaf_rcb.h"
#include "hns_dsaf_misc.h"
-const static char *g_dsaf_mode_match[DSAF_MODE_MAX] = {
+static const char *g_dsaf_mode_match[DSAF_MODE_MAX] = {
[DSAF_MODE_DISABLE_2PORT_64VM] = "2port-64vf",
[DSAF_MODE_DISABLE_6PORT_0VM] = "6port-16rss",
[DSAF_MODE_DISABLE_6PORT_16VM] = "6port-16vf",
diff --git a/drivers/net/ethernet/hisilicon/hns3/hnae3.c b/drivers/net/ethernet/hisilicon/hns3/hnae3.c
index 528f624..03ca7d9 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hnae3.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hnae3.c
@@ -138,12 +138,28 @@ EXPORT_SYMBOL(hnae3_register_client);
void hnae3_unregister_client(struct hnae3_client *client)
{
+ struct hnae3_client *client_tmp;
struct hnae3_ae_dev *ae_dev;
+ bool existed = false;
if (!client)
return;
mutex_lock(&hnae3_common_lock);
+
+ list_for_each_entry(client_tmp, &hnae3_client_list, node) {
+ if (client_tmp->type == client->type) {
+ existed = true;
+ break;
+ }
+ }
+
+ if (!existed) {
+ mutex_unlock(&hnae3_common_lock);
+ pr_err("client %s does not exist!\n", client->name);
+ return;
+ }
+
/* un-initialize the client on every matched port */
list_for_each_entry(ae_dev, &hnae3_ae_dev_list, node) {
hnae3_uninit_client_instance(client, ae_dev);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hnae3.h b/drivers/net/ethernet/hisilicon/hns3/hnae3.h
index 3e21533..c4b7bf8 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hnae3.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hnae3.h
@@ -146,6 +146,12 @@ enum hnae3_reset_notify_type {
HNAE3_RESTORE_CLIENT,
};
+enum hnae3_hw_error_type {
+ HNAE3_PPU_POISON_ERROR,
+ HNAE3_CMDQ_ECC_ERROR,
+ HNAE3_IMP_RD_POISON_ERROR,
+};
+
enum hnae3_reset_type {
HNAE3_VF_RESET,
HNAE3_VF_FUNC_RESET,
@@ -210,7 +216,8 @@ struct hnae3_client_ops {
int (*setup_tc)(struct hnae3_handle *handle, u8 tc);
int (*reset_notify)(struct hnae3_handle *handle,
enum hnae3_reset_notify_type type);
- enum hnae3_reset_type (*process_hw_error)(struct hnae3_handle *handle);
+ void (*process_hw_error)(struct hnae3_handle *handle,
+ enum hnae3_hw_error_type);
};
#define HNAE3_CLIENT_NAME_LENGTH 16
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
index 7070d25..28961a6 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c
@@ -39,7 +39,7 @@ static int hns3_dbg_queue_info(struct hnae3_handle *h,
if (queue_num >= h->kinfo.num_tqps) {
dev_err(&h->pdev->dev,
- "Queue number(%u) is out of range(%u)\n", queue_num,
+ "Queue number(%u) is out of range(0-%u)\n", queue_num,
h->kinfo.num_tqps - 1);
return -EINVAL;
}
@@ -166,6 +166,7 @@ static int hns3_dbg_bd_info(struct hnae3_handle *h, const char *cmd_buf)
struct hns3_enet_ring *ring;
u32 tx_index, rx_index;
u32 q_num, value;
+ dma_addr_t addr;
int cnt;
cnt = sscanf(&cmd_buf[8], "%u %u", &q_num, &tx_index);
@@ -177,7 +178,7 @@ static int hns3_dbg_bd_info(struct hnae3_handle *h, const char *cmd_buf)
}
if (q_num >= h->kinfo.num_tqps) {
- dev_err(dev, "Queue number(%u) is out of range(%u)\n", q_num,
+ dev_err(dev, "Queue number(%u) is out of range(0-%u)\n", q_num,
h->kinfo.num_tqps - 1);
return -EINVAL;
}
@@ -188,14 +189,15 @@ static int hns3_dbg_bd_info(struct hnae3_handle *h, const char *cmd_buf)
tx_index = (cnt == 1) ? value : tx_index;
if (tx_index >= ring->desc_num) {
- dev_err(dev, "bd index (%u) is out of range(%u)\n", tx_index,
+ dev_err(dev, "bd index(%u) is out of range(0-%u)\n", tx_index,
ring->desc_num - 1);
return -EINVAL;
}
tx_desc = &ring->desc[tx_index];
+ addr = le64_to_cpu(tx_desc->addr);
dev_info(dev, "TX Queue Num: %u, BD Index: %u\n", q_num, tx_index);
- dev_info(dev, "(TX) addr: 0x%llx\n", tx_desc->addr);
+ dev_info(dev, "(TX)addr: %pad\n", &addr);
dev_info(dev, "(TX)vlan_tag: %u\n", tx_desc->tx.vlan_tag);
dev_info(dev, "(TX)send_size: %u\n", tx_desc->tx.send_size);
dev_info(dev, "(TX)vlan_tso: %u\n", tx_desc->tx.type_cs_vlan_tso);
@@ -217,8 +219,10 @@ static int hns3_dbg_bd_info(struct hnae3_handle *h, const char *cmd_buf)
rx_index = (cnt == 1) ? value : tx_index;
rx_desc = &ring->desc[rx_index];
+ addr = le64_to_cpu(rx_desc->addr);
dev_info(dev, "RX Queue Num: %u, BD Index: %u\n", q_num, rx_index);
- dev_info(dev, "(RX)addr: 0x%llx\n", rx_desc->addr);
+ dev_info(dev, "(RX)addr: %pad\n", &addr);
+ dev_info(dev, "(RX)l234_info: %u\n", rx_desc->rx.l234_info);
dev_info(dev, "(RX)pkt_len: %u\n", rx_desc->rx.pkt_len);
dev_info(dev, "(RX)size: %u\n", rx_desc->rx.size);
dev_info(dev, "(RX)rss_hash: %u\n", rx_desc->rx.rss_hash);
@@ -238,16 +242,16 @@ static void hns3_dbg_help(struct hnae3_handle *h)
char printf_buf[HNS3_DBG_BUF_LEN];
dev_info(&h->pdev->dev, "available commands\n");
- dev_info(&h->pdev->dev, "queue info [number]\n");
+ dev_info(&h->pdev->dev, "queue info <number>\n");
dev_info(&h->pdev->dev, "queue map\n");
- dev_info(&h->pdev->dev, "bd info [q_num] <bd index>\n");
+ dev_info(&h->pdev->dev, "bd info <q_num> <bd index>\n");
if (!hns3_is_phys_func(h->pdev))
return;
dev_info(&h->pdev->dev, "dump fd tcam\n");
dev_info(&h->pdev->dev, "dump tc\n");
- dev_info(&h->pdev->dev, "dump tm map [q_num]\n");
+ dev_info(&h->pdev->dev, "dump tm map <q_num>\n");
dev_info(&h->pdev->dev, "dump tm\n");
dev_info(&h->pdev->dev, "dump qos pause cfg\n");
dev_info(&h->pdev->dev, "dump qos pri map\n");
@@ -259,20 +263,20 @@ static void hns3_dbg_help(struct hnae3_handle *h)
dev_info(&h->pdev->dev, "dump mac tnl status\n");
memset(printf_buf, 0, HNS3_DBG_BUF_LEN);
- strncat(printf_buf, "dump reg [[bios common] [ssu <prt_id>]",
+ strncat(printf_buf, "dump reg [[bios common] [ssu <port_id>]",
HNS3_DBG_BUF_LEN - 1);
strncat(printf_buf + strlen(printf_buf),
- " [igu egu <prt_id>] [rpu <tc_queue_num>]",
+ " [igu egu <port_id>] [rpu <tc_queue_num>]",
HNS3_DBG_BUF_LEN - strlen(printf_buf) - 1);
strncat(printf_buf + strlen(printf_buf),
- " [rtc] [ppp] [rcb] [tqp <q_num>]]\n",
+ " [rtc] [ppp] [rcb] [tqp <queue_num>]]\n",
HNS3_DBG_BUF_LEN - strlen(printf_buf) - 1);
dev_info(&h->pdev->dev, "%s", printf_buf);
memset(printf_buf, 0, HNS3_DBG_BUF_LEN);
- strncat(printf_buf, "dump reg dcb [port_id] [pri_id] [pg_id]",
+ strncat(printf_buf, "dump reg dcb <port_id> <pri_id> <pg_id>",
HNS3_DBG_BUF_LEN - 1);
- strncat(printf_buf + strlen(printf_buf), " [rq_id] [nq_id] [qset_id]\n",
+ strncat(printf_buf + strlen(printf_buf), " <rq_id> <nq_id> <qset_id>\n",
HNS3_DBG_BUF_LEN - strlen(printf_buf) - 1);
dev_info(&h->pdev->dev, "%s", printf_buf);
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
index a11d514..616cad0 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.c
@@ -2006,7 +2006,8 @@ static pci_ers_result_t hns3_slot_reset(struct pci_dev *pdev)
ops = ae_dev->ops;
/* request the reset */
- if (ops->reset_event && ops->get_reset_level) {
+ if (ops->reset_event && ops->get_reset_level &&
+ ops->set_default_reset_request) {
if (ae_dev->hw_err_reset_req) {
reset_type = ops->get_reset_level(ae_dev,
&ae_dev->hw_err_reset_req);
@@ -4410,6 +4411,30 @@ static int hns3_reset_notify(struct hnae3_handle *handle,
return ret;
}
+static int hns3_change_channels(struct hnae3_handle *handle, u32 new_tqp_num,
+ bool rxfh_configured)
+{
+ int ret;
+
+ ret = handle->ae_algo->ops->set_channels(handle, new_tqp_num,
+ rxfh_configured);
+ if (ret) {
+ dev_err(&handle->pdev->dev,
+ "Change tqp num(%u) fail.\n", new_tqp_num);
+ return ret;
+ }
+
+ ret = hns3_reset_notify(handle, HNAE3_INIT_CLIENT);
+ if (ret)
+ return ret;
+
+ ret = hns3_reset_notify(handle, HNAE3_UP_CLIENT);
+ if (ret)
+ hns3_reset_notify(handle, HNAE3_UNINIT_CLIENT);
+
+ return ret;
+}
+
int hns3_set_channels(struct net_device *netdev,
struct ethtool_channels *ch)
{
@@ -4450,24 +4475,46 @@ int hns3_set_channels(struct net_device *netdev,
return ret;
org_tqp_num = h->kinfo.num_tqps;
- ret = h->ae_algo->ops->set_channels(h, new_tqp_num, rxfh_configured);
+ ret = hns3_change_channels(h, new_tqp_num, rxfh_configured);
if (ret) {
- ret = h->ae_algo->ops->set_channels(h, org_tqp_num,
- rxfh_configured);
- if (ret) {
- /* If revert to old tqp failed, fatal error occurred */
- dev_err(&netdev->dev,
- "Revert to old tqp num fail, ret=%d", ret);
- return ret;
- }
- dev_info(&netdev->dev,
- "Change tqp num fail, Revert to old tqp num");
- }
- ret = hns3_reset_notify(h, HNAE3_INIT_CLIENT);
- if (ret)
- return ret;
+ int ret1;
- return hns3_reset_notify(h, HNAE3_UP_CLIENT);
+ netdev_warn(netdev,
+ "Change channels fail, revert to old value\n");
+ ret1 = hns3_change_channels(h, org_tqp_num, rxfh_configured);
+ if (ret1) {
+ netdev_err(netdev,
+ "revert to old channel fail\n");
+ return ret1;
+ }
+
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct hns3_hw_error_info hns3_hw_err[] = {
+ { .type = HNAE3_PPU_POISON_ERROR,
+ .msg = "PPU poison" },
+ { .type = HNAE3_CMDQ_ECC_ERROR,
+ .msg = "IMP CMDQ error" },
+ { .type = HNAE3_IMP_RD_POISON_ERROR,
+ .msg = "IMP RD poison" },
+};
+
+static void hns3_process_hw_error(struct hnae3_handle *handle,
+ enum hnae3_hw_error_type type)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(hns3_hw_err); i++) {
+ if (hns3_hw_err[i].type == type) {
+ dev_err(&handle->pdev->dev, "Detected %s!\n",
+ hns3_hw_err[i].msg);
+ break;
+ }
+ }
}
static const struct hnae3_client_ops client_ops = {
@@ -4476,6 +4523,7 @@ static const struct hnae3_client_ops client_ops = {
.link_status_change = hns3_link_status_change,
.setup_tc = hns3_client_setup_tc,
.reset_notify = hns3_reset_notify,
+ .process_hw_error = hns3_process_hw_error,
};
/* hns3_init_module - Driver registration routine
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
index e37e64e..2110fa3 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h
@@ -552,6 +552,11 @@ union l4_hdr_info {
unsigned char *hdr;
};
+struct hns3_hw_error_info {
+ enum hnae3_hw_error_type type;
+ const char *msg;
+};
+
static inline int ring_space(struct hns3_enet_ring *ring)
{
/* This smp_load_acquire() pairs with smp_store_release() in
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
index 0332d6f..680c350 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c
@@ -59,7 +59,7 @@ static const struct hns3_stats hns3_rxq_stats[] = {
#define HNS3_TQP_STATS_COUNT (HNS3_TXQ_STATS_COUNT + HNS3_RXQ_STATS_COUNT)
-#define HNS3_SELF_TEST_TYPE_NUM 3
+#define HNS3_SELF_TEST_TYPE_NUM 4
#define HNS3_NIC_LB_TEST_PKT_NUM 1
#define HNS3_NIC_LB_TEST_RING_ID 0
#define HNS3_NIC_LB_TEST_PACKET_SIZE 128
@@ -89,6 +89,7 @@ static int hns3_lp_setup(struct net_device *ndev, enum hnae3_loop loop, bool en)
case HNAE3_LOOP_SERIAL_SERDES:
case HNAE3_LOOP_PARALLEL_SERDES:
case HNAE3_LOOP_APP:
+ case HNAE3_LOOP_PHY:
ret = h->ae_algo->ops->set_loopback(h, loop, en);
break;
default:
@@ -96,7 +97,7 @@ static int hns3_lp_setup(struct net_device *ndev, enum hnae3_loop loop, bool en)
break;
}
- if (ret)
+ if (ret || h->pdev->revision >= 0x21)
return ret;
if (en) {
@@ -143,7 +144,10 @@ static int hns3_lp_down(struct net_device *ndev, enum hnae3_loop loop_mode)
static void hns3_lp_setup_skb(struct sk_buff *skb)
{
+#define HNS3_NIC_LB_DST_MAC_ADDR 0x1f
+
struct net_device *ndev = skb->dev;
+ struct hnae3_handle *handle;
unsigned char *packet;
struct ethhdr *ethh;
unsigned int i;
@@ -159,7 +163,9 @@ static void hns3_lp_setup_skb(struct sk_buff *skb)
* before the packet reaches mac or serdes, which will defect
* the purpose of mac or serdes selftest.
*/
- ethh->h_dest[5] += 0x1f;
+ handle = hns3_get_handle(ndev);
+ if (handle->pdev->revision == 0x20)
+ ethh->h_dest[5] += HNS3_NIC_LB_DST_MAC_ADDR;
eth_zero_addr(ethh->h_source);
ethh->h_proto = htons(ETH_P_ARP);
skb_reset_mac_header(skb);
@@ -330,6 +336,10 @@ static void hns3_self_test(struct net_device *ndev,
st_param[HNAE3_LOOP_PARALLEL_SERDES][1] =
h->flags & HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK;
+ st_param[HNAE3_LOOP_PHY][0] = HNAE3_LOOP_PHY;
+ st_param[HNAE3_LOOP_PHY][1] =
+ h->flags & HNAE3_SUPPORT_PHY_LOOPBACK;
+
if (if_running)
ndev->netdev_ops->ndo_stop(ndev);
@@ -716,6 +726,12 @@ static int hns3_check_ksettings_param(const struct net_device *netdev,
u8 duplex;
int ret;
+ /* hw doesn't support use specified speed and duplex to negotiate,
+ * unnecessary to check them when autoneg on.
+ */
+ if (cmd->base.autoneg)
+ return 0;
+
if (ops->get_ksettings_an_result) {
ops->get_ksettings_an_result(handle, &autoneg, &speed, &duplex);
if (cmd->base.autoneg == autoneg && cmd->base.speed == speed &&
@@ -777,6 +793,15 @@ static int hns3_set_link_ksettings(struct net_device *netdev,
return ret;
}
+ /* hw doesn't support use specified speed and duplex to negotiate,
+ * ignore them when autoneg on.
+ */
+ if (cmd->base.autoneg) {
+ netdev_info(netdev,
+ "autoneg is on, ignore the speed and duplex\n");
+ return 0;
+ }
+
if (ops->cfg_mac_speed_dup_h)
ret = ops->cfg_mac_speed_dup_h(handle, cmd->base.speed,
cmd->base.duplex);
@@ -898,9 +923,11 @@ static struct hns3_enet_ring *hns3_backup_ringparam(struct hns3_nic_priv *priv)
if (!tmp_rings)
return NULL;
- for (i = 0; i < handle->kinfo.num_tqps * 2; i++)
+ for (i = 0; i < handle->kinfo.num_tqps * 2; i++) {
memcpy(&tmp_rings[i], priv->ring_data[i].ring,
sizeof(struct hns3_enet_ring));
+ tmp_rings[i].skb = NULL;
+ }
return tmp_rings;
}
@@ -1385,6 +1412,7 @@ static const struct ethtool_ops hns3vf_ethtool_ops = {
.set_rxfh = hns3_set_rss,
.get_link_ksettings = hns3_get_link_ksettings,
.get_channels = hns3_get_channels,
+ .set_channels = hns3_set_channels,
.get_coalesce = hns3_get_coalesce,
.set_coalesce = hns3_set_coalesce,
.get_regs_len = hns3_get_regs_len,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h
index 29979be..4821fe0 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h
@@ -223,6 +223,9 @@ enum hclge_opcode_type {
HCLGE_OPC_MAC_ETHTYPE_ADD = 0x1010,
HCLGE_OPC_MAC_ETHTYPE_REMOVE = 0x1011,
+ /* MAC VLAN commands */
+ HCLGE_OPC_MAC_VLAN_SWITCH_PARAM = 0x1033,
+
/* VLAN commands */
HCLGE_OPC_VLAN_FILTER_CTRL = 0x1100,
HCLGE_OPC_VLAN_FILTER_PF_CFG = 0x1101,
@@ -771,6 +774,31 @@ struct hclge_vlan_filter_vf_cfg_cmd {
u8 vf_bitmap[16];
};
+#define HCLGE_SWITCH_ANTI_SPOOF_B 0U
+#define HCLGE_SWITCH_ALW_LPBK_B 1U
+#define HCLGE_SWITCH_ALW_LCL_LPBK_B 2U
+#define HCLGE_SWITCH_ALW_DST_OVRD_B 3U
+#define HCLGE_SWITCH_NO_MASK 0x0
+#define HCLGE_SWITCH_ANTI_SPOOF_MASK 0xFE
+#define HCLGE_SWITCH_ALW_LPBK_MASK 0xFD
+#define HCLGE_SWITCH_ALW_LCL_LPBK_MASK 0xFB
+#define HCLGE_SWITCH_LW_DST_OVRD_MASK 0xF7
+
+struct hclge_mac_vlan_switch_cmd {
+ u8 roce_sel;
+ u8 rsv1[3];
+ __le32 func_id;
+ u8 switch_param;
+ u8 rsv2[3];
+ u8 param_mask;
+ u8 rsv3[11];
+};
+
+enum hclge_mac_vlan_cfg_sel {
+ HCLGE_MAC_VLAN_NIC_SEL = 0,
+ HCLGE_MAC_VLAN_ROCE_SEL,
+};
+
#define HCLGE_ACCEPT_TAG1_B 0
#define HCLGE_ACCEPT_UNTAG1_B 1
#define HCLGE_PORT_INS_TAG1_EN_B 2
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c
index 814e0f0..c063301 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_dcb.c
@@ -198,6 +198,28 @@ static int hclge_client_setup_tc(struct hclge_dev *hdev)
return 0;
}
+static int hclge_notify_down_uinit(struct hclge_dev *hdev)
+{
+ int ret;
+
+ ret = hclge_notify_client(hdev, HNAE3_DOWN_CLIENT);
+ if (ret)
+ return ret;
+
+ return hclge_notify_client(hdev, HNAE3_UNINIT_CLIENT);
+}
+
+static int hclge_notify_init_up(struct hclge_dev *hdev)
+{
+ int ret;
+
+ ret = hclge_notify_client(hdev, HNAE3_INIT_CLIENT);
+ if (ret)
+ return ret;
+
+ return hclge_notify_client(hdev, HNAE3_UP_CLIENT);
+}
+
static int hclge_ieee_setets(struct hnae3_handle *h, struct ieee_ets *ets)
{
struct hclge_vport *vport = hclge_get_vport(h);
@@ -218,11 +240,7 @@ static int hclge_ieee_setets(struct hnae3_handle *h, struct ieee_ets *ets)
if (map_changed) {
netif_dbg(h, drv, netdev, "set ets\n");
- ret = hclge_notify_client(hdev, HNAE3_DOWN_CLIENT);
- if (ret)
- return ret;
-
- ret = hclge_notify_client(hdev, HNAE3_UNINIT_CLIENT);
+ ret = hclge_notify_down_uinit(hdev);
if (ret)
return ret;
}
@@ -242,11 +260,7 @@ static int hclge_ieee_setets(struct hnae3_handle *h, struct ieee_ets *ets)
if (ret)
goto err_out;
- ret = hclge_notify_client(hdev, HNAE3_INIT_CLIENT);
- if (ret)
- return ret;
-
- ret = hclge_notify_client(hdev, HNAE3_UP_CLIENT);
+ ret = hclge_notify_init_up(hdev);
if (ret)
return ret;
}
@@ -257,10 +271,8 @@ static int hclge_ieee_setets(struct hnae3_handle *h, struct ieee_ets *ets)
if (!map_changed)
return ret;
- if (hclge_notify_client(hdev, HNAE3_INIT_CLIENT))
- return ret;
+ hclge_notify_init_up(hdev);
- hclge_notify_client(hdev, HNAE3_UP_CLIENT);
return ret;
}
@@ -330,7 +342,7 @@ static int hclge_ieee_setpfc(struct hnae3_handle *h, struct ieee_pfc *pfc)
hdev->tm_info.pfc_en = pfc->pfc_en;
netif_dbg(h, drv, netdev,
- "set pfc: pfc_en=%u, pfc_map=%u, num_tc=%u\n",
+ "set pfc: pfc_en=%x, pfc_map=%x, num_tc=%u\n",
pfc->pfc_en, pfc_map, hdev->tm_info.num_tc);
hclge_tm_pfc_info_update(hdev);
@@ -383,11 +395,7 @@ static int hclge_setup_tc(struct hnae3_handle *h, u8 tc, u8 *prio_tc)
if (ret)
return -EINVAL;
- ret = hclge_notify_client(hdev, HNAE3_DOWN_CLIENT);
- if (ret)
- return ret;
-
- ret = hclge_notify_client(hdev, HNAE3_UNINIT_CLIENT);
+ ret = hclge_notify_down_uinit(hdev);
if (ret)
return ret;
@@ -409,17 +417,11 @@ static int hclge_setup_tc(struct hnae3_handle *h, u8 tc, u8 *prio_tc)
else
hdev->flag &= ~HCLGE_FLAG_MQPRIO_ENABLE;
- ret = hclge_notify_client(hdev, HNAE3_INIT_CLIENT);
- if (ret)
- return ret;
-
- return hclge_notify_client(hdev, HNAE3_UP_CLIENT);
+ return hclge_notify_init_up(hdev);
err_out:
- if (hclge_notify_client(hdev, HNAE3_INIT_CLIENT))
- return ret;
+ hclge_notify_init_up(hdev);
- hclge_notify_client(hdev, HNAE3_UP_CLIENT);
return ret;
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
index 025184a..d0128d7 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c
@@ -4,24 +4,92 @@
#include <linux/device.h>
#include "hclge_debugfs.h"
-#include "hclge_cmd.h"
#include "hclge_main.h"
#include "hclge_tm.h"
#include "hnae3.h"
+static struct hclge_dbg_reg_type_info hclge_dbg_reg_info[] = {
+ { .reg_type = "bios common",
+ .dfx_msg = &hclge_dbg_bios_common_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_bios_common_reg),
+ .offset = HCLGE_DBG_DFX_BIOS_OFFSET,
+ .cmd = HCLGE_OPC_DFX_BIOS_COMMON_REG } },
+ { .reg_type = "ssu",
+ .dfx_msg = &hclge_dbg_ssu_reg_0[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_ssu_reg_0),
+ .offset = HCLGE_DBG_DFX_SSU_0_OFFSET,
+ .cmd = HCLGE_OPC_DFX_SSU_REG_0 } },
+ { .reg_type = "ssu",
+ .dfx_msg = &hclge_dbg_ssu_reg_1[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_ssu_reg_1),
+ .offset = HCLGE_DBG_DFX_SSU_1_OFFSET,
+ .cmd = HCLGE_OPC_DFX_SSU_REG_1 } },
+ { .reg_type = "ssu",
+ .dfx_msg = &hclge_dbg_ssu_reg_2[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_ssu_reg_2),
+ .offset = HCLGE_DBG_DFX_SSU_2_OFFSET,
+ .cmd = HCLGE_OPC_DFX_SSU_REG_2 } },
+ { .reg_type = "igu egu",
+ .dfx_msg = &hclge_dbg_igu_egu_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_igu_egu_reg),
+ .offset = HCLGE_DBG_DFX_IGU_OFFSET,
+ .cmd = HCLGE_OPC_DFX_IGU_EGU_REG } },
+ { .reg_type = "rpu",
+ .dfx_msg = &hclge_dbg_rpu_reg_0[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_rpu_reg_0),
+ .offset = HCLGE_DBG_DFX_RPU_0_OFFSET,
+ .cmd = HCLGE_OPC_DFX_RPU_REG_0 } },
+ { .reg_type = "rpu",
+ .dfx_msg = &hclge_dbg_rpu_reg_1[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_rpu_reg_1),
+ .offset = HCLGE_DBG_DFX_RPU_1_OFFSET,
+ .cmd = HCLGE_OPC_DFX_RPU_REG_1 } },
+ { .reg_type = "ncsi",
+ .dfx_msg = &hclge_dbg_ncsi_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_ncsi_reg),
+ .offset = HCLGE_DBG_DFX_NCSI_OFFSET,
+ .cmd = HCLGE_OPC_DFX_NCSI_REG } },
+ { .reg_type = "rtc",
+ .dfx_msg = &hclge_dbg_rtc_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_rtc_reg),
+ .offset = HCLGE_DBG_DFX_RTC_OFFSET,
+ .cmd = HCLGE_OPC_DFX_RTC_REG } },
+ { .reg_type = "ppp",
+ .dfx_msg = &hclge_dbg_ppp_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_ppp_reg),
+ .offset = HCLGE_DBG_DFX_PPP_OFFSET,
+ .cmd = HCLGE_OPC_DFX_PPP_REG } },
+ { .reg_type = "rcb",
+ .dfx_msg = &hclge_dbg_rcb_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_rcb_reg),
+ .offset = HCLGE_DBG_DFX_RCB_OFFSET,
+ .cmd = HCLGE_OPC_DFX_RCB_REG } },
+ { .reg_type = "tqp",
+ .dfx_msg = &hclge_dbg_tqp_reg[0],
+ .reg_msg = { .msg_num = ARRAY_SIZE(hclge_dbg_tqp_reg),
+ .offset = HCLGE_DBG_DFX_TQP_OFFSET,
+ .cmd = HCLGE_OPC_DFX_TQP_REG } },
+};
+
static int hclge_dbg_get_dfx_bd_num(struct hclge_dev *hdev, int offset)
{
- struct hclge_desc desc[4];
+#define HCLGE_GET_DFX_REG_TYPE_CNT 4
+
+ struct hclge_desc desc[HCLGE_GET_DFX_REG_TYPE_CNT];
+ int entries_per_desc;
+ int index;
int ret;
ret = hclge_query_bd_num_cmd_send(hdev, desc);
if (ret) {
dev_err(&hdev->pdev->dev,
- "get dfx bdnum fail, status is %d.\n", ret);
+ "get dfx bdnum fail, ret = %d\n", ret);
return ret;
}
- return (int)desc[offset / 6].data[offset % 6];
+ entries_per_desc = ARRAY_SIZE(desc[0].data);
+ index = offset % entries_per_desc;
+ return (int)desc[offset / entries_per_desc].data[index];
}
static int hclge_dbg_cmd_send(struct hclge_dev *hdev,
@@ -42,35 +110,40 @@ static int hclge_dbg_cmd_send(struct hclge_dev *hdev,
}
ret = hclge_cmd_send(&hdev->hw, desc_src, bd_num);
- if (ret) {
+ if (ret)
dev_err(&hdev->pdev->dev,
- "read reg cmd send fail, status is %d.\n", ret);
- return ret;
- }
-
+ "cmd(0x%x) send fail, ret = %d\n", cmd, ret);
return ret;
}
static void hclge_dbg_dump_reg_common(struct hclge_dev *hdev,
- struct hclge_dbg_dfx_message *dfx_message,
- const char *cmd_buf, int msg_num,
- int offset, enum hclge_opcode_type cmd)
+ struct hclge_dbg_reg_type_info *reg_info,
+ const char *cmd_buf)
{
-#define BD_DATA_NUM 6
+#define IDX_OFFSET 1
+ const char *s = &cmd_buf[strlen(reg_info->reg_type) + IDX_OFFSET];
+ struct hclge_dbg_dfx_message *dfx_message = reg_info->dfx_msg;
+ struct hclge_dbg_reg_common_msg *reg_msg = ®_info->reg_msg;
struct hclge_desc *desc_src;
struct hclge_desc *desc;
+ int entries_per_desc;
int bd_num, buf_len;
+ int index = 0;
+ int min_num;
int ret, i;
- int index;
- int max;
- ret = kstrtouint(cmd_buf, 10, &index);
- index = (ret != 0) ? 0 : index;
+ if (*s) {
+ ret = kstrtouint(s, 0, &index);
+ index = (ret != 0) ? 0 : index;
+ }
- bd_num = hclge_dbg_get_dfx_bd_num(hdev, offset);
- if (bd_num <= 0)
+ bd_num = hclge_dbg_get_dfx_bd_num(hdev, reg_msg->offset);
+ if (bd_num <= 0) {
+ dev_err(&hdev->pdev->dev, "get cmd(%d) bd num(%d) failed\n",
+ reg_msg->offset, bd_num);
return;
+ }
buf_len = sizeof(struct hclge_desc) * bd_num;
desc_src = kzalloc(buf_len, GFP_KERNEL);
@@ -80,22 +153,23 @@ static void hclge_dbg_dump_reg_common(struct hclge_dev *hdev,
}
desc = desc_src;
- ret = hclge_dbg_cmd_send(hdev, desc, index, bd_num, cmd);
- if (ret != HCLGE_CMD_EXEC_SUCCESS) {
+ ret = hclge_dbg_cmd_send(hdev, desc, index, bd_num, reg_msg->cmd);
+ if (ret) {
kfree(desc_src);
return;
}
- max = (bd_num * BD_DATA_NUM) <= msg_num ?
- (bd_num * BD_DATA_NUM) : msg_num;
+ entries_per_desc = ARRAY_SIZE(desc->data);
+ min_num = min_t(int, bd_num * entries_per_desc, reg_msg->msg_num);
desc = desc_src;
- for (i = 0; i < max; i++) {
- ((i > 0) && ((i % BD_DATA_NUM) == 0)) ? desc++ : desc;
+ for (i = 0; i < min_num; i++) {
+ if (i > 0 && (i % entries_per_desc) == 0)
+ desc++;
if (dfx_message->flag)
dev_info(&hdev->pdev->dev, "%s: 0x%x\n",
dfx_message->message,
- desc->data[i % BD_DATA_NUM]);
+ desc->data[i % entries_per_desc]);
dfx_message++;
}
@@ -205,95 +279,25 @@ static void hclge_dbg_dump_dcb(struct hclge_dev *hdev, const char *cmd_buf)
static void hclge_dbg_dump_reg_cmd(struct hclge_dev *hdev, const char *cmd_buf)
{
- int msg_num;
+ struct hclge_dbg_reg_type_info *reg_info;
+ bool has_dump = false;
+ int i;
- if (strncmp(&cmd_buf[9], "bios common", 11) == 0) {
- msg_num = sizeof(hclge_dbg_bios_common_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_bios_common_reg,
- &cmd_buf[21], msg_num,
- HCLGE_DBG_DFX_BIOS_OFFSET,
- HCLGE_OPC_DFX_BIOS_COMMON_REG);
- } else if (strncmp(&cmd_buf[9], "ssu", 3) == 0) {
- msg_num = sizeof(hclge_dbg_ssu_reg_0) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_ssu_reg_0,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_SSU_0_OFFSET,
- HCLGE_OPC_DFX_SSU_REG_0);
+ for (i = 0; i < ARRAY_SIZE(hclge_dbg_reg_info); i++) {
+ reg_info = &hclge_dbg_reg_info[i];
+ if (!strncmp(cmd_buf, reg_info->reg_type,
+ strlen(reg_info->reg_type))) {
+ hclge_dbg_dump_reg_common(hdev, reg_info, cmd_buf);
+ has_dump = true;
+ }
+ }
- msg_num = sizeof(hclge_dbg_ssu_reg_1) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_ssu_reg_1,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_SSU_1_OFFSET,
- HCLGE_OPC_DFX_SSU_REG_1);
+ if (strncmp(cmd_buf, "dcb", 3) == 0) {
+ hclge_dbg_dump_dcb(hdev, &cmd_buf[sizeof("dcb")]);
+ has_dump = true;
+ }
- msg_num = sizeof(hclge_dbg_ssu_reg_2) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_ssu_reg_2,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_SSU_2_OFFSET,
- HCLGE_OPC_DFX_SSU_REG_2);
- } else if (strncmp(&cmd_buf[9], "igu egu", 7) == 0) {
- msg_num = sizeof(hclge_dbg_igu_egu_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_igu_egu_reg,
- &cmd_buf[17], msg_num,
- HCLGE_DBG_DFX_IGU_OFFSET,
- HCLGE_OPC_DFX_IGU_EGU_REG);
- } else if (strncmp(&cmd_buf[9], "rpu", 3) == 0) {
- msg_num = sizeof(hclge_dbg_rpu_reg_0) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_rpu_reg_0,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_RPU_0_OFFSET,
- HCLGE_OPC_DFX_RPU_REG_0);
-
- msg_num = sizeof(hclge_dbg_rpu_reg_1) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_rpu_reg_1,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_RPU_1_OFFSET,
- HCLGE_OPC_DFX_RPU_REG_1);
- } else if (strncmp(&cmd_buf[9], "ncsi", 4) == 0) {
- msg_num = sizeof(hclge_dbg_ncsi_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_ncsi_reg,
- &cmd_buf[14], msg_num,
- HCLGE_DBG_DFX_NCSI_OFFSET,
- HCLGE_OPC_DFX_NCSI_REG);
- } else if (strncmp(&cmd_buf[9], "rtc", 3) == 0) {
- msg_num = sizeof(hclge_dbg_rtc_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_rtc_reg,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_RTC_OFFSET,
- HCLGE_OPC_DFX_RTC_REG);
- } else if (strncmp(&cmd_buf[9], "ppp", 3) == 0) {
- msg_num = sizeof(hclge_dbg_ppp_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_ppp_reg,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_PPP_OFFSET,
- HCLGE_OPC_DFX_PPP_REG);
- } else if (strncmp(&cmd_buf[9], "rcb", 3) == 0) {
- msg_num = sizeof(hclge_dbg_rcb_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_rcb_reg,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_RCB_OFFSET,
- HCLGE_OPC_DFX_RCB_REG);
- } else if (strncmp(&cmd_buf[9], "tqp", 3) == 0) {
- msg_num = sizeof(hclge_dbg_tqp_reg) /
- sizeof(struct hclge_dbg_dfx_message);
- hclge_dbg_dump_reg_common(hdev, hclge_dbg_tqp_reg,
- &cmd_buf[13], msg_num,
- HCLGE_DBG_DFX_TQP_OFFSET,
- HCLGE_OPC_DFX_TQP_REG);
- } else if (strncmp(&cmd_buf[9], "dcb", 3) == 0) {
- hclge_dbg_dump_dcb(hdev, &cmd_buf[13]);
- } else {
+ if (!has_dump) {
dev_info(&hdev->pdev->dev, "unknown command\n");
return;
}
@@ -327,7 +331,7 @@ static void hclge_dbg_dump_tc(struct hclge_dev *hdev)
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
- dev_err(&hdev->pdev->dev, "dump tc fail, status is %d.\n", ret);
+ dev_err(&hdev->pdev->dev, "dump tc fail, ret = %d\n", ret);
return;
}
@@ -429,7 +433,7 @@ static void hclge_dbg_dump_tm_pg(struct hclge_dev *hdev)
return;
err_tm_pg_cmd_send:
- dev_err(&hdev->pdev->dev, "dump tm_pg fail(0x%x), status is %d\n",
+ dev_err(&hdev->pdev->dev, "dump tm_pg fail(0x%x), ret = %d\n",
cmd, ret);
}
@@ -541,7 +545,7 @@ static void hclge_dbg_dump_tm(struct hclge_dev *hdev)
return;
err_tm_cmd_send:
- dev_err(&hdev->pdev->dev, "dump tm fail(0x%x), status is %d\n",
+ dev_err(&hdev->pdev->dev, "dump tm fail(0x%x), ret = %d\n",
cmd, ret);
}
@@ -560,7 +564,7 @@ static void hclge_dbg_dump_tm_map(struct hclge_dev *hdev,
int pri_id, ret;
u32 i;
- ret = kstrtouint(&cmd_buf[12], 10, &queue_id);
+ ret = kstrtouint(cmd_buf, 0, &queue_id);
queue_id = (ret != 0) ? 0 : queue_id;
cmd = HCLGE_OPC_TM_NQ_TO_QS_LINK;
@@ -630,7 +634,7 @@ static void hclge_dbg_dump_tm_map(struct hclge_dev *hdev,
return;
err_tm_map_cmd_send:
- dev_err(&hdev->pdev->dev, "dump tqp map fail(0x%x), status is %d\n",
+ dev_err(&hdev->pdev->dev, "dump tqp map fail(0x%x), ret = %d\n",
cmd, ret);
}
@@ -644,7 +648,7 @@ static void hclge_dbg_dump_qos_pause_cfg(struct hclge_dev *hdev)
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
- dev_err(&hdev->pdev->dev, "dump checksum fail, status is %d.\n",
+ dev_err(&hdev->pdev->dev, "dump checksum fail, ret = %d\n",
ret);
return;
}
@@ -668,7 +672,7 @@ static void hclge_dbg_dump_qos_pri_map(struct hclge_dev *hdev)
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
- "dump qos pri map fail, status is %d.\n", ret);
+ "dump qos pri map fail, ret = %d\n", ret);
return;
}
@@ -770,7 +774,8 @@ static void hclge_dbg_dump_qos_buf_cfg(struct hclge_dev *hdev)
rx_priv_wl = (struct hclge_rx_priv_wl_buf *)desc[1].data;
for (i = 0; i < HCLGE_TC_NUM_ONE_DESC; i++)
dev_info(&hdev->pdev->dev,
- "rx_priv_wl_tc_%d: high: 0x%x, low: 0x%x\n", i + 4,
+ "rx_priv_wl_tc_%d: high: 0x%x, low: 0x%x\n",
+ i + HCLGE_TC_NUM_ONE_DESC,
rx_priv_wl->tc_wl[i].high, rx_priv_wl->tc_wl[i].low);
cmd = HCLGE_OPC_RX_COM_THRD_ALLOC;
@@ -792,14 +797,15 @@ static void hclge_dbg_dump_qos_buf_cfg(struct hclge_dev *hdev)
rx_com_thrd = (struct hclge_rx_com_thrd *)desc[1].data;
for (i = 0; i < HCLGE_TC_NUM_ONE_DESC; i++)
dev_info(&hdev->pdev->dev,
- "rx_com_thrd_tc_%d: high: 0x%x, low: 0x%x\n", i + 4,
+ "rx_com_thrd_tc_%d: high: 0x%x, low: 0x%x\n",
+ i + HCLGE_TC_NUM_ONE_DESC,
rx_com_thrd->com_thrd[i].high,
rx_com_thrd->com_thrd[i].low);
return;
err_qos_cmd_send:
dev_err(&hdev->pdev->dev,
- "dump qos buf cfg fail(0x%x), status is %d\n", cmd, ret);
+ "dump qos buf cfg fail(0x%x), ret = %d\n", cmd, ret);
}
static void hclge_dbg_dump_mng_table(struct hclge_dev *hdev)
@@ -925,25 +931,39 @@ static void hclge_dbg_fd_tcam(struct hclge_dev *hdev)
static void hclge_dbg_dump_rst_info(struct hclge_dev *hdev)
{
- dev_info(&hdev->pdev->dev, "PF reset count: %d\n",
+ dev_info(&hdev->pdev->dev, "PF reset count: %u\n",
hdev->rst_stats.pf_rst_cnt);
- dev_info(&hdev->pdev->dev, "FLR reset count: %d\n",
+ dev_info(&hdev->pdev->dev, "FLR reset count: %u\n",
hdev->rst_stats.flr_rst_cnt);
- dev_info(&hdev->pdev->dev, "CORE reset count: %d\n",
- hdev->rst_stats.core_rst_cnt);
- dev_info(&hdev->pdev->dev, "GLOBAL reset count: %d\n",
+ dev_info(&hdev->pdev->dev, "GLOBAL reset count: %u\n",
hdev->rst_stats.global_rst_cnt);
- dev_info(&hdev->pdev->dev, "IMP reset count: %d\n",
+ dev_info(&hdev->pdev->dev, "IMP reset count: %u\n",
hdev->rst_stats.imp_rst_cnt);
- dev_info(&hdev->pdev->dev, "reset done count: %d\n",
+ dev_info(&hdev->pdev->dev, "reset done count: %u\n",
hdev->rst_stats.reset_done_cnt);
- dev_info(&hdev->pdev->dev, "HW reset done count: %d\n",
+ dev_info(&hdev->pdev->dev, "HW reset done count: %u\n",
hdev->rst_stats.hw_reset_done_cnt);
- dev_info(&hdev->pdev->dev, "reset count: %d\n",
+ dev_info(&hdev->pdev->dev, "reset count: %u\n",
hdev->rst_stats.reset_cnt);
+ dev_info(&hdev->pdev->dev, "reset count: %u\n",
+ hdev->rst_stats.reset_cnt);
+ dev_info(&hdev->pdev->dev, "reset fail count: %u\n",
+ hdev->rst_stats.reset_fail_cnt);
+ dev_info(&hdev->pdev->dev, "vector0 interrupt enable status: 0x%x\n",
+ hclge_read_dev(&hdev->hw, HCLGE_MISC_VECTOR_REG_BASE));
+ dev_info(&hdev->pdev->dev, "reset interrupt source: 0x%x\n",
+ hclge_read_dev(&hdev->hw, HCLGE_MISC_RESET_STS_REG));
+ dev_info(&hdev->pdev->dev, "reset interrupt status: 0x%x\n",
+ hclge_read_dev(&hdev->hw, HCLGE_MISC_VECTOR_INT_STS));
+ dev_info(&hdev->pdev->dev, "hardware reset status: 0x%x\n",
+ hclge_read_dev(&hdev->hw, HCLGE_GLOBAL_RESET_REG));
+ dev_info(&hdev->pdev->dev, "handshake status: 0x%x\n",
+ hclge_read_dev(&hdev->hw, HCLGE_NIC_CSQ_DEPTH_REG));
+ dev_info(&hdev->pdev->dev, "function reset status: 0x%x\n",
+ hclge_read_dev(&hdev->hw, HCLGE_FUN_RST_ING));
}
-void hclge_dbg_get_m7_stats_info(struct hclge_dev *hdev)
+static void hclge_dbg_get_m7_stats_info(struct hclge_dev *hdev)
{
struct hclge_desc *desc_src, *desc_tmp;
struct hclge_get_m7_bd_cmd *req;
@@ -957,7 +977,7 @@ void hclge_dbg_get_m7_stats_info(struct hclge_dev *hdev)
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
if (ret) {
dev_err(&hdev->pdev->dev,
- "get firmware statistics bd number failed, ret=%d\n",
+ "get firmware statistics bd number failed, ret = %d\n",
ret);
return;
}
@@ -978,7 +998,7 @@ void hclge_dbg_get_m7_stats_info(struct hclge_dev *hdev)
if (ret) {
kfree(desc_src);
dev_err(&hdev->pdev->dev,
- "get firmware statistics failed, ret=%d\n", ret);
+ "get firmware statistics failed, ret = %d\n", ret);
return;
}
@@ -1092,6 +1112,9 @@ static void hclge_dbg_dump_mac_tnl_status(struct hclge_dev *hdev)
int hclge_dbg_run_cmd(struct hnae3_handle *handle, const char *cmd_buf)
{
+#define DUMP_REG "dump reg"
+#define DUMP_TM_MAP "dump tm map"
+
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
@@ -1099,8 +1122,8 @@ int hclge_dbg_run_cmd(struct hnae3_handle *handle, const char *cmd_buf)
hclge_dbg_fd_tcam(hdev);
} else if (strncmp(cmd_buf, "dump tc", 7) == 0) {
hclge_dbg_dump_tc(hdev);
- } else if (strncmp(cmd_buf, "dump tm map", 11) == 0) {
- hclge_dbg_dump_tm_map(hdev, cmd_buf);
+ } else if (strncmp(cmd_buf, DUMP_TM_MAP, strlen(DUMP_TM_MAP)) == 0) {
+ hclge_dbg_dump_tm_map(hdev, &cmd_buf[sizeof(DUMP_TM_MAP)]);
} else if (strncmp(cmd_buf, "dump tm", 7) == 0) {
hclge_dbg_dump_tm(hdev);
} else if (strncmp(cmd_buf, "dump qos pause cfg", 18) == 0) {
@@ -1111,8 +1134,8 @@ int hclge_dbg_run_cmd(struct hnae3_handle *handle, const char *cmd_buf)
hclge_dbg_dump_qos_buf_cfg(hdev);
} else if (strncmp(cmd_buf, "dump mng tbl", 12) == 0) {
hclge_dbg_dump_mng_table(hdev);
- } else if (strncmp(cmd_buf, "dump reg", 8) == 0) {
- hclge_dbg_dump_reg_cmd(hdev, cmd_buf);
+ } else if (strncmp(cmd_buf, DUMP_REG, strlen(DUMP_REG)) == 0) {
+ hclge_dbg_dump_reg_cmd(hdev, &cmd_buf[sizeof(DUMP_REG)]);
} else if (strncmp(cmd_buf, "dump reset info", 15) == 0) {
hclge_dbg_dump_rst_info(hdev);
} else if (strncmp(cmd_buf, "dump m7 info", 12) == 0) {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h
index d055fda..38b7932 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.h
@@ -4,6 +4,9 @@
#ifndef __HCLGE_DEBUGFS_H
#define __HCLGE_DEBUGFS_H
+#include <linux/etherdevice.h>
+#include "hclge_cmd.h"
+
#define HCLGE_DBG_BUF_LEN 256
#define HCLGE_DBG_MNG_TBL_MAX 64
@@ -63,9 +66,23 @@ struct hclge_dbg_bitmap_cmd {
};
};
+struct hclge_dbg_reg_common_msg {
+ int msg_num;
+ int offset;
+ enum hclge_opcode_type cmd;
+};
+
+#define HCLGE_DBG_MAX_DFX_MSG_LEN 60
struct hclge_dbg_dfx_message {
int flag;
- char message[60];
+ char message[HCLGE_DBG_MAX_DFX_MSG_LEN];
+};
+
+#define HCLGE_DBG_MAC_REG_TYPE_LEN 32
+struct hclge_dbg_reg_type_info {
+ const char *reg_type;
+ struct hclge_dbg_dfx_message *dfx_msg;
+ struct hclge_dbg_reg_common_msg reg_msg;
};
#pragma pack()
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c
index 2425b3f..58c6231 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c
@@ -930,32 +930,44 @@ static int hclge_config_ppu_error_interrupts(struct hclge_dev *hdev, u32 cmd,
/* configure PPU error interrupts */
if (cmd == HCLGE_PPU_MPF_ECC_INT_CMD) {
hclge_cmd_setup_basic_desc(&desc[0], cmd, false);
- desc[0].flag |= HCLGE_CMD_FLAG_NEXT;
+ desc[0].flag |= cpu_to_le16(HCLGE_CMD_FLAG_NEXT);
hclge_cmd_setup_basic_desc(&desc[1], cmd, false);
if (en) {
- desc[0].data[0] = HCLGE_PPU_MPF_ABNORMAL_INT0_EN;
- desc[0].data[1] = HCLGE_PPU_MPF_ABNORMAL_INT1_EN;
- desc[1].data[3] = HCLGE_PPU_MPF_ABNORMAL_INT3_EN;
- desc[1].data[4] = HCLGE_PPU_MPF_ABNORMAL_INT2_EN;
+ desc[0].data[0] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT0_EN);
+ desc[0].data[1] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT1_EN);
+ desc[1].data[3] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT3_EN);
+ desc[1].data[4] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT2_EN);
}
- desc[1].data[0] = HCLGE_PPU_MPF_ABNORMAL_INT0_EN_MASK;
- desc[1].data[1] = HCLGE_PPU_MPF_ABNORMAL_INT1_EN_MASK;
- desc[1].data[2] = HCLGE_PPU_MPF_ABNORMAL_INT2_EN_MASK;
- desc[1].data[3] |= HCLGE_PPU_MPF_ABNORMAL_INT3_EN_MASK;
+ desc[1].data[0] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT0_EN_MASK);
+ desc[1].data[1] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT1_EN_MASK);
+ desc[1].data[2] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT2_EN_MASK);
+ desc[1].data[3] |=
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT3_EN_MASK);
desc_num = 2;
} else if (cmd == HCLGE_PPU_MPF_OTHER_INT_CMD) {
hclge_cmd_setup_basic_desc(&desc[0], cmd, false);
if (en)
- desc[0].data[0] = HCLGE_PPU_MPF_ABNORMAL_INT2_EN2;
+ desc[0].data[0] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT2_EN2);
- desc[0].data[2] = HCLGE_PPU_MPF_ABNORMAL_INT2_EN2_MASK;
+ desc[0].data[2] =
+ cpu_to_le32(HCLGE_PPU_MPF_ABNORMAL_INT2_EN2_MASK);
} else if (cmd == HCLGE_PPU_PF_OTHER_INT_CMD) {
hclge_cmd_setup_basic_desc(&desc[0], cmd, false);
if (en)
- desc[0].data[0] = HCLGE_PPU_PF_ABNORMAL_INT_EN;
+ desc[0].data[0] =
+ cpu_to_le32(HCLGE_PPU_PF_ABNORMAL_INT_EN);
- desc[0].data[2] = HCLGE_PPU_PF_ABNORMAL_INT_EN_MASK;
+ desc[0].data[2] =
+ cpu_to_le32(HCLGE_PPU_PF_ABNORMAL_INT_EN_MASK);
} else {
dev_err(dev, "Invalid cmd to configure PPU error interrupts\n");
return -EINVAL;
@@ -1313,10 +1325,12 @@ static int hclge_handle_pf_ras_error(struct hclge_dev *hdev,
/* log PPU(RCB) errors */
desc_data = (__le32 *)&desc[3];
status = le32_to_cpu(*desc_data) & HCLGE_PPU_PF_INT_RAS_MASK;
- if (status)
+ if (status) {
hclge_log_error(dev, "PPU_PF_ABNORMAL_INT_ST0",
&hclge_ppu_pf_abnormal_int[0], status,
&ae_dev->hw_err_reset_req);
+ hclge_report_hw_error(hdev, HNAE3_PPU_POISON_ERROR);
+ }
/* clear all PF RAS errors */
hclge_cmd_reuse_desc(&desc[0], false);
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.h
index 7ea8bb2..876fd81a 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.h
@@ -5,6 +5,7 @@
#define __HCLGE_ERR_H
#include "hclge_main.h"
+#include "hnae3.h"
#define HCLGE_MPF_RAS_INT_MIN_BD_NUM 10
#define HCLGE_PF_RAS_INT_MIN_BD_NUM 4
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
index 9d64c43..fd7f943 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c
@@ -53,6 +53,8 @@
#define HCLGE_DFX_TQP_BD_OFFSET 11
#define HCLGE_DFX_SSU_2_BD_OFFSET 12
+#define HCLGE_LINK_STATUS_MS 10
+
static int hclge_set_mac_mtu(struct hclge_dev *hdev, int new_mps);
static int hclge_init_vlan_config(struct hclge_dev *hdev);
static void hclge_sync_vlan_filter(struct hclge_dev *hdev);
@@ -64,6 +66,7 @@ static void hclge_rfs_filter_expire(struct hclge_dev *hdev);
static void hclge_clear_arfs_rules(struct hnae3_handle *handle);
static enum hnae3_reset_type hclge_get_reset_level(struct hnae3_ae_dev *ae_dev,
unsigned long *addr);
+static int hclge_set_default_loopback(struct hclge_dev *hdev);
static struct hnae3_ae_algo ae_algo;
@@ -364,6 +367,50 @@ static const enum hclge_opcode_type hclge_dfx_reg_opcode_list[] = {
HCLGE_OPC_DFX_SSU_REG_2
};
+static const struct key_info meta_data_key_info[] = {
+ { PACKET_TYPE_ID, 6},
+ { IP_FRAGEMENT, 1},
+ { ROCE_TYPE, 1},
+ { NEXT_KEY, 5},
+ { VLAN_NUMBER, 2},
+ { SRC_VPORT, 12},
+ { DST_VPORT, 12},
+ { TUNNEL_PACKET, 1},
+};
+
+static const struct key_info tuple_key_info[] = {
+ { OUTER_DST_MAC, 48},
+ { OUTER_SRC_MAC, 48},
+ { OUTER_VLAN_TAG_FST, 16},
+ { OUTER_VLAN_TAG_SEC, 16},
+ { OUTER_ETH_TYPE, 16},
+ { OUTER_L2_RSV, 16},
+ { OUTER_IP_TOS, 8},
+ { OUTER_IP_PROTO, 8},
+ { OUTER_SRC_IP, 32},
+ { OUTER_DST_IP, 32},
+ { OUTER_L3_RSV, 16},
+ { OUTER_SRC_PORT, 16},
+ { OUTER_DST_PORT, 16},
+ { OUTER_L4_RSV, 32},
+ { OUTER_TUN_VNI, 24},
+ { OUTER_TUN_FLOW_ID, 8},
+ { INNER_DST_MAC, 48},
+ { INNER_SRC_MAC, 48},
+ { INNER_VLAN_TAG_FST, 16},
+ { INNER_VLAN_TAG_SEC, 16},
+ { INNER_ETH_TYPE, 16},
+ { INNER_L2_RSV, 16},
+ { INNER_IP_TOS, 8},
+ { INNER_IP_PROTO, 8},
+ { INNER_SRC_IP, 32},
+ { INNER_DST_IP, 32},
+ { INNER_L3_RSV, 16},
+ { INNER_SRC_PORT, 16},
+ { INNER_DST_PORT, 16},
+ { INNER_L4_RSV, 32},
+};
+
static int hclge_mac_update_stats_defective(struct hclge_dev *hdev)
{
#define HCLGE_MAC_CMD_NUM 21
@@ -698,6 +745,12 @@ static int hclge_get_sset_count(struct hnae3_handle *handle, int stringset)
count += 2;
handle->flags |= HNAE3_SUPPORT_SERDES_SERIAL_LOOPBACK;
handle->flags |= HNAE3_SUPPORT_SERDES_PARALLEL_LOOPBACK;
+
+ if (hdev->hw.mac.phydev) {
+ count += 1;
+ handle->flags |= HNAE3_SUPPORT_PHY_LOOPBACK;
+ }
+
} else if (stringset == ETH_SS_STATS) {
count = ARRAY_SIZE(g_mac_stats_string) +
hclge_tqps_get_sset_count(handle, stringset);
@@ -2547,6 +2600,10 @@ static int hclge_mac_init(struct hclge_dev *hdev)
return ret;
}
+ ret = hclge_set_default_loopback(hdev);
+ if (ret)
+ return ret;
+
ret = hclge_buffer_alloc(hdev);
if (ret)
dev_err(&hdev->pdev->dev,
@@ -3221,6 +3278,38 @@ static int hclge_func_reset_sync_vf(struct hclge_dev *hdev)
return -ETIME;
}
+void hclge_report_hw_error(struct hclge_dev *hdev,
+ enum hnae3_hw_error_type type)
+{
+ struct hnae3_client *client = hdev->nic_client;
+ u16 i;
+
+ if (!client || !client->ops->process_hw_error ||
+ !test_bit(HCLGE_STATE_NIC_REGISTERED, &hdev->state))
+ return;
+
+ for (i = 0; i < hdev->num_vmdq_vport + 1; i++)
+ client->ops->process_hw_error(&hdev->vport[i].nic, type);
+}
+
+static void hclge_handle_imp_error(struct hclge_dev *hdev)
+{
+ u32 reg_val;
+
+ reg_val = hclge_read_dev(&hdev->hw, HCLGE_PF_OTHER_INT_REG);
+ if (reg_val & BIT(HCLGE_VECTOR0_IMP_RD_POISON_B)) {
+ hclge_report_hw_error(hdev, HNAE3_IMP_RD_POISON_ERROR);
+ reg_val &= ~BIT(HCLGE_VECTOR0_IMP_RD_POISON_B);
+ hclge_write_dev(&hdev->hw, HCLGE_PF_OTHER_INT_REG, reg_val);
+ }
+
+ if (reg_val & BIT(HCLGE_VECTOR0_IMP_CMDQ_ERR_B)) {
+ hclge_report_hw_error(hdev, HNAE3_CMDQ_ECC_ERROR);
+ reg_val &= ~BIT(HCLGE_VECTOR0_IMP_CMDQ_ERR_B);
+ hclge_write_dev(&hdev->hw, HCLGE_PF_OTHER_INT_REG, reg_val);
+ }
+}
+
int hclge_func_reset_cmd(struct hclge_dev *hdev, int func_id)
{
struct hclge_desc desc;
@@ -3427,6 +3516,7 @@ static int hclge_reset_prepare_wait(struct hclge_dev *hdev)
hdev->rst_stats.flr_rst_cnt++;
break;
case HNAE3_IMP_RESET:
+ hclge_handle_imp_error(hdev);
reg_val = hclge_read_dev(&hdev->hw, HCLGE_PF_OTHER_INT_REG);
hclge_write_dev(&hdev->hw, HCLGE_PF_OTHER_INT_REG,
BIT(HCLGE_VECTOR0_IMP_RESET_INT_B) | reg_val);
@@ -3451,19 +3541,18 @@ static bool hclge_reset_err_handle(struct hclge_dev *hdev)
dev_info(&hdev->pdev->dev, "Reset pending %lu\n",
hdev->reset_pending);
return true;
- } else if ((hdev->reset_type != HNAE3_IMP_RESET) &&
- (hclge_read_dev(&hdev->hw, HCLGE_GLOBAL_RESET_REG) &
- BIT(HCLGE_IMP_RESET_BIT))) {
+ } else if (hclge_read_dev(&hdev->hw, HCLGE_MISC_VECTOR_INT_STS) &
+ HCLGE_RESET_INT_M) {
dev_info(&hdev->pdev->dev,
- "reset failed because IMP Reset is pending\n");
+ "reset failed because new reset interrupt\n");
hclge_clear_reset_cause(hdev);
return false;
- } else if (hdev->reset_fail_cnt < MAX_RESET_FAIL_CNT) {
- hdev->reset_fail_cnt++;
+ } else if (hdev->rst_stats.reset_fail_cnt < MAX_RESET_FAIL_CNT) {
+ hdev->rst_stats.reset_fail_cnt++;
set_bit(hdev->reset_type, &hdev->reset_pending);
dev_info(&hdev->pdev->dev,
"re-schedule reset task(%d)\n",
- hdev->reset_fail_cnt);
+ hdev->rst_stats.reset_fail_cnt);
return true;
}
@@ -3535,6 +3624,7 @@ static int hclge_reset_stack(struct hclge_dev *hdev)
static void hclge_reset(struct hclge_dev *hdev)
{
struct hnae3_ae_dev *ae_dev = pci_get_drvdata(hdev->pdev);
+ enum hnae3_reset_type reset_level;
int ret;
/* Initialize ae_dev reset status as well, in case enet layer wants to
@@ -3589,7 +3679,8 @@ static void hclge_reset(struct hclge_dev *hdev)
/* ignore RoCE notify error if it fails HCLGE_RESET_MAX_FAIL_CNT - 1
* times
*/
- if (ret && hdev->reset_fail_cnt < HCLGE_RESET_MAX_FAIL_CNT - 1)
+ if (ret &&
+ hdev->rst_stats.reset_fail_cnt < HCLGE_RESET_MAX_FAIL_CNT - 1)
goto err_reset;
rtnl_lock();
@@ -3605,7 +3696,7 @@ static void hclge_reset(struct hclge_dev *hdev)
goto err_reset;
hdev->last_reset_time = jiffies;
- hdev->reset_fail_cnt = 0;
+ hdev->rst_stats.reset_fail_cnt = 0;
hdev->rst_stats.reset_done_cnt++;
ae_dev->reset_type = HNAE3_NONE_RESET;
@@ -3613,10 +3704,10 @@ static void hclge_reset(struct hclge_dev *hdev)
* it should be handled as soon as possible. since some errors
* need this kind of reset to fix.
*/
- hdev->reset_level = hclge_get_reset_level(ae_dev,
- &hdev->default_reset_request);
- if (hdev->reset_level != HNAE3_NONE_RESET)
- set_bit(hdev->reset_level, &hdev->reset_request);
+ reset_level = hclge_get_reset_level(ae_dev,
+ &hdev->default_reset_request);
+ if (reset_level != HNAE3_NONE_RESET)
+ set_bit(reset_level, &hdev->reset_request);
return;
@@ -3661,7 +3752,7 @@ static void hclge_reset_event(struct pci_dev *pdev, struct hnae3_handle *handle)
else if (time_after(jiffies, (hdev->last_reset_time + 4 * 5 * HZ)))
hdev->reset_level = HNAE3_FUNC_RESET;
- dev_info(&hdev->pdev->dev, "received reset event , reset type is %d",
+ dev_info(&hdev->pdev->dev, "received reset event, reset type is %d\n",
hdev->reset_level);
/* request reset & schedule reset task */
@@ -6089,7 +6180,7 @@ static void hclge_enable_fd(struct hnae3_handle *handle, bool enable)
bool clear;
hdev->fd_en = enable;
- clear = hdev->fd_active_type == HCLGE_FD_ARFS_ACTIVE ? true : false;
+ clear = hdev->fd_active_type == HCLGE_FD_ARFS_ACTIVE;
if (!enable)
hclge_del_all_fd_entries(handle, clear);
else
@@ -6127,6 +6218,89 @@ static void hclge_cfg_mac_mode(struct hclge_dev *hdev, bool enable)
"mac enable fail, ret =%d.\n", ret);
}
+static int hclge_config_switch_param(struct hclge_dev *hdev, int vfid,
+ u8 switch_param, u8 param_mask)
+{
+ struct hclge_mac_vlan_switch_cmd *req;
+ struct hclge_desc desc;
+ u32 func_id;
+ int ret;
+
+ func_id = hclge_get_port_number(HOST_PORT, 0, vfid, 0);
+ req = (struct hclge_mac_vlan_switch_cmd *)desc.data;
+ hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_MAC_VLAN_SWITCH_PARAM,
+ false);
+ req->roce_sel = HCLGE_MAC_VLAN_NIC_SEL;
+ req->func_id = cpu_to_le32(func_id);
+ req->switch_param = switch_param;
+ req->param_mask = param_mask;
+
+ ret = hclge_cmd_send(&hdev->hw, &desc, 1);
+ if (ret)
+ dev_err(&hdev->pdev->dev,
+ "set mac vlan switch parameter fail, ret = %d\n", ret);
+ return ret;
+}
+
+static void hclge_phy_link_status_wait(struct hclge_dev *hdev,
+ int link_ret)
+{
+#define HCLGE_PHY_LINK_STATUS_NUM 200
+
+ struct phy_device *phydev = hdev->hw.mac.phydev;
+ int i = 0;
+ int ret;
+
+ do {
+ ret = phy_read_status(phydev);
+ if (ret) {
+ dev_err(&hdev->pdev->dev,
+ "phy update link status fail, ret = %d\n", ret);
+ return;
+ }
+
+ if (phydev->link == link_ret)
+ break;
+
+ msleep(HCLGE_LINK_STATUS_MS);
+ } while (++i < HCLGE_PHY_LINK_STATUS_NUM);
+}
+
+static int hclge_mac_link_status_wait(struct hclge_dev *hdev, int link_ret)
+{
+#define HCLGE_MAC_LINK_STATUS_NUM 100
+
+ int i = 0;
+ int ret;
+
+ do {
+ ret = hclge_get_mac_link_status(hdev);
+ if (ret < 0)
+ return ret;
+ else if (ret == link_ret)
+ return 0;
+
+ msleep(HCLGE_LINK_STATUS_MS);
+ } while (++i < HCLGE_MAC_LINK_STATUS_NUM);
+ return -EBUSY;
+}
+
+static int hclge_mac_phy_link_status_wait(struct hclge_dev *hdev, bool en,
+ bool is_phy)
+{
+#define HCLGE_LINK_STATUS_DOWN 0
+#define HCLGE_LINK_STATUS_UP 1
+
+ int link_ret;
+
+ link_ret = en ? HCLGE_LINK_STATUS_UP : HCLGE_LINK_STATUS_DOWN;
+
+ if (is_phy)
+ hclge_phy_link_status_wait(hdev, link_ret);
+
+ return hclge_mac_link_status_wait(hdev, link_ret);
+}
+
static int hclge_set_app_loopback(struct hclge_dev *hdev, bool en)
{
struct hclge_config_mac_mode_cmd *req;
@@ -6163,20 +6337,14 @@ static int hclge_set_app_loopback(struct hclge_dev *hdev, bool en)
return ret;
}
-static int hclge_set_serdes_loopback(struct hclge_dev *hdev, bool en,
+static int hclge_cfg_serdes_loopback(struct hclge_dev *hdev, bool en,
enum hnae3_loop loop_mode)
{
#define HCLGE_SERDES_RETRY_MS 10
#define HCLGE_SERDES_RETRY_NUM 100
-#define HCLGE_MAC_LINK_STATUS_MS 10
-#define HCLGE_MAC_LINK_STATUS_NUM 100
-#define HCLGE_MAC_LINK_STATUS_DOWN 0
-#define HCLGE_MAC_LINK_STATUS_UP 1
-
struct hclge_serdes_lb_cmd *req;
struct hclge_desc desc;
- int mac_link_ret = 0;
int ret, i = 0;
u8 loop_mode_b;
@@ -6199,10 +6367,8 @@ static int hclge_set_serdes_loopback(struct hclge_dev *hdev, bool en,
if (en) {
req->enable = loop_mode_b;
req->mask = loop_mode_b;
- mac_link_ret = HCLGE_MAC_LINK_STATUS_UP;
} else {
req->mask = loop_mode_b;
- mac_link_ret = HCLGE_MAC_LINK_STATUS_DOWN;
}
ret = hclge_cmd_send(&hdev->hw, &desc, 1);
@@ -6232,21 +6398,84 @@ static int hclge_set_serdes_loopback(struct hclge_dev *hdev, bool en,
dev_err(&hdev->pdev->dev, "serdes loopback set failed in fw\n");
return -EIO;
}
+ return ret;
+}
+
+static int hclge_set_serdes_loopback(struct hclge_dev *hdev, bool en,
+ enum hnae3_loop loop_mode)
+{
+ int ret;
+
+ ret = hclge_cfg_serdes_loopback(hdev, en, loop_mode);
+ if (ret)
+ return ret;
hclge_cfg_mac_mode(hdev, en);
- i = 0;
- do {
- /* serdes Internal loopback, independent of the network cable.*/
- msleep(HCLGE_MAC_LINK_STATUS_MS);
- ret = hclge_get_mac_link_status(hdev);
- if (ret == mac_link_ret)
- return 0;
- } while (++i < HCLGE_MAC_LINK_STATUS_NUM);
+ ret = hclge_mac_phy_link_status_wait(hdev, en, FALSE);
+ if (ret)
+ dev_err(&hdev->pdev->dev,
+ "serdes loopback config mac mode timeout\n");
- dev_err(&hdev->pdev->dev, "config mac mode timeout\n");
+ return ret;
+}
- return -EBUSY;
+static int hclge_enable_phy_loopback(struct hclge_dev *hdev,
+ struct phy_device *phydev)
+{
+ int ret;
+
+ if (!phydev->suspended) {
+ ret = phy_suspend(phydev);
+ if (ret)
+ return ret;
+ }
+
+ ret = phy_resume(phydev);
+ if (ret)
+ return ret;
+
+ return phy_loopback(phydev, true);
+}
+
+static int hclge_disable_phy_loopback(struct hclge_dev *hdev,
+ struct phy_device *phydev)
+{
+ int ret;
+
+ ret = phy_loopback(phydev, false);
+ if (ret)
+ return ret;
+
+ return phy_suspend(phydev);
+}
+
+static int hclge_set_phy_loopback(struct hclge_dev *hdev, bool en)
+{
+ struct phy_device *phydev = hdev->hw.mac.phydev;
+ int ret;
+
+ if (!phydev)
+ return -ENOTSUPP;
+
+ if (en)
+ ret = hclge_enable_phy_loopback(hdev, phydev);
+ else
+ ret = hclge_disable_phy_loopback(hdev, phydev);
+ if (ret) {
+ dev_err(&hdev->pdev->dev,
+ "set phy loopback fail, ret = %d\n", ret);
+ return ret;
+ }
+
+ hclge_cfg_mac_mode(hdev, en);
+
+ ret = hclge_mac_phy_link_status_wait(hdev, en, TRUE);
+ if (ret)
+ dev_err(&hdev->pdev->dev,
+ "phy loopback config mac mode timeout\n");
+
+ return ret;
}
static int hclge_tqp_enable(struct hclge_dev *hdev, unsigned int tqp_id,
@@ -6278,6 +6507,20 @@ static int hclge_set_loopback(struct hnae3_handle *handle,
struct hclge_dev *hdev = vport->back;
int i, ret;
+ /* Loopback can be enabled in three places: SSU, MAC, and serdes. By
+ * default, SSU loopback is enabled, so if the SMAC and the DMAC are
+ * the same, the packets are looped back in the SSU. If SSU loopback
+ * is disabled, packets can reach MAC even if SMAC is the same as DMAC.
+ */
+ if (hdev->pdev->revision >= 0x21) {
+ u8 switch_param = en ? 0 : BIT(HCLGE_SWITCH_ALW_LPBK_B);
+
+ ret = hclge_config_switch_param(hdev, PF_VPORT_ID, switch_param,
+ HCLGE_SWITCH_ALW_LPBK_MASK);
+ if (ret)
+ return ret;
+ }
+
switch (loop_mode) {
case HNAE3_LOOP_APP:
ret = hclge_set_app_loopback(hdev, en);
@@ -6286,6 +6529,9 @@ static int hclge_set_loopback(struct hnae3_handle *handle,
case HNAE3_LOOP_PARALLEL_SERDES:
ret = hclge_set_serdes_loopback(hdev, en, loop_mode);
break;
+ case HNAE3_LOOP_PHY:
+ ret = hclge_set_phy_loopback(hdev, en);
+ break;
default:
ret = -ENOTSUPP;
dev_err(&hdev->pdev->dev,
@@ -6306,6 +6552,22 @@ static int hclge_set_loopback(struct hnae3_handle *handle,
return 0;
}
+static int hclge_set_default_loopback(struct hclge_dev *hdev)
+{
+ int ret;
+
+ ret = hclge_set_app_loopback(hdev, false);
+ if (ret)
+ return ret;
+
+ ret = hclge_cfg_serdes_loopback(hdev, false, HNAE3_LOOP_SERIAL_SERDES);
+ if (ret)
+ return ret;
+
+ return hclge_cfg_serdes_loopback(hdev, false,
+ HNAE3_LOOP_PARALLEL_SERDES);
+}
+
static void hclge_reset_tqp_stats(struct hnae3_handle *handle)
{
struct hclge_vport *vport = hclge_get_vport(handle);
@@ -7193,7 +7455,7 @@ static int hclge_set_mac_addr(struct hnae3_handle *handle, void *p,
is_broadcast_ether_addr(new_addr) ||
is_multicast_ether_addr(new_addr)) {
dev_err(&hdev->pdev->dev,
- "Change uc mac err! invalid mac:%p.\n",
+ "Change uc mac err! invalid mac:%pM.\n",
new_addr);
return -EINVAL;
}
@@ -7298,7 +7560,7 @@ static void hclge_enable_vlan_filter(struct hnae3_handle *handle, bool enable)
}
static int hclge_set_vf_vlan_common(struct hclge_dev *hdev, u16 vfid,
- bool is_kill, u16 vlan, u8 qos,
+ bool is_kill, u16 vlan,
__be16 proto)
{
#define HCLGE_MAX_VF_BYTES 16
@@ -7409,7 +7671,7 @@ static int hclge_set_port_vlan_filter(struct hclge_dev *hdev, __be16 proto,
}
static int hclge_set_vlan_filter_hw(struct hclge_dev *hdev, __be16 proto,
- u16 vport_id, u16 vlan_id, u8 qos,
+ u16 vport_id, u16 vlan_id,
bool is_kill)
{
u16 vport_idx, vport_num = 0;
@@ -7419,7 +7681,7 @@ static int hclge_set_vlan_filter_hw(struct hclge_dev *hdev, __be16 proto,
return 0;
ret = hclge_set_vf_vlan_common(hdev, vport_id, is_kill, vlan_id,
- 0, proto);
+ proto);
if (ret) {
dev_err(&hdev->pdev->dev,
"Set %d vport vlan filter config fail, ret =%d.\n",
@@ -7463,6 +7725,7 @@ static int hclge_set_vlan_tx_offload_cfg(struct hclge_vport *vport)
struct hclge_vport_vtag_tx_cfg_cmd *req;
struct hclge_dev *hdev = vport->back;
struct hclge_desc desc;
+ u16 bmap_index;
int status;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_VLAN_PORT_TX_CFG, false);
@@ -7485,8 +7748,10 @@ static int hclge_set_vlan_tx_offload_cfg(struct hclge_vport *vport)
hnae3_set_bit(req->vport_vlan_cfg, HCLGE_CFG_NIC_ROCE_SEL_B, 0);
req->vf_offset = vport->vport_id / HCLGE_VF_NUM_PER_CMD;
- req->vf_bitmap[req->vf_offset] =
- 1 << (vport->vport_id % HCLGE_VF_NUM_PER_BYTE);
+ bmap_index = vport->vport_id % HCLGE_VF_NUM_PER_CMD /
+ HCLGE_VF_NUM_PER_BYTE;
+ req->vf_bitmap[bmap_index] =
+ 1U << (vport->vport_id % HCLGE_VF_NUM_PER_BYTE);
status = hclge_cmd_send(&hdev->hw, &desc, 1);
if (status)
@@ -7503,6 +7768,7 @@ static int hclge_set_vlan_rx_offload_cfg(struct hclge_vport *vport)
struct hclge_vport_vtag_rx_cfg_cmd *req;
struct hclge_dev *hdev = vport->back;
struct hclge_desc desc;
+ u16 bmap_index;
int status;
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_VLAN_PORT_RX_CFG, false);
@@ -7518,8 +7784,10 @@ static int hclge_set_vlan_rx_offload_cfg(struct hclge_vport *vport)
vcfg->vlan2_vlan_prionly ? 1 : 0);
req->vf_offset = vport->vport_id / HCLGE_VF_NUM_PER_CMD;
- req->vf_bitmap[req->vf_offset] =
- 1 << (vport->vport_id % HCLGE_VF_NUM_PER_BYTE);
+ bmap_index = vport->vport_id % HCLGE_VF_NUM_PER_CMD /
+ HCLGE_VF_NUM_PER_BYTE;
+ req->vf_bitmap[bmap_index] =
+ 1U << (vport->vport_id % HCLGE_VF_NUM_PER_BYTE);
status = hclge_cmd_send(&hdev->hw, &desc, 1);
if (status)
@@ -7706,7 +7974,7 @@ static int hclge_add_vport_all_vlan_table(struct hclge_vport *vport)
if (!vlan->hd_tbl_status) {
ret = hclge_set_vlan_filter_hw(hdev, htons(ETH_P_8021Q),
vport->vport_id,
- vlan->vlan_id, 0, false);
+ vlan->vlan_id, false);
if (ret) {
dev_err(&hdev->pdev->dev,
"restore vport vlan list failed, ret=%d\n",
@@ -7732,7 +8000,7 @@ static void hclge_rm_vport_vlan_table(struct hclge_vport *vport, u16 vlan_id,
hclge_set_vlan_filter_hw(hdev,
htons(ETH_P_8021Q),
vport->vport_id,
- vlan_id, 0,
+ vlan_id,
true);
list_del(&vlan->node);
@@ -7752,7 +8020,7 @@ void hclge_rm_vport_all_vlan_table(struct hclge_vport *vport, bool is_del_list)
hclge_set_vlan_filter_hw(hdev,
htons(ETH_P_8021Q),
vport->vport_id,
- vlan->vlan_id, 0,
+ vlan->vlan_id,
true);
vlan->hd_tbl_status = false;
@@ -7785,7 +8053,7 @@ static void hclge_restore_vlan_table(struct hnae3_handle *handle)
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_vport_vlan_cfg *vlan, *tmp;
struct hclge_dev *hdev = vport->back;
- u16 vlan_proto, qos;
+ u16 vlan_proto;
u16 state, vlan_id;
int i;
@@ -7794,12 +8062,11 @@ static void hclge_restore_vlan_table(struct hnae3_handle *handle)
vport = &hdev->vport[i];
vlan_proto = vport->port_base_vlan_cfg.vlan_info.vlan_proto;
vlan_id = vport->port_base_vlan_cfg.vlan_info.vlan_tag;
- qos = vport->port_base_vlan_cfg.vlan_info.qos;
state = vport->port_base_vlan_cfg.state;
if (state != HNAE3_PORT_BASE_VLAN_DISABLE) {
hclge_set_vlan_filter_hw(hdev, htons(vlan_proto),
- vport->vport_id, vlan_id, qos,
+ vport->vport_id, vlan_id,
false);
continue;
}
@@ -7809,7 +8076,7 @@ static void hclge_restore_vlan_table(struct hnae3_handle *handle)
hclge_set_vlan_filter_hw(hdev,
htons(ETH_P_8021Q),
vport->vport_id,
- vlan->vlan_id, 0,
+ vlan->vlan_id,
false);
}
}
@@ -7849,12 +8116,12 @@ static int hclge_update_vlan_filter_entries(struct hclge_vport *vport,
htons(new_info->vlan_proto),
vport->vport_id,
new_info->vlan_tag,
- new_info->qos, false);
+ false);
}
ret = hclge_set_vlan_filter_hw(hdev, htons(old_info->vlan_proto),
vport->vport_id, old_info->vlan_tag,
- old_info->qos, true);
+ true);
if (ret)
return ret;
@@ -7881,7 +8148,7 @@ int hclge_update_port_base_vlan_cfg(struct hclge_vport *vport, u16 state,
htons(vlan_info->vlan_proto),
vport->vport_id,
vlan_info->vlan_tag,
- vlan_info->qos, false);
+ false);
if (ret)
return ret;
@@ -7890,7 +8157,7 @@ int hclge_update_port_base_vlan_cfg(struct hclge_vport *vport, u16 state,
htons(old_vlan_info->vlan_proto),
vport->vport_id,
old_vlan_info->vlan_tag,
- old_vlan_info->qos, true);
+ true);
if (ret)
return ret;
@@ -8011,7 +8278,7 @@ int hclge_set_vlan_filter(struct hnae3_handle *handle, __be16 proto,
*/
if (handle->port_base_vlan_state == HNAE3_PORT_BASE_VLAN_DISABLE) {
ret = hclge_set_vlan_filter_hw(hdev, proto, vport->vport_id,
- vlan_id, 0, is_kill);
+ vlan_id, is_kill);
writen_to_tbl = true;
}
@@ -8047,7 +8314,7 @@ static void hclge_sync_vlan_filter(struct hclge_dev *hdev)
while (vlan_id != VLAN_N_VID) {
ret = hclge_set_vlan_filter_hw(hdev, htons(ETH_P_8021Q),
vport->vport_id, vlan_id,
- 0, true);
+ true);
if (ret && ret != -EINVAL)
return;
@@ -8218,11 +8485,12 @@ int hclge_reset_tqp(struct hnae3_handle *handle, u16 queue_id)
}
while (reset_try_times++ < HCLGE_TQP_RESET_TRY_TIMES) {
- /* Wait for tqp hw reset */
- msleep(20);
reset_status = hclge_get_reset_status(hdev, queue_gid);
if (reset_status)
break;
+
+ /* Wait for tqp hw reset */
+ usleep_range(1000, 1200);
}
if (reset_try_times >= HCLGE_TQP_RESET_TRY_TIMES) {
@@ -8256,11 +8524,12 @@ void hclge_reset_vf_queue(struct hclge_vport *vport, u16 queue_id)
}
while (reset_try_times++ < HCLGE_TQP_RESET_TRY_TIMES) {
- /* Wait for tqp hw reset */
- msleep(20);
reset_status = hclge_get_reset_status(hdev, queue_gid);
if (reset_status)
break;
+
+ /* Wait for tqp hw reset */
+ usleep_range(1000, 1200);
}
if (reset_try_times >= HCLGE_TQP_RESET_TRY_TIMES) {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
index 7c28933..3e9574a 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h
@@ -119,7 +119,7 @@
#define HCLGE_DEFAULT_UMV_SPACE_PER_PF \
(HCLGE_UMV_TBL_SIZE / HCLGE_MAX_PF_NUM)
-#define HCLGE_TQP_RESET_TRY_TIMES 10
+#define HCLGE_TQP_RESET_TRY_TIMES 200
#define HCLGE_PHY_PAGE_MDIX 0
#define HCLGE_PHY_PAGE_COPPER 0
@@ -148,6 +148,8 @@ enum HLCGE_PORT_TYPE {
NETWORK_PORT
};
+#define PF_VPORT_ID 0
+
#define HCLGE_PF_ID_S 0
#define HCLGE_PF_ID_M GENMASK(2, 0)
#define HCLGE_VF_ID_S 3
@@ -164,6 +166,7 @@ enum HLCGE_PORT_TYPE {
#define HCLGE_GLOBAL_RESET_BIT 0
#define HCLGE_CORE_RESET_BIT 1
#define HCLGE_IMP_RESET_BIT 2
+#define HCLGE_RESET_INT_M GENMASK(2, 0)
#define HCLGE_FUN_RST_ING 0x20C00
#define HCLGE_FUN_RST_ING_B 0
@@ -178,6 +181,8 @@ enum HLCGE_PORT_TYPE {
#define HCLGE_VECTOR0_RX_CMDQ_INT_B 1
#define HCLGE_VECTOR0_IMP_RESET_INT_B 1
+#define HCLGE_VECTOR0_IMP_CMDQ_ERR_B 4U
+#define HCLGE_VECTOR0_IMP_RD_POISON_B 5U
#define HCLGE_MAC_DEFAULT_FRAME \
(ETH_HLEN + ETH_FCS_LEN + 2 * VLAN_HLEN + ETH_DATA_LEN)
@@ -539,50 +544,6 @@ struct key_info {
u8 key_length; /* use bit as unit */
};
-static const struct key_info meta_data_key_info[] = {
- { PACKET_TYPE_ID, 6},
- { IP_FRAGEMENT, 1},
- { ROCE_TYPE, 1},
- { NEXT_KEY, 5},
- { VLAN_NUMBER, 2},
- { SRC_VPORT, 12},
- { DST_VPORT, 12},
- { TUNNEL_PACKET, 1},
-};
-
-static const struct key_info tuple_key_info[] = {
- { OUTER_DST_MAC, 48},
- { OUTER_SRC_MAC, 48},
- { OUTER_VLAN_TAG_FST, 16},
- { OUTER_VLAN_TAG_SEC, 16},
- { OUTER_ETH_TYPE, 16},
- { OUTER_L2_RSV, 16},
- { OUTER_IP_TOS, 8},
- { OUTER_IP_PROTO, 8},
- { OUTER_SRC_IP, 32},
- { OUTER_DST_IP, 32},
- { OUTER_L3_RSV, 16},
- { OUTER_SRC_PORT, 16},
- { OUTER_DST_PORT, 16},
- { OUTER_L4_RSV, 32},
- { OUTER_TUN_VNI, 24},
- { OUTER_TUN_FLOW_ID, 8},
- { INNER_DST_MAC, 48},
- { INNER_SRC_MAC, 48},
- { INNER_VLAN_TAG_FST, 16},
- { INNER_VLAN_TAG_SEC, 16},
- { INNER_ETH_TYPE, 16},
- { INNER_L2_RSV, 16},
- { INNER_IP_TOS, 8},
- { INNER_IP_PROTO, 8},
- { INNER_SRC_IP, 32},
- { INNER_DST_IP, 32},
- { INNER_L3_RSV, 16},
- { INNER_SRC_PORT, 16},
- { INNER_DST_PORT, 16},
- { INNER_L4_RSV, 32},
-};
-
#define MAX_KEY_LENGTH 400
#define MAX_KEY_DWORDS DIV_ROUND_UP(MAX_KEY_LENGTH / 8, 4)
#define MAX_KEY_BYTES (MAX_KEY_DWORDS * 4)
@@ -698,6 +659,7 @@ struct hclge_rst_stats {
u32 global_rst_cnt; /* the number of GLOBAL */
u32 imp_rst_cnt; /* the number of IMP reset */
u32 reset_cnt; /* the number of reset */
+ u32 reset_fail_cnt; /* the number of reset fail */
};
/* time and register status when mac tunnel interruption occur */
@@ -764,7 +726,6 @@ struct hclge_dev {
unsigned long reset_request; /* reset has been requested */
unsigned long reset_pending; /* client rst is pending to be served */
struct hclge_rst_stats rst_stats;
- u32 reset_fail_cnt;
u32 fw_version;
u16 num_vmdq_vport; /* Num vmdq vport this PF has set up */
u16 num_tqps; /* Num task queue pairs of this PF */
@@ -1030,4 +991,6 @@ int hclge_push_vf_port_base_vlan_info(struct hclge_vport *vport, u8 vfid,
void hclge_task_schedule(struct hclge_dev *hdev, unsigned long delay_time);
int hclge_query_bd_num_cmd_send(struct hclge_dev *hdev,
struct hclge_desc *desc);
+void hclge_report_hw_error(struct hclge_dev *hdev,
+ enum hnae3_hw_error_type type);
#endif
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
index e829101..9f0e35f 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c
@@ -81,16 +81,13 @@ static int hclge_shaper_para_calc(u32 ir, u8 shaper_level,
return 0;
} else if (ir_calc > ir) {
/* Increasing the denominator to select ir_s value */
- while (ir_calc > ir) {
+ while (ir_calc >= ir && ir) {
ir_s_calc++;
ir_calc = DIVISOR_IR_B_126 / (tick * (1 << ir_s_calc));
}
- if (ir_calc == ir)
- *ir_b = 126;
- else
- *ir_b = (ir * tick * (1 << ir_s_calc) +
- (DIVISOR_CLK >> 1)) / DIVISOR_CLK;
+ *ir_b = (ir * tick * (1 << ir_s_calc) + (DIVISOR_CLK >> 1)) /
+ DIVISOR_CLK;
} else {
/* Increasing the numerator to select ir_u value */
u32 numerator;
@@ -104,7 +101,7 @@ static int hclge_shaper_para_calc(u32 ir, u8 shaper_level,
if (ir_calc == ir) {
*ir_b = 126;
} else {
- u32 denominator = (DIVISOR_CLK * (1 << --ir_u_calc));
+ u32 denominator = DIVISOR_CLK * (1 << --ir_u_calc);
*ir_b = (ir * tick + (denominator >> 1)) / denominator;
}
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c
index 4c2c945..d5d1cc5 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c
@@ -74,7 +74,7 @@ static bool hclgevf_cmd_csq_done(struct hclgevf_hw *hw)
static bool hclgevf_is_special_opcode(u16 opcode)
{
- u16 spec_opcode[] = {0x30, 0x31, 0x32};
+ static const u16 spec_opcode[] = {0x30, 0x31, 0x32};
int i;
for (i = 0; i < ARRAY_SIZE(spec_opcode); i++) {
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
index 594cae8..e3090b3 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c
@@ -743,7 +743,7 @@ static int hclgevf_get_rss(struct hnae3_handle *handle, u32 *indir, u8 *key,
}
static int hclgevf_set_rss(struct hnae3_handle *handle, const u32 *indir,
- const u8 *key, const u8 hfunc)
+ const u8 *key, const u8 hfunc)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
@@ -2060,9 +2060,10 @@ static int hclgevf_config_gro(struct hclgevf_dev *hdev, bool en)
static int hclgevf_rss_init_hw(struct hclgevf_dev *hdev)
{
struct hclgevf_rss_cfg *rss_cfg = &hdev->rss_cfg;
- int i, ret;
+ int ret;
+ u32 i;
- rss_cfg->rss_size = hdev->rss_size_max;
+ rss_cfg->rss_size = hdev->nic.kinfo.rss_size;
if (hdev->pdev->revision >= 0x21) {
rss_cfg->hash_algo = HCLGEVF_RSS_HASH_ALGO_SIMPLE;
@@ -2099,13 +2100,13 @@ static int hclgevf_rss_init_hw(struct hclgevf_dev *hdev)
/* Initialize RSS indirect table */
for (i = 0; i < HCLGEVF_RSS_IND_TBL_SIZE; i++)
- rss_cfg->rss_indirection_tbl[i] = i % hdev->rss_size_max;
+ rss_cfg->rss_indirection_tbl[i] = i % rss_cfg->rss_size;
ret = hclgevf_set_rss_indir_table(hdev);
if (ret)
return ret;
- return hclgevf_set_rss_tc_mode(hdev, hdev->rss_size_max);
+ return hclgevf_set_rss_tc_mode(hdev, rss_cfg->rss_size);
}
static int hclgevf_init_vlan_config(struct hclgevf_dev *hdev)
@@ -2835,6 +2836,77 @@ static void hclgevf_get_tqps_and_rss_info(struct hnae3_handle *handle,
*max_rss_size = hdev->rss_size_max;
}
+static void hclgevf_update_rss_size(struct hnae3_handle *handle,
+ u32 new_tqps_num)
+{
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
+ u16 max_rss_size;
+
+ kinfo->req_rss_size = new_tqps_num;
+
+ max_rss_size = min_t(u16, hdev->rss_size_max,
+ hdev->num_tqps / kinfo->num_tc);
+
+ /* Use the user's configuration when it is not larger than
+ * max_rss_size, otherwise, use the maximum specification value.
+ */
+ if (kinfo->req_rss_size != kinfo->rss_size && kinfo->req_rss_size &&
+ kinfo->req_rss_size <= max_rss_size)
+ kinfo->rss_size = kinfo->req_rss_size;
+ else if (kinfo->rss_size > max_rss_size ||
+ (!kinfo->req_rss_size && kinfo->rss_size < max_rss_size))
+ kinfo->rss_size = max_rss_size;
+
+ kinfo->num_tqps = kinfo->num_tc * kinfo->rss_size;
+}
+
+static int hclgevf_set_channels(struct hnae3_handle *handle, u32 new_tqps_num,
+ bool rxfh_configured)
+{
+ struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
+ struct hnae3_knic_private_info *kinfo = &handle->kinfo;
+ u16 cur_rss_size = kinfo->rss_size;
+ u16 cur_tqps = kinfo->num_tqps;
+ u32 *rss_indir;
+ unsigned int i;
+ int ret;
+
+ hclgevf_update_rss_size(handle, new_tqps_num);
+
+ ret = hclgevf_set_rss_tc_mode(hdev, kinfo->rss_size);
+ if (ret)
+ return ret;
+
+ /* RSS indirection table has been configuared by user */
+ if (rxfh_configured)
+ goto out;
+
+ /* Reinitializes the rss indirect table according to the new RSS size */
+ rss_indir = kcalloc(HCLGEVF_RSS_IND_TBL_SIZE, sizeof(u32), GFP_KERNEL);
+ if (!rss_indir)
+ return -ENOMEM;
+
+ for (i = 0; i < HCLGEVF_RSS_IND_TBL_SIZE; i++)
+ rss_indir[i] = i % kinfo->rss_size;
+
+ ret = hclgevf_set_rss(handle, rss_indir, NULL, 0);
+ if (ret)
+ dev_err(&hdev->pdev->dev, "set rss indir table fail, ret=%d\n",
+ ret);
+
+ kfree(rss_indir);
+
+out:
+ if (!ret)
+ dev_info(&hdev->pdev->dev,
+ "Channels changed, rss_size from %u to %u, tqps from %u to %u",
+ cur_rss_size, kinfo->rss_size,
+ cur_tqps, kinfo->rss_size * kinfo->num_tc);
+
+ return ret;
+}
+
static int hclgevf_get_status(struct hnae3_handle *handle)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
@@ -3042,6 +3114,7 @@ static const struct hnae3_ae_ops hclgevf_ops = {
.enable_hw_strip_rxvtag = hclgevf_en_hw_strip_rxvtag,
.reset_event = hclgevf_reset_event,
.set_default_reset_request = hclgevf_set_def_reset_request,
+ .set_channels = hclgevf_set_channels,
.get_channels = hclgevf_get_channels,
.get_tqps_and_rss_info = hclgevf_get_tqps_and_rss_info,
.get_regs_len = hclgevf_get_regs_len,
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
index 6a96987..a108191 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c
@@ -277,9 +277,9 @@ void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev)
switch (msg_q[0]) {
case HCLGE_MBX_LINK_STAT_CHANGE:
- link_status = le16_to_cpu(msg_q[1]);
+ link_status = msg_q[1];
memcpy(&speed, &msg_q[2], sizeof(speed));
- duplex = (u8)le16_to_cpu(msg_q[4]);
+ duplex = (u8)msg_q[4];
/* update upper layer with new link link status */
hclgevf_update_link_status(hdev, link_status);
@@ -287,7 +287,7 @@ void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev)
break;
case HCLGE_MBX_LINK_STAT_MODE:
- idx = (u8)le16_to_cpu(msg_q[1]);
+ idx = (u8)msg_q[1];
if (idx)
memcpy(&hdev->hw.mac.supported, &msg_q[2],
sizeof(unsigned long));
@@ -301,14 +301,14 @@ void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev)
* has been completely reset. After this stack should
* eventually be re-initialized.
*/
- reset_type = le16_to_cpu(msg_q[1]);
+ reset_type = (enum hnae3_reset_type)msg_q[1];
set_bit(reset_type, &hdev->reset_pending);
set_bit(HCLGEVF_RESET_PENDING, &hdev->reset_state);
hclgevf_reset_task_schedule(hdev);
break;
case HCLGE_MBX_PUSH_VLAN_INFO:
- state = le16_to_cpu(msg_q[1]);
+ state = msg_q[1];
vlan_info = &msg_q[1];
hclgevf_update_port_base_vlan_info(hdev, state,
(u8 *)vlan_info, 8);
diff --git a/drivers/net/ethernet/ibm/ehea/ehea_main.c b/drivers/net/ethernet/ibm/ehea/ehea_main.c
index cca71ba..13e30eb 100644
--- a/drivers/net/ethernet/ibm/ehea/ehea_main.c
+++ b/drivers/net/ethernet/ibm/ehea/ehea_main.c
@@ -1577,20 +1577,16 @@ static int ehea_clean_portres(struct ehea_port *port, struct ehea_port_res *pr)
ehea_destroy_eq(pr->eq);
for (i = 0; i < pr->rq1_skba.len; i++)
- if (pr->rq1_skba.arr[i])
- dev_kfree_skb(pr->rq1_skba.arr[i]);
+ dev_kfree_skb(pr->rq1_skba.arr[i]);
for (i = 0; i < pr->rq2_skba.len; i++)
- if (pr->rq2_skba.arr[i])
- dev_kfree_skb(pr->rq2_skba.arr[i]);
+ dev_kfree_skb(pr->rq2_skba.arr[i]);
for (i = 0; i < pr->rq3_skba.len; i++)
- if (pr->rq3_skba.arr[i])
- dev_kfree_skb(pr->rq3_skba.arr[i]);
+ dev_kfree_skb(pr->rq3_skba.arr[i]);
for (i = 0; i < pr->sq_skba.len; i++)
- if (pr->sq_skba.arr[i])
- dev_kfree_skb(pr->sq_skba.arr[i]);
+ dev_kfree_skb(pr->sq_skba.arr[i]);
vfree(pr->rq1_skba.arr);
vfree(pr->rq2_skba.arr);
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
index 07efa2b..4f83f97 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -1983,6 +1983,10 @@ static void __ibmvnic_reset(struct work_struct *work)
rwi = get_next_rwi(adapter);
while (rwi) {
+ if (adapter->state == VNIC_REMOVING ||
+ adapter->state == VNIC_REMOVED)
+ goto out;
+
if (adapter->force_reset_recovery) {
adapter->force_reset_recovery = false;
rc = do_hard_reset(adapter, rwi, reset_state);
@@ -2007,7 +2011,7 @@ static void __ibmvnic_reset(struct work_struct *work)
netdev_dbg(adapter->netdev, "Reset failed\n");
free_all_rwi(adapter);
}
-
+out:
adapter->resetting = false;
if (we_lock_rtnl)
rtnl_unlock();
diff --git a/drivers/net/ethernet/intel/e1000/e1000_ethtool.c b/drivers/net/ethernet/intel/e1000/e1000_ethtool.c
index a410085..71d3d88 100644
--- a/drivers/net/ethernet/intel/e1000/e1000_ethtool.c
+++ b/drivers/net/ethernet/intel/e1000/e1000_ethtool.c
@@ -937,8 +937,7 @@ static void e1000_free_desc_rings(struct e1000_adapter *adapter)
txdr->buffer_info[i].dma,
txdr->buffer_info[i].length,
DMA_TO_DEVICE);
- if (txdr->buffer_info[i].skb)
- dev_kfree_skb(txdr->buffer_info[i].skb);
+ dev_kfree_skb(txdr->buffer_info[i].skb);
}
}
diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c
index 6b6ba1c..86493fe 100644
--- a/drivers/net/ethernet/intel/e1000/e1000_main.c
+++ b/drivers/net/ethernet/intel/e1000/e1000_main.c
@@ -4175,8 +4175,7 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_adapter *adapter,
/* an error means any chain goes out the window
* too
*/
- if (rx_ring->rx_skb_top)
- dev_kfree_skb(rx_ring->rx_skb_top);
+ dev_kfree_skb(rx_ring->rx_skb_top);
rx_ring->rx_skb_top = NULL;
goto next_desc;
}
diff --git a/drivers/net/ethernet/intel/e1000e/ethtool.c b/drivers/net/ethernet/intel/e1000e/ethtool.c
index 0834269..de8c581 100644
--- a/drivers/net/ethernet/intel/e1000e/ethtool.c
+++ b/drivers/net/ethernet/intel/e1000e/ethtool.c
@@ -1126,8 +1126,7 @@ static void e1000_free_desc_rings(struct e1000_adapter *adapter)
buffer_info->dma,
buffer_info->length,
DMA_TO_DEVICE);
- if (buffer_info->skb)
- dev_kfree_skb(buffer_info->skb);
+ dev_kfree_skb(buffer_info->skb);
}
}
@@ -1139,8 +1138,7 @@ static void e1000_free_desc_rings(struct e1000_adapter *adapter)
dma_unmap_single(&pdev->dev,
buffer_info->dma,
2048, DMA_FROM_DEVICE);
- if (buffer_info->skb)
- dev_kfree_skb(buffer_info->skb);
+ dev_kfree_skb(buffer_info->skb);
}
}
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 8a3f035..d7d56e4 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -1780,8 +1780,8 @@ static irqreturn_t e1000_intr_msi(int __always_unused irq, void *data)
}
/* guard against interrupt when we're going down */
if (!test_bit(__E1000_DOWN, &adapter->state))
- queue_delayed_work(adapter->e1000_workqueue,
- &adapter->watchdog_task, 1);
+ mod_delayed_work(adapter->e1000_workqueue,
+ &adapter->watchdog_task, HZ);
}
/* Reset on uncorrectable ECC error */
@@ -1861,8 +1861,8 @@ static irqreturn_t e1000_intr(int __always_unused irq, void *data)
}
/* guard against interrupt when we're going down */
if (!test_bit(__E1000_DOWN, &adapter->state))
- queue_delayed_work(adapter->e1000_workqueue,
- &adapter->watchdog_task, 1);
+ mod_delayed_work(adapter->e1000_workqueue,
+ &adapter->watchdog_task, HZ);
}
/* Reset on uncorrectable ECC error */
@@ -1907,8 +1907,8 @@ static irqreturn_t e1000_msix_other(int __always_unused irq, void *data)
hw->mac.get_link_status = true;
/* guard against interrupt when we're going down */
if (!test_bit(__E1000_DOWN, &adapter->state))
- queue_delayed_work(adapter->e1000_workqueue,
- &adapter->watchdog_task, 1);
+ mod_delayed_work(adapter->e1000_workqueue,
+ &adapter->watchdog_task, HZ);
}
if (!test_bit(__E1000_DOWN, &adapter->state))
diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_main.c b/drivers/net/ethernet/intel/fm10k/fm10k_main.c
index e0a2be5..2be9222 100644
--- a/drivers/net/ethernet/intel/fm10k/fm10k_main.c
+++ b/drivers/net/ethernet/intel/fm10k/fm10k_main.c
@@ -1073,9 +1073,11 @@ netdev_tx_t fm10k_xmit_frame_ring(struct sk_buff *skb,
* + 2 desc gap to keep tail from touching head
* otherwise try next time
*/
- for (f = 0; f < skb_shinfo(skb)->nr_frags; f++)
- count += TXD_USE_COUNT(skb_frag_size(
- &skb_shinfo(skb)->frags[f]));
+ for (f = 0; f < skb_shinfo(skb)->nr_frags; f++) {
+ skb_frag_t *frag = &skb_shinfo(skb)->frags[f];
+
+ count += TXD_USE_COUNT(skb_frag_size(frag));
+ }
if (fm10k_maybe_stop_tx(tx_ring, count + 3)) {
tx_ring->tx_stats.tx_busy++;
diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c
index d3e8548..09f7a24 100644
--- a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c
+++ b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c
@@ -253,8 +253,7 @@ static void fm10k_clean_rx_ring(struct fm10k_ring *rx_ring)
if (!rx_ring->rx_buffer)
return;
- if (rx_ring->skb)
- dev_kfree_skb(rx_ring->skb);
+ dev_kfree_skb(rx_ring->skb);
rx_ring->skb = NULL;
/* Free all the Rx ring sk_buffs */
diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h
index 3e535d3..f1a1bd3 100644
--- a/drivers/net/ethernet/intel/i40e/i40e.h
+++ b/drivers/net/ethernet/intel/i40e/i40e.h
@@ -131,7 +131,6 @@ enum i40e_state_t {
__I40E_PF_RESET_REQUESTED,
__I40E_CORE_RESET_REQUESTED,
__I40E_GLOBAL_RESET_REQUESTED,
- __I40E_EMP_RESET_REQUESTED,
__I40E_EMP_RESET_INTR_RECEIVED,
__I40E_SUSPENDED,
__I40E_PTP_TX_IN_PROGRESS,
diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq.c b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
index 814acbe..72c0488 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq.c
@@ -610,8 +610,10 @@ i40e_status i40e_init_adminq(struct i40e_hw *hw)
if (hw->aq.api_maj_ver > 1 ||
(hw->aq.api_maj_ver == 1 &&
- hw->aq.api_min_ver >= 8))
+ hw->aq.api_min_ver >= 8)) {
hw->flags |= I40E_HW_FLAG_FW_LLDP_PERSISTENT;
+ hw->flags |= I40E_HW_FLAG_DROP_MODE;
+ }
if (hw->aq.api_maj_ver > I40E_FW_API_VERSION_MAJOR) {
ret_code = I40E_ERR_FIRMWARE_API_VERSION;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
index 6536023..21cccec 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h
@@ -11,8 +11,8 @@
*/
#define I40E_FW_API_VERSION_MAJOR 0x0001
-#define I40E_FW_API_VERSION_MINOR_X722 0x0008
-#define I40E_FW_API_VERSION_MINOR_X710 0x0008
+#define I40E_FW_API_VERSION_MINOR_X722 0x0009
+#define I40E_FW_API_VERSION_MINOR_X710 0x0009
#define I40E_FW_MINOR_VERSION(_h) ((_h)->mac.type == I40E_MAC_XL710 ? \
I40E_FW_API_VERSION_MINOR_X710 : \
@@ -2051,20 +2051,21 @@ I40E_CHECK_CMD_LENGTH(i40e_aq_set_phy_config);
struct i40e_aq_set_mac_config {
__le16 max_frame_size;
u8 params;
-#define I40E_AQ_SET_MAC_CONFIG_CRC_EN 0x04
-#define I40E_AQ_SET_MAC_CONFIG_PACING_MASK 0x78
-#define I40E_AQ_SET_MAC_CONFIG_PACING_SHIFT 3
-#define I40E_AQ_SET_MAC_CONFIG_PACING_NONE 0x0
-#define I40E_AQ_SET_MAC_CONFIG_PACING_1B_13TX 0xF
-#define I40E_AQ_SET_MAC_CONFIG_PACING_1DW_9TX 0x9
-#define I40E_AQ_SET_MAC_CONFIG_PACING_1DW_4TX 0x8
-#define I40E_AQ_SET_MAC_CONFIG_PACING_3DW_7TX 0x7
-#define I40E_AQ_SET_MAC_CONFIG_PACING_2DW_3TX 0x6
-#define I40E_AQ_SET_MAC_CONFIG_PACING_1DW_1TX 0x5
-#define I40E_AQ_SET_MAC_CONFIG_PACING_3DW_2TX 0x4
-#define I40E_AQ_SET_MAC_CONFIG_PACING_7DW_3TX 0x3
-#define I40E_AQ_SET_MAC_CONFIG_PACING_4DW_1TX 0x2
-#define I40E_AQ_SET_MAC_CONFIG_PACING_9DW_1TX 0x1
+#define I40E_AQ_SET_MAC_CONFIG_CRC_EN 0x04
+#define I40E_AQ_SET_MAC_CONFIG_PACING_MASK 0x78
+#define I40E_AQ_SET_MAC_CONFIG_PACING_SHIFT 3
+#define I40E_AQ_SET_MAC_CONFIG_PACING_NONE 0x0
+#define I40E_AQ_SET_MAC_CONFIG_PACING_1B_13TX 0xF
+#define I40E_AQ_SET_MAC_CONFIG_PACING_1DW_9TX 0x9
+#define I40E_AQ_SET_MAC_CONFIG_PACING_1DW_4TX 0x8
+#define I40E_AQ_SET_MAC_CONFIG_PACING_3DW_7TX 0x7
+#define I40E_AQ_SET_MAC_CONFIG_PACING_2DW_3TX 0x6
+#define I40E_AQ_SET_MAC_CONFIG_PACING_1DW_1TX 0x5
+#define I40E_AQ_SET_MAC_CONFIG_PACING_3DW_2TX 0x4
+#define I40E_AQ_SET_MAC_CONFIG_PACING_7DW_3TX 0x3
+#define I40E_AQ_SET_MAC_CONFIG_PACING_4DW_1TX 0x2
+#define I40E_AQ_SET_MAC_CONFIG_PACING_9DW_1TX 0x1
+#define I40E_AQ_SET_MAC_CONFIG_DROP_BLOCKING_PACKET_EN 0x80
u8 tx_timer_priority; /* bitmap */
__le16 tx_timer_value;
__le16 fc_refresh_threshold;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_common.c b/drivers/net/ethernet/intel/i40e/i40e_common.c
index 906cf68..d37c6e0 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_common.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_common.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2013 - 2018 Intel Corporation. */
+#include "i40e.h"
#include "i40e_type.h"
#include "i40e_adminq.h"
#include "i40e_prototype.h"
@@ -13,7 +14,7 @@
* This function sets the mac type of the adapter based on the
* vendor ID and device ID stored in the hw structure.
**/
-static i40e_status i40e_set_mac_type(struct i40e_hw *hw)
+i40e_status i40e_set_mac_type(struct i40e_hw *hw)
{
i40e_status status = 0;
@@ -1577,19 +1578,22 @@ i40e_status i40e_aq_get_phy_capabilities(struct i40e_hw *hw,
status = i40e_asq_send_command(hw, &desc, abilities,
abilities_size, cmd_details);
- if (status)
- break;
-
- if (hw->aq.asq_last_status == I40E_AQ_RC_EIO) {
+ switch (hw->aq.asq_last_status) {
+ case I40E_AQ_RC_EIO:
status = I40E_ERR_UNKNOWN_PHY;
break;
- } else if (hw->aq.asq_last_status == I40E_AQ_RC_EAGAIN) {
+ case I40E_AQ_RC_EAGAIN:
usleep_range(1000, 2000);
total_delay++;
status = I40E_ERR_TIMEOUT;
+ break;
+ /* also covers I40E_AQ_RC_OK */
+ default:
+ break;
}
- } while ((hw->aq.asq_last_status != I40E_AQ_RC_OK) &&
- (total_delay < max_delay));
+
+ } while ((hw->aq.asq_last_status == I40E_AQ_RC_EAGAIN) &&
+ (total_delay < max_delay));
if (status)
return status;
@@ -1643,25 +1647,15 @@ enum i40e_status_code i40e_aq_set_phy_config(struct i40e_hw *hw,
return status;
}
-/**
- * i40e_set_fc
- * @hw: pointer to the hw struct
- * @aq_failures: buffer to return AdminQ failure information
- * @atomic_restart: whether to enable atomic link restart
- *
- * Set the requested flow control mode using set_phy_config.
- **/
-enum i40e_status_code i40e_set_fc(struct i40e_hw *hw, u8 *aq_failures,
- bool atomic_restart)
+static noinline_for_stack enum i40e_status_code
+i40e_set_fc_status(struct i40e_hw *hw,
+ struct i40e_aq_get_phy_abilities_resp *abilities,
+ bool atomic_restart)
{
- enum i40e_fc_mode fc_mode = hw->fc.requested_mode;
- struct i40e_aq_get_phy_abilities_resp abilities;
struct i40e_aq_set_phy_config config;
- enum i40e_status_code status;
+ enum i40e_fc_mode fc_mode = hw->fc.requested_mode;
u8 pause_mask = 0x0;
- *aq_failures = 0x0;
-
switch (fc_mode) {
case I40E_FC_FULL:
pause_mask |= I40E_AQ_PHY_FLAG_PAUSE_TX;
@@ -1677,6 +1671,48 @@ enum i40e_status_code i40e_set_fc(struct i40e_hw *hw, u8 *aq_failures,
break;
}
+ memset(&config, 0, sizeof(struct i40e_aq_set_phy_config));
+ /* clear the old pause settings */
+ config.abilities = abilities->abilities & ~(I40E_AQ_PHY_FLAG_PAUSE_TX) &
+ ~(I40E_AQ_PHY_FLAG_PAUSE_RX);
+ /* set the new abilities */
+ config.abilities |= pause_mask;
+ /* If the abilities have changed, then set the new config */
+ if (config.abilities == abilities->abilities)
+ return 0;
+
+ /* Auto restart link so settings take effect */
+ if (atomic_restart)
+ config.abilities |= I40E_AQ_PHY_ENABLE_ATOMIC_LINK;
+ /* Copy over all the old settings */
+ config.phy_type = abilities->phy_type;
+ config.phy_type_ext = abilities->phy_type_ext;
+ config.link_speed = abilities->link_speed;
+ config.eee_capability = abilities->eee_capability;
+ config.eeer = abilities->eeer_val;
+ config.low_power_ctrl = abilities->d3_lpan;
+ config.fec_config = abilities->fec_cfg_curr_mod_ext_info &
+ I40E_AQ_PHY_FEC_CONFIG_MASK;
+
+ return i40e_aq_set_phy_config(hw, &config, NULL);
+}
+
+/**
+ * i40e_set_fc
+ * @hw: pointer to the hw struct
+ * @aq_failures: buffer to return AdminQ failure information
+ * @atomic_restart: whether to enable atomic link restart
+ *
+ * Set the requested flow control mode using set_phy_config.
+ **/
+enum i40e_status_code i40e_set_fc(struct i40e_hw *hw, u8 *aq_failures,
+ bool atomic_restart)
+{
+ struct i40e_aq_get_phy_abilities_resp abilities;
+ enum i40e_status_code status;
+
+ *aq_failures = 0x0;
+
/* Get the current phy config */
status = i40e_aq_get_phy_capabilities(hw, false, false, &abilities,
NULL);
@@ -1685,31 +1721,10 @@ enum i40e_status_code i40e_set_fc(struct i40e_hw *hw, u8 *aq_failures,
return status;
}
- memset(&config, 0, sizeof(struct i40e_aq_set_phy_config));
- /* clear the old pause settings */
- config.abilities = abilities.abilities & ~(I40E_AQ_PHY_FLAG_PAUSE_TX) &
- ~(I40E_AQ_PHY_FLAG_PAUSE_RX);
- /* set the new abilities */
- config.abilities |= pause_mask;
- /* If the abilities have changed, then set the new config */
- if (config.abilities != abilities.abilities) {
- /* Auto restart link so settings take effect */
- if (atomic_restart)
- config.abilities |= I40E_AQ_PHY_ENABLE_ATOMIC_LINK;
- /* Copy over all the old settings */
- config.phy_type = abilities.phy_type;
- config.phy_type_ext = abilities.phy_type_ext;
- config.link_speed = abilities.link_speed;
- config.eee_capability = abilities.eee_capability;
- config.eeer = abilities.eeer_val;
- config.low_power_ctrl = abilities.d3_lpan;
- config.fec_config = abilities.fec_cfg_curr_mod_ext_info &
- I40E_AQ_PHY_FEC_CONFIG_MASK;
- status = i40e_aq_set_phy_config(hw, &config, NULL);
+ status = i40e_set_fc_status(hw, &abilities, atomic_restart);
+ if (status)
+ *aq_failures |= I40E_SET_FC_AQ_FAIL_SET;
- if (status)
- *aq_failures |= I40E_SET_FC_AQ_FAIL_SET;
- }
/* Update the link info */
status = i40e_update_link_info(hw);
if (status) {
@@ -2537,7 +2552,7 @@ i40e_status i40e_get_link_status(struct i40e_hw *hw, bool *link_up)
* i40e_updatelink_status - update status of the HW network link
* @hw: pointer to the hw struct
**/
-i40e_status i40e_update_link_info(struct i40e_hw *hw)
+noinline_for_stack i40e_status i40e_update_link_info(struct i40e_hw *hw)
{
struct i40e_aq_get_phy_abilities_resp abilities;
i40e_status status = 0;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_dcb.c b/drivers/net/ethernet/intel/i40e/i40e_dcb.c
index 292eeb3..200a1cb 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_dcb.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_dcb.c
@@ -877,7 +877,23 @@ i40e_status i40e_init_dcb(struct i40e_hw *hw, bool enable_mib_change)
return I40E_NOT_SUPPORTED;
/* Read LLDP NVM area */
- ret = i40e_read_lldp_cfg(hw, &lldp_cfg);
+ if (hw->flags & I40E_HW_FLAG_FW_LLDP_PERSISTENT) {
+ u8 offset = 0;
+
+ if (hw->mac.type == I40E_MAC_XL710)
+ offset = I40E_LLDP_CURRENT_STATUS_XL710_OFFSET;
+ else if (hw->mac.type == I40E_MAC_X722)
+ offset = I40E_LLDP_CURRENT_STATUS_X722_OFFSET;
+ else
+ return I40E_NOT_SUPPORTED;
+
+ ret = i40e_read_nvm_module_data(hw,
+ I40E_SR_EMP_SR_SETTINGS_PTR,
+ offset, 1,
+ &lldp_cfg.adminstatus);
+ } else {
+ ret = i40e_read_lldp_cfg(hw, &lldp_cfg);
+ }
if (ret)
return I40E_ERR_NOT_READY;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_dcb.h b/drivers/net/ethernet/intel/i40e/i40e_dcb.h
index ddb48ae..2a80c5d 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_dcb.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_dcb.h
@@ -30,6 +30,8 @@
#define I40E_CEE_SUBTYPE_APP_PRI 4
#define I40E_CEE_MAX_FEAT_TYPE 3
+#define I40E_LLDP_CURRENT_STATUS_XL710_OFFSET 0x2B
+#define I40E_LLDP_CURRENT_STATUS_X722_OFFSET 0x31
/* Defines for LLDP TLV header */
#define I40E_LLDP_TLV_LEN_SHIFT 0
#define I40E_LLDP_TLV_LEN_MASK (0x01FF << I40E_LLDP_TLV_LEN_SHIFT)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_debugfs.c b/drivers/net/ethernet/intel/i40e/i40e_debugfs.c
index 4123289..99ea543 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_debugfs.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_debugfs.c
@@ -1125,10 +1125,6 @@ static ssize_t i40e_dbg_command_write(struct file *filp,
dev_info(&pf->pdev->dev, "debugfs: forcing GlobR\n");
i40e_do_reset_safe(pf, BIT(__I40E_GLOBAL_RESET_REQUESTED));
- } else if (strncmp(cmd_buf, "empr", 4) == 0) {
- dev_info(&pf->pdev->dev, "debugfs: forcing EMPR\n");
- i40e_do_reset_safe(pf, BIT(__I40E_EMP_RESET_REQUESTED));
-
} else if (strncmp(cmd_buf, "read", 4) == 0) {
u32 address;
u32 value;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
index 01e4615b..41e1240 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_ethtool.c
@@ -5137,6 +5137,12 @@ static int i40e_get_module_info(struct net_device *netdev,
/* Module is not SFF-8472 compliant */
modinfo->type = ETH_MODULE_SFF_8079;
modinfo->eeprom_len = ETH_MODULE_SFF_8079_LEN;
+ } else if (!(sff8472_swap & I40E_MODULE_SFF_DDM_IMPLEMENTED)) {
+ /* Module is SFF-8472 compliant but doesn't implement
+ * Digital Diagnostic Monitoring (DDM).
+ */
+ modinfo->type = ETH_MODULE_SFF_8079;
+ modinfo->eeprom_len = ETH_MODULE_SFF_8079_LEN;
} else {
modinfo->type = ETH_MODULE_SFF_8472;
modinfo->eeprom_len = ETH_MODULE_SFF_8472_LEN;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_hmc.c b/drivers/net/ethernet/intel/i40e/i40e_hmc.c
index 19ce93d..163ee8c 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_hmc.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_hmc.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2013 - 2018 Intel Corporation. */
+#include "i40e.h"
#include "i40e_osdep.h"
#include "i40e_register.h"
#include "i40e_status.h"
diff --git a/drivers/net/ethernet/intel/i40e/i40e_lan_hmc.c b/drivers/net/ethernet/intel/i40e/i40e_lan_hmc.c
index 994011c..be24d42 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_lan_hmc.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_lan_hmc.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright(c) 2013 - 2018 Intel Corporation. */
+#include "i40e.h"
#include "i40e_osdep.h"
#include "i40e_register.h"
#include "i40e_type.h"
@@ -963,7 +964,7 @@ static i40e_status i40e_set_hmc_context(u8 *context_bytes,
/**
* i40e_hmc_get_object_va - retrieves an object's virtual address
- * @hmc_info: pointer to i40e_hmc_info struct
+ * @hw: the hardware struct, from which we obtain the i40e_hmc_info pointer
* @object_base: pointer to u64 to get the va
* @rsrc_type: the hmc resource type
* @obj_idx: hmc object index
@@ -972,16 +973,16 @@ static i40e_status i40e_set_hmc_context(u8 *context_bytes,
* base pointer. This function is used for LAN Queue contexts.
**/
static
-i40e_status i40e_hmc_get_object_va(struct i40e_hmc_info *hmc_info,
- u8 **object_base,
- enum i40e_hmc_lan_rsrc_type rsrc_type,
- u32 obj_idx)
+i40e_status i40e_hmc_get_object_va(struct i40e_hw *hw, u8 **object_base,
+ enum i40e_hmc_lan_rsrc_type rsrc_type,
+ u32 obj_idx)
{
+ struct i40e_hmc_info *hmc_info = &hw->hmc;
u32 obj_offset_in_sd, obj_offset_in_pd;
- i40e_status ret_code = 0;
struct i40e_hmc_sd_entry *sd_entry;
struct i40e_hmc_pd_entry *pd_entry;
u32 pd_idx, pd_lmt, rel_pd_idx;
+ i40e_status ret_code = 0;
u64 obj_offset_in_fpm;
u32 sd_idx, sd_lmt;
@@ -1047,7 +1048,7 @@ i40e_status i40e_clear_lan_tx_queue_context(struct i40e_hw *hw,
i40e_status err;
u8 *context_bytes;
- err = i40e_hmc_get_object_va(&hw->hmc, &context_bytes,
+ err = i40e_hmc_get_object_va(hw, &context_bytes,
I40E_HMC_LAN_TX, queue);
if (err < 0)
return err;
@@ -1068,7 +1069,7 @@ i40e_status i40e_set_lan_tx_queue_context(struct i40e_hw *hw,
i40e_status err;
u8 *context_bytes;
- err = i40e_hmc_get_object_va(&hw->hmc, &context_bytes,
+ err = i40e_hmc_get_object_va(hw, &context_bytes,
I40E_HMC_LAN_TX, queue);
if (err < 0)
return err;
@@ -1088,7 +1089,7 @@ i40e_status i40e_clear_lan_rx_queue_context(struct i40e_hw *hw,
i40e_status err;
u8 *context_bytes;
- err = i40e_hmc_get_object_va(&hw->hmc, &context_bytes,
+ err = i40e_hmc_get_object_va(hw, &context_bytes,
I40E_HMC_LAN_RX, queue);
if (err < 0)
return err;
@@ -1109,7 +1110,7 @@ i40e_status i40e_set_lan_rx_queue_context(struct i40e_hw *hw,
i40e_status err;
u8 *context_bytes;
- err = i40e_hmc_get_object_va(&hw->hmc, &context_bytes,
+ err = i40e_hmc_get_object_va(hw, &context_bytes,
I40E_HMC_LAN_RX, queue);
if (err < 0)
return err;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 6d456e5..e9f2f27 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -73,6 +73,7 @@ static const struct pci_device_id i40e_pci_tbl[] = {
{PCI_VDEVICE(INTEL, I40E_DEV_ID_QSFP_C), 0},
{PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_BASE_T), 0},
{PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_BASE_T4), 0},
+ {PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_BASE_T_BC), 0},
{PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_SFP), 0},
{PCI_VDEVICE(INTEL, I40E_DEV_ID_10G_B), 0},
{PCI_VDEVICE(INTEL, I40E_DEV_ID_KX_X722), 0},
@@ -534,6 +535,10 @@ void i40e_pf_reset_stats(struct i40e_pf *pf)
sizeof(pf->veb[i]->stats));
memset(&pf->veb[i]->stats_offsets, 0,
sizeof(pf->veb[i]->stats_offsets));
+ memset(&pf->veb[i]->tc_stats, 0,
+ sizeof(pf->veb[i]->tc_stats));
+ memset(&pf->veb[i]->tc_stats_offsets, 0,
+ sizeof(pf->veb[i]->tc_stats_offsets));
pf->veb[i]->stat_offsets_loaded = false;
}
}
@@ -3364,7 +3369,7 @@ static int i40e_vsi_configure_tx(struct i40e_vsi *vsi)
for (i = 0; (i < vsi->num_queue_pairs) && !err; i++)
err = i40e_configure_tx_ring(vsi->tx_rings[i]);
- if (!i40e_enabled_xdp_vsi(vsi))
+ if (err || !i40e_enabled_xdp_vsi(vsi))
return err;
for (i = 0; (i < vsi->num_queue_pairs) && !err; i++)
@@ -6416,50 +6421,6 @@ static int i40e_resume_port_tx(struct i40e_pf *pf)
}
/**
- * i40e_update_dcb_config
- * @hw: pointer to the HW struct
- * @enable_mib_change: enable MIB change event
- *
- * Update DCB configuration from the firmware
- **/
-static enum i40e_status_code
-i40e_update_dcb_config(struct i40e_hw *hw, bool enable_mib_change)
-{
- struct i40e_lldp_variables lldp_cfg;
- i40e_status ret;
-
- if (!hw->func_caps.dcb)
- return I40E_NOT_SUPPORTED;
-
- /* Read LLDP NVM area */
- ret = i40e_read_lldp_cfg(hw, &lldp_cfg);
- if (ret)
- return I40E_ERR_NOT_READY;
-
- /* Get DCBX status */
- ret = i40e_get_dcbx_status(hw, &hw->dcbx_status);
- if (ret)
- return ret;
-
- /* Check the DCBX Status */
- if (hw->dcbx_status == I40E_DCBX_STATUS_DONE ||
- hw->dcbx_status == I40E_DCBX_STATUS_IN_PROGRESS) {
- /* Get current DCBX configuration */
- ret = i40e_get_dcb_config(hw);
- if (ret)
- return ret;
- } else if (hw->dcbx_status == I40E_DCBX_STATUS_DISABLED) {
- return I40E_ERR_NOT_READY;
- }
-
- /* Configure the LLDP MIB change event */
- if (enable_mib_change)
- ret = i40e_aq_cfg_lldp_mib_change_event(hw, true, NULL);
-
- return ret;
-}
-
-/**
* i40e_init_pf_dcb - Initialize DCB configuration
* @pf: PF being configured
*
@@ -6481,7 +6442,7 @@ static int i40e_init_pf_dcb(struct i40e_pf *pf)
goto out;
}
- err = i40e_update_dcb_config(hw, true);
+ err = i40e_init_dcb(hw, true);
if (!err) {
/* Device/Function is not DCBX capable */
if ((!hw->func_caps.dcb) ||
@@ -12570,7 +12531,8 @@ static int i40e_xdp_setup(struct i40e_vsi *vsi,
if (need_reset && prog)
for (i = 0; i < vsi->num_queue_pairs; i++)
if (vsi->xdp_rings[i]->xsk_umem)
- (void)i40e_xsk_async_xmit(vsi->netdev, i);
+ (void)i40e_xsk_wakeup(vsi->netdev, i,
+ XDP_WAKEUP_RX);
return 0;
}
@@ -12892,7 +12854,7 @@ static const struct net_device_ops i40e_netdev_ops = {
.ndo_bridge_setlink = i40e_ndo_bridge_setlink,
.ndo_bpf = i40e_xdp,
.ndo_xdp_xmit = i40e_xdp_xmit,
- .ndo_xsk_async_xmit = i40e_xsk_async_xmit,
+ .ndo_xsk_wakeup = i40e_xsk_wakeup,
.ndo_dfwd_add_station = i40e_fwd_add,
.ndo_dfwd_del_station = i40e_fwd_del,
};
@@ -14578,9 +14540,20 @@ void i40e_set_fec_in_flags(u8 fec_cfg, u32 *flags)
**/
static bool i40e_check_recovery_mode(struct i40e_pf *pf)
{
- u32 val = rd32(&pf->hw, I40E_GL_FWSTS);
+ u32 val = rd32(&pf->hw, I40E_GL_FWSTS) & I40E_GL_FWSTS_FWS1B_MASK;
+ bool is_recovery_mode = false;
- if (val & I40E_GL_FWSTS_FWS1B_MASK) {
+ if (pf->hw.mac.type == I40E_MAC_XL710)
+ is_recovery_mode =
+ val == I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_CORER_MASK ||
+ val == I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_GLOBR_MASK ||
+ val == I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_TRANSITION_MASK ||
+ val == I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_NVM_MASK;
+ if (pf->hw.mac.type == I40E_MAC_X722)
+ is_recovery_mode =
+ val == I40E_X722_GL_FWSTS_FWS1B_REC_MOD_CORER_MASK ||
+ val == I40E_X722_GL_FWSTS_FWS1B_REC_MOD_GLOBR_MASK;
+ if (is_recovery_mode) {
dev_notice(&pf->pdev->dev, "Firmware recovery mode detected. Limiting functionality.\n");
dev_notice(&pf->pdev->dev, "Refer to the Intel(R) Ethernet Adapters and Devices User Guide for details on firmware recovery mode.\n");
set_bit(__I40E_RECOVERY_MODE, pf->state);
@@ -14594,6 +14567,51 @@ static bool i40e_check_recovery_mode(struct i40e_pf *pf)
}
/**
+ * i40e_pf_loop_reset - perform reset in a loop.
+ * @pf: board private structure
+ *
+ * This function is useful when a NIC is about to enter recovery mode.
+ * When a NIC's internal data structures are corrupted the NIC's
+ * firmware is going to enter recovery mode.
+ * Right after a POR it takes about 7 minutes for firmware to enter
+ * recovery mode. Until that time a NIC is in some kind of intermediate
+ * state. After that time period the NIC almost surely enters
+ * recovery mode. The only way for a driver to detect intermediate
+ * state is to issue a series of pf-resets and check a return value.
+ * If a PF reset returns success then the firmware could be in recovery
+ * mode so the caller of this code needs to check for recovery mode
+ * if this function returns success. There is a little chance that
+ * firmware will hang in intermediate state forever.
+ * Since waiting 7 minutes is quite a lot of time this function waits
+ * 10 seconds and then gives up by returning an error.
+ *
+ * Return 0 on success, negative on failure.
+ **/
+static i40e_status i40e_pf_loop_reset(struct i40e_pf *pf)
+{
+ const unsigned short MAX_CNT = 1000;
+ const unsigned short MSECS = 10;
+ struct i40e_hw *hw = &pf->hw;
+ i40e_status ret;
+ int cnt;
+
+ for (cnt = 0; cnt < MAX_CNT; ++cnt) {
+ ret = i40e_pf_reset(hw);
+ if (!ret)
+ break;
+ msleep(MSECS);
+ }
+
+ if (cnt == MAX_CNT) {
+ dev_info(&pf->pdev->dev, "PF reset failed: %d\n", ret);
+ return ret;
+ }
+
+ pf->pfr_count++;
+ return ret;
+}
+
+/**
* i40e_init_recovery_mode - initialize subsystems needed in recovery mode
* @pf: board private structure
* @hw: ptr to the hardware info
@@ -14821,14 +14839,22 @@ static int i40e_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* Reset here to make sure all is clean and to define PF 'n' */
i40e_clear_hw(hw);
- if (!i40e_check_recovery_mode(pf)) {
- err = i40e_pf_reset(hw);
- if (err) {
- dev_info(&pdev->dev, "Initial pf_reset failed: %d\n", err);
- goto err_pf_reset;
- }
- pf->pfr_count++;
+
+ err = i40e_set_mac_type(hw);
+ if (err) {
+ dev_warn(&pdev->dev, "unidentified MAC or BLANK NVM: %d\n",
+ err);
+ goto err_pf_reset;
}
+
+ err = i40e_pf_loop_reset(pf);
+ if (err) {
+ dev_info(&pdev->dev, "Initial pf_reset failed: %d\n", err);
+ goto err_pf_reset;
+ }
+
+ i40e_check_recovery_mode(pf);
+
hw->aq.num_arq_entries = I40E_AQ_LEN;
hw->aq.num_asq_entries = I40E_AQ_LEN;
hw->aq.arq_buf_size = I40E_MAX_AQ_BUF_SIZE;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_nvm.c b/drivers/net/ethernet/intel/i40e/i40e_nvm.c
index c508b75..e4d8d20b 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_nvm.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_nvm.c
@@ -322,6 +322,77 @@ i40e_status i40e_read_nvm_word(struct i40e_hw *hw, u16 offset,
}
/**
+ * i40e_read_nvm_module_data - Reads NVM Buffer to specified memory location
+ * @hw: pointer to the HW structure
+ * @module_ptr: Pointer to module in words with respect to NVM beginning
+ * @offset: offset in words from module start
+ * @words_data_size: Words to read from NVM
+ * @data_ptr: Pointer to memory location where resulting buffer will be stored
+ **/
+i40e_status i40e_read_nvm_module_data(struct i40e_hw *hw,
+ u8 module_ptr, u16 offset,
+ u16 words_data_size,
+ u16 *data_ptr)
+{
+ i40e_status status;
+ u16 ptr_value = 0;
+ u32 flat_offset;
+
+ if (module_ptr != 0) {
+ status = i40e_read_nvm_word(hw, module_ptr, &ptr_value);
+ if (status) {
+ i40e_debug(hw, I40E_DEBUG_ALL,
+ "Reading nvm word failed.Error code: %d.\n",
+ status);
+ return I40E_ERR_NVM;
+ }
+ }
+#define I40E_NVM_INVALID_PTR_VAL 0x7FFF
+#define I40E_NVM_INVALID_VAL 0xFFFF
+
+ /* Pointer not initialized */
+ if (ptr_value == I40E_NVM_INVALID_PTR_VAL ||
+ ptr_value == I40E_NVM_INVALID_VAL)
+ return I40E_ERR_BAD_PTR;
+
+ /* Check whether the module is in SR mapped area or outside */
+ if (ptr_value & I40E_PTR_TYPE) {
+ /* Pointer points outside of the Shared RAM mapped area */
+ ptr_value &= ~I40E_PTR_TYPE;
+
+ /* PtrValue in 4kB units, need to convert to words */
+ ptr_value /= 2;
+ flat_offset = ((u32)ptr_value * 0x1000) + (u32)offset;
+ status = i40e_acquire_nvm(hw, I40E_RESOURCE_READ);
+ if (!status) {
+ status = i40e_aq_read_nvm(hw, 0, 2 * flat_offset,
+ 2 * words_data_size,
+ data_ptr, true, NULL);
+ i40e_release_nvm(hw);
+ if (status) {
+ i40e_debug(hw, I40E_DEBUG_ALL,
+ "Reading nvm aq failed.Error code: %d.\n",
+ status);
+ return I40E_ERR_NVM;
+ }
+ } else {
+ return I40E_ERR_NVM;
+ }
+ } else {
+ /* Read from the Shadow RAM */
+ status = i40e_read_nvm_buffer(hw, ptr_value + offset,
+ &words_data_size, data_ptr);
+ if (status) {
+ i40e_debug(hw, I40E_DEBUG_ALL,
+ "Reading nvm buffer failed.Error code: %d.\n",
+ status);
+ }
+ }
+
+ return status;
+}
+
+/**
* i40e_read_nvm_buffer_srctl - Reads Shadow RAM buffer via SRCTL register
* @hw: pointer to the HW structure
* @offset: offset of the Shadow RAM word to read (0x000000 - 0x001FFF).
@@ -430,6 +501,36 @@ static i40e_status __i40e_read_nvm_buffer(struct i40e_hw *hw,
}
/**
+ * i40e_read_nvm_buffer - Reads Shadow RAM buffer and acquire lock if necessary
+ * @hw: pointer to the HW structure
+ * @offset: offset of the Shadow RAM word to read (0x000000 - 0x001FFF).
+ * @words: (in) number of words to read; (out) number of words actually read
+ * @data: words read from the Shadow RAM
+ *
+ * Reads 16 bit words (data buffer) from the SR using the i40e_read_nvm_srrd()
+ * method. The buffer read is preceded by the NVM ownership take
+ * and followed by the release.
+ **/
+i40e_status i40e_read_nvm_buffer(struct i40e_hw *hw, u16 offset,
+ u16 *words, u16 *data)
+{
+ i40e_status ret_code = 0;
+
+ if (hw->flags & I40E_HW_FLAG_AQ_SRCTL_ACCESS_ENABLE) {
+ ret_code = i40e_acquire_nvm(hw, I40E_RESOURCE_READ);
+ if (!ret_code) {
+ ret_code = i40e_read_nvm_buffer_aq(hw, offset, words,
+ data);
+ i40e_release_nvm(hw);
+ }
+ } else {
+ ret_code = i40e_read_nvm_buffer_srctl(hw, offset, words, data);
+ }
+
+ return ret_code;
+}
+
+/**
* i40e_write_nvm_aq - Writes Shadow RAM.
* @hw: pointer to the HW structure.
* @module_pointer: module pointer location in words from the NVM beginning
diff --git a/drivers/net/ethernet/intel/i40e/i40e_osdep.h b/drivers/net/ethernet/intel/i40e/i40e_osdep.h
index a07574b..c302ef2 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_osdep.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_osdep.h
@@ -18,7 +18,10 @@
* actual OS primitives
*/
-#define hw_dbg(hw, S, A...) do {} while (0)
+#define hw_dbg(hw, S, A...) \
+do { \
+ dev_dbg(&((struct i40e_pf *)hw->back)->pdev->dev, S, ##A); \
+} while (0)
#define wr32(a, reg, value) writel((value), ((a)->hw_addr + (reg)))
#define rd32(a, reg) readl((a)->hw_addr + (reg))
diff --git a/drivers/net/ethernet/intel/i40e/i40e_prototype.h b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
index eac88bc..5250441 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_prototype.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_prototype.h
@@ -315,6 +315,12 @@ i40e_status i40e_acquire_nvm(struct i40e_hw *hw,
void i40e_release_nvm(struct i40e_hw *hw);
i40e_status i40e_read_nvm_word(struct i40e_hw *hw, u16 offset,
u16 *data);
+i40e_status i40e_read_nvm_module_data(struct i40e_hw *hw,
+ u8 module_ptr, u16 offset,
+ u16 words_data_size,
+ u16 *data_ptr);
+i40e_status i40e_read_nvm_buffer(struct i40e_hw *hw, u16 offset,
+ u16 *words, u16 *data);
i40e_status i40e_update_nvm_checksum(struct i40e_hw *hw);
i40e_status i40e_validate_nvm_checksum(struct i40e_hw *hw,
u16 *checksum);
@@ -326,6 +332,8 @@ void i40e_nvmupd_check_wait_event(struct i40e_hw *hw, u16 opcode,
void i40e_nvmupd_clear_wait_state(struct i40e_hw *hw);
void i40e_set_pci_config_data(struct i40e_hw *hw, u16 link_status);
+i40e_status i40e_set_mac_type(struct i40e_hw *hw);
+
extern struct i40e_rx_ptype_decoded i40e_ptype_lookup[];
static inline struct i40e_rx_ptype_decoded decode_rx_desc_ptype(u8 ptype)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_register.h b/drivers/net/ethernet/intel/i40e/i40e_register.h
index 52e3680..d35d690 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_register.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_register.h
@@ -58,7 +58,7 @@
#define I40E_PF_ARQLEN_ARQCRIT_SHIFT 30
#define I40E_PF_ARQLEN_ARQCRIT_MASK I40E_MASK(0x1, I40E_PF_ARQLEN_ARQCRIT_SHIFT)
#define I40E_PF_ARQLEN_ARQENABLE_SHIFT 31
-#define I40E_PF_ARQLEN_ARQENABLE_MASK I40E_MASK(0x1, I40E_PF_ARQLEN_ARQENABLE_SHIFT)
+#define I40E_PF_ARQLEN_ARQENABLE_MASK I40E_MASK(0x1u, I40E_PF_ARQLEN_ARQENABLE_SHIFT)
#define I40E_PF_ARQT 0x00080480 /* Reset: EMPR */
#define I40E_PF_ARQT_ARQT_SHIFT 0
#define I40E_PF_ARQT_ARQT_MASK I40E_MASK(0x3FF, I40E_PF_ARQT_ARQT_SHIFT)
@@ -81,7 +81,7 @@
#define I40E_PF_ATQLEN_ATQCRIT_SHIFT 30
#define I40E_PF_ATQLEN_ATQCRIT_MASK I40E_MASK(0x1, I40E_PF_ATQLEN_ATQCRIT_SHIFT)
#define I40E_PF_ATQLEN_ATQENABLE_SHIFT 31
-#define I40E_PF_ATQLEN_ATQENABLE_MASK I40E_MASK(0x1, I40E_PF_ATQLEN_ATQENABLE_SHIFT)
+#define I40E_PF_ATQLEN_ATQENABLE_MASK I40E_MASK(0x1u, I40E_PF_ATQLEN_ATQENABLE_SHIFT)
#define I40E_PF_ATQT 0x00080400 /* Reset: EMPR */
#define I40E_PF_ATQT_ATQT_SHIFT 0
#define I40E_PF_ATQT_ATQT_MASK I40E_MASK(0x3FF, I40E_PF_ATQT_ATQT_SHIFT)
@@ -108,7 +108,7 @@
#define I40E_VF_ARQLEN_ARQCRIT_SHIFT 30
#define I40E_VF_ARQLEN_ARQCRIT_MASK I40E_MASK(0x1, I40E_VF_ARQLEN_ARQCRIT_SHIFT)
#define I40E_VF_ARQLEN_ARQENABLE_SHIFT 31
-#define I40E_VF_ARQLEN_ARQENABLE_MASK I40E_MASK(0x1, I40E_VF_ARQLEN_ARQENABLE_SHIFT)
+#define I40E_VF_ARQLEN_ARQENABLE_MASK I40E_MASK(0x1u, I40E_VF_ARQLEN_ARQENABLE_SHIFT)
#define I40E_VF_ARQT(_VF) (0x00082C00 + ((_VF) * 4)) /* _i=0...127 */ /* Reset: EMPR */
#define I40E_VF_ARQT_MAX_INDEX 127
#define I40E_VF_ARQT_ARQT_SHIFT 0
@@ -136,7 +136,7 @@
#define I40E_VF_ATQLEN_ATQCRIT_SHIFT 30
#define I40E_VF_ATQLEN_ATQCRIT_MASK I40E_MASK(0x1, I40E_VF_ATQLEN_ATQCRIT_SHIFT)
#define I40E_VF_ATQLEN_ATQENABLE_SHIFT 31
-#define I40E_VF_ATQLEN_ATQENABLE_MASK I40E_MASK(0x1, I40E_VF_ATQLEN_ATQENABLE_SHIFT)
+#define I40E_VF_ATQLEN_ATQENABLE_MASK I40E_MASK(0x1u, I40E_VF_ATQLEN_ATQENABLE_SHIFT)
#define I40E_VF_ATQT(_VF) (0x00082800 + ((_VF) * 4)) /* _i=0...127 */ /* Reset: EMPR */
#define I40E_VF_ATQT_MAX_INDEX 127
#define I40E_VF_ATQT_ATQT_SHIFT 0
@@ -259,7 +259,7 @@
#define I40E_PRTDCB_RETSTCC_UPINTC_MODE_SHIFT 30
#define I40E_PRTDCB_RETSTCC_UPINTC_MODE_MASK I40E_MASK(0x1, I40E_PRTDCB_RETSTCC_UPINTC_MODE_SHIFT)
#define I40E_PRTDCB_RETSTCC_ETSTC_SHIFT 31
-#define I40E_PRTDCB_RETSTCC_ETSTC_MASK I40E_MASK(0x1, I40E_PRTDCB_RETSTCC_ETSTC_SHIFT)
+#define I40E_PRTDCB_RETSTCC_ETSTC_MASK I40E_MASK(0x1u, I40E_PRTDCB_RETSTCC_ETSTC_SHIFT)
#define I40E_PRTDCB_RPPMC 0x001223A0 /* Reset: CORER */
#define I40E_PRTDCB_RPPMC_LANRPPM_SHIFT 0
#define I40E_PRTDCB_RPPMC_LANRPPM_MASK I40E_MASK(0xFF, I40E_PRTDCB_RPPMC_LANRPPM_SHIFT)
@@ -363,6 +363,12 @@
#define I40E_GL_FWSTS_FWRI_MASK I40E_MASK(0x1, I40E_GL_FWSTS_FWRI_SHIFT)
#define I40E_GL_FWSTS_FWS1B_SHIFT 16
#define I40E_GL_FWSTS_FWS1B_MASK I40E_MASK(0xFF, I40E_GL_FWSTS_FWS1B_SHIFT)
+#define I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_CORER_MASK I40E_MASK(0x30, I40E_GL_FWSTS_FWS1B_SHIFT)
+#define I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_GLOBR_MASK I40E_MASK(0x31, I40E_GL_FWSTS_FWS1B_SHIFT)
+#define I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_TRANSITION_MASK I40E_MASK(0x32, I40E_GL_FWSTS_FWS1B_SHIFT)
+#define I40E_XL710_GL_FWSTS_FWS1B_REC_MOD_NVM_MASK I40E_MASK(0x33, I40E_GL_FWSTS_FWS1B_SHIFT)
+#define I40E_X722_GL_FWSTS_FWS1B_REC_MOD_CORER_MASK I40E_MASK(0xB, I40E_GL_FWSTS_FWS1B_SHIFT)
+#define I40E_X722_GL_FWSTS_FWS1B_REC_MOD_GLOBR_MASK I40E_MASK(0xC, I40E_GL_FWSTS_FWS1B_SHIFT)
#define I40E_GLGEN_CLKSTAT 0x000B8184 /* Reset: POR */
#define I40E_GLGEN_CLKSTAT_CLKMODE_SHIFT 0
#define I40E_GLGEN_CLKSTAT_CLKMODE_MASK I40E_MASK(0x1, I40E_GLGEN_CLKSTAT_CLKMODE_SHIFT)
@@ -503,7 +509,7 @@
#define I40E_GLGEN_MSCA_MDICMD_SHIFT 30
#define I40E_GLGEN_MSCA_MDICMD_MASK I40E_MASK(0x1, I40E_GLGEN_MSCA_MDICMD_SHIFT)
#define I40E_GLGEN_MSCA_MDIINPROGEN_SHIFT 31
-#define I40E_GLGEN_MSCA_MDIINPROGEN_MASK I40E_MASK(0x1, I40E_GLGEN_MSCA_MDIINPROGEN_SHIFT)
+#define I40E_GLGEN_MSCA_MDIINPROGEN_MASK I40E_MASK(0x1u, I40E_GLGEN_MSCA_MDIINPROGEN_SHIFT)
#define I40E_GLGEN_MSRWD(_i) (0x0008819C + ((_i) * 4)) /* _i=0...3 */ /* Reset: POR */
#define I40E_GLGEN_MSRWD_MAX_INDEX 3
#define I40E_GLGEN_MSRWD_MDIWRDATA_SHIFT 0
@@ -1242,14 +1248,14 @@
#define I40E_GLLAN_TXPRE_QDIS_SET_QDIS_SHIFT 30
#define I40E_GLLAN_TXPRE_QDIS_SET_QDIS_MASK I40E_MASK(0x1, I40E_GLLAN_TXPRE_QDIS_SET_QDIS_SHIFT)
#define I40E_GLLAN_TXPRE_QDIS_CLEAR_QDIS_SHIFT 31
-#define I40E_GLLAN_TXPRE_QDIS_CLEAR_QDIS_MASK I40E_MASK(0x1, I40E_GLLAN_TXPRE_QDIS_CLEAR_QDIS_SHIFT)
+#define I40E_GLLAN_TXPRE_QDIS_CLEAR_QDIS_MASK I40E_MASK(0x1u, I40E_GLLAN_TXPRE_QDIS_CLEAR_QDIS_SHIFT)
#define I40E_PFLAN_QALLOC 0x001C0400 /* Reset: CORER */
#define I40E_PFLAN_QALLOC_FIRSTQ_SHIFT 0
#define I40E_PFLAN_QALLOC_FIRSTQ_MASK I40E_MASK(0x7FF, I40E_PFLAN_QALLOC_FIRSTQ_SHIFT)
#define I40E_PFLAN_QALLOC_LASTQ_SHIFT 16
#define I40E_PFLAN_QALLOC_LASTQ_MASK I40E_MASK(0x7FF, I40E_PFLAN_QALLOC_LASTQ_SHIFT)
#define I40E_PFLAN_QALLOC_VALID_SHIFT 31
-#define I40E_PFLAN_QALLOC_VALID_MASK I40E_MASK(0x1, I40E_PFLAN_QALLOC_VALID_SHIFT)
+#define I40E_PFLAN_QALLOC_VALID_MASK I40E_MASK(0x1u, I40E_PFLAN_QALLOC_VALID_SHIFT)
#define I40E_QRX_ENA(_Q) (0x00120000 + ((_Q) * 4)) /* _i=0...1535 */ /* Reset: PFR */
#define I40E_QRX_ENA_MAX_INDEX 1535
#define I40E_QRX_ENA_QENA_REQ_SHIFT 0
@@ -1658,7 +1664,7 @@
#define I40E_GLNVM_SRCTL_START_SHIFT 30
#define I40E_GLNVM_SRCTL_START_MASK I40E_MASK(0x1, I40E_GLNVM_SRCTL_START_SHIFT)
#define I40E_GLNVM_SRCTL_DONE_SHIFT 31
-#define I40E_GLNVM_SRCTL_DONE_MASK I40E_MASK(0x1, I40E_GLNVM_SRCTL_DONE_SHIFT)
+#define I40E_GLNVM_SRCTL_DONE_MASK I40E_MASK(0x1u, I40E_GLNVM_SRCTL_DONE_SHIFT)
#define I40E_GLNVM_SRDATA 0x000B6114 /* Reset: POR */
#define I40E_GLNVM_SRDATA_WRDATA_SHIFT 0
#define I40E_GLNVM_SRDATA_WRDATA_MASK I40E_MASK(0xFFFF, I40E_GLNVM_SRDATA_WRDATA_SHIFT)
@@ -3025,7 +3031,7 @@
#define I40E_PF_VT_PFALLOC_LASTVF_SHIFT 8
#define I40E_PF_VT_PFALLOC_LASTVF_MASK I40E_MASK(0xFF, I40E_PF_VT_PFALLOC_LASTVF_SHIFT)
#define I40E_PF_VT_PFALLOC_VALID_SHIFT 31
-#define I40E_PF_VT_PFALLOC_VALID_MASK I40E_MASK(0x1, I40E_PF_VT_PFALLOC_VALID_SHIFT)
+#define I40E_PF_VT_PFALLOC_VALID_MASK I40E_MASK(0x1u, I40E_PF_VT_PFALLOC_VALID_SHIFT)
#define I40E_VP_MDET_RX(_VF) (0x0012A000 + ((_VF) * 4)) /* _i=0...127 */ /* Reset: CORER */
#define I40E_VP_MDET_RX_MAX_INDEX 127
#define I40E_VP_MDET_RX_VALID_SHIFT 0
@@ -3161,7 +3167,7 @@
#define I40E_VF_ARQLEN1_ARQCRIT_SHIFT 30
#define I40E_VF_ARQLEN1_ARQCRIT_MASK I40E_MASK(0x1, I40E_VF_ARQLEN1_ARQCRIT_SHIFT)
#define I40E_VF_ARQLEN1_ARQENABLE_SHIFT 31
-#define I40E_VF_ARQLEN1_ARQENABLE_MASK I40E_MASK(0x1, I40E_VF_ARQLEN1_ARQENABLE_SHIFT)
+#define I40E_VF_ARQLEN1_ARQENABLE_MASK I40E_MASK(0x1u, I40E_VF_ARQLEN1_ARQENABLE_SHIFT)
#define I40E_VF_ARQT1 0x00007000 /* Reset: EMPR */
#define I40E_VF_ARQT1_ARQT_SHIFT 0
#define I40E_VF_ARQT1_ARQT_MASK I40E_MASK(0x3FF, I40E_VF_ARQT1_ARQT_SHIFT)
@@ -3184,7 +3190,7 @@
#define I40E_VF_ATQLEN1_ATQCRIT_SHIFT 30
#define I40E_VF_ATQLEN1_ATQCRIT_MASK I40E_MASK(0x1, I40E_VF_ATQLEN1_ATQCRIT_SHIFT)
#define I40E_VF_ATQLEN1_ATQENABLE_SHIFT 31
-#define I40E_VF_ATQLEN1_ATQENABLE_MASK I40E_MASK(0x1, I40E_VF_ATQLEN1_ATQENABLE_SHIFT)
+#define I40E_VF_ATQLEN1_ATQENABLE_MASK I40E_MASK(0x1u, I40E_VF_ATQLEN1_ATQENABLE_SHIFT)
#define I40E_VF_ATQT1 0x00008400 /* Reset: EMPR */
#define I40E_VF_ATQT1_ATQT_SHIFT 0
#define I40E_VF_ATQT1_ATQT_MASK I40E_MASK(0x3FF, I40E_VF_ATQT1_ATQT_SHIFT)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_type.h b/drivers/net/ethernet/intel/i40e/i40e_type.h
index 8f43aa4..b43ec94 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_type.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_type.h
@@ -443,6 +443,7 @@ struct i40e_nvm_access {
#define I40E_MODULE_SFF_8472_COMP 0x5E
#define I40E_MODULE_SFF_8472_SWAP 0x5C
#define I40E_MODULE_SFF_ADDR_MODE 0x04
+#define I40E_MODULE_SFF_DDM_IMPLEMENTED 0x40
#define I40E_MODULE_TYPE_QSFP_PLUS 0x0D
#define I40E_MODULE_TYPE_QSFP28 0x11
#define I40E_MODULE_QSFP_MAX_LEN 640
@@ -623,6 +624,7 @@ struct i40e_hw {
#define I40E_HW_FLAG_NVM_READ_REQUIRES_LOCK BIT_ULL(3)
#define I40E_HW_FLAG_FW_LLDP_STOPPABLE BIT_ULL(4)
#define I40E_HW_FLAG_FW_LLDP_PERSISTENT BIT_ULL(5)
+#define I40E_HW_FLAG_DROP_MODE BIT_ULL(7)
u64 flags;
/* Used in set switch config AQ command */
@@ -1316,6 +1318,7 @@ struct i40e_hw_port_stats {
#define I40E_SR_VPD_PTR 0x2F
#define I40E_SR_PCIE_ALT_AUTO_LOAD_PTR 0x3E
#define I40E_SR_SW_CHECKSUM_WORD 0x3F
+#define I40E_SR_EMP_SR_SETTINGS_PTR 0x48
/* Auxiliary field, mask and shift definition for Shadow RAM and NVM Flash */
#define I40E_SR_VPD_MODULE_MAX_SIZE 1024
diff --git a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
index 4601f9e..f8aa4de 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
@@ -3967,10 +3967,15 @@ int i40e_ndo_set_vf_mac(struct net_device *netdev, int vf_id, u8 *mac)
/* When the VF is resetting wait until it is done.
* It can take up to 200 milliseconds,
* but wait for up to 300 milliseconds to be safe.
+ * If the VF is indeed in reset, the vsi pointer has
+ * to show on the newly loaded vsi under pf->vsi[id].
*/
for (i = 0; i < 15; i++) {
- if (test_bit(I40E_VF_STATE_INIT, &vf->vf_states))
+ if (test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
+ if (i > 0)
+ vsi = pf->vsi[vf->lan_vsi_idx];
break;
+ }
msleep(20);
}
if (!test_bit(I40E_VF_STATE_INIT, &vf->vf_states)) {
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index 32bad01..0373bc6 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -116,7 +116,7 @@ static int i40e_xsk_umem_enable(struct i40e_vsi *vsi, struct xdp_umem *umem,
return err;
/* Kick start the NAPI context so that receiving will start */
- err = i40e_xsk_async_xmit(vsi->netdev, qid);
+ err = i40e_xsk_wakeup(vsi->netdev, qid, XDP_WAKEUP_RX);
if (err)
return err;
}
@@ -190,7 +190,9 @@ int i40e_xsk_umem_setup(struct i40e_vsi *vsi, struct xdp_umem *umem,
**/
static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
{
+ struct xdp_umem *umem = rx_ring->xsk_umem;
int err, result = I40E_XDP_PASS;
+ u64 offset = umem->headroom;
struct i40e_ring *xdp_ring;
struct bpf_prog *xdp_prog;
u32 act;
@@ -201,7 +203,10 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
*/
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
act = bpf_prog_run_xdp(xdp_prog, xdp);
- xdp->handle += xdp->data - xdp->data_hard_start;
+ offset += xdp->data - xdp->data_hard_start;
+
+ xdp->handle = xsk_umem_adjust_offset(umem, xdp->handle, offset);
+
switch (act) {
case XDP_PASS:
break;
@@ -262,7 +267,7 @@ static bool i40e_alloc_buffer_zc(struct i40e_ring *rx_ring,
bi->addr = xdp_umem_get_data(umem, handle);
bi->addr += hr;
- bi->handle = handle + umem->headroom;
+ bi->handle = xsk_umem_adjust_offset(umem, handle, umem->headroom);
xsk_umem_discard_addr(umem);
return true;
@@ -299,7 +304,7 @@ static bool i40e_alloc_buffer_slow_zc(struct i40e_ring *rx_ring,
bi->addr = xdp_umem_get_data(umem, handle);
bi->addr += hr;
- bi->handle = handle + umem->headroom;
+ bi->handle = xsk_umem_adjust_offset(umem, handle, umem->headroom);
xsk_umem_discard_addr_rq(umem);
return true;
@@ -420,8 +425,6 @@ static void i40e_reuse_rx_buffer_zc(struct i40e_ring *rx_ring,
struct i40e_rx_buffer *old_bi)
{
struct i40e_rx_buffer *new_bi = &rx_ring->rx_bi[rx_ring->next_to_alloc];
- unsigned long mask = (unsigned long)rx_ring->xsk_umem->chunk_mask;
- u64 hr = rx_ring->xsk_umem->headroom + XDP_PACKET_HEADROOM;
u16 nta = rx_ring->next_to_alloc;
/* update, and store next to alloc */
@@ -429,14 +432,9 @@ static void i40e_reuse_rx_buffer_zc(struct i40e_ring *rx_ring,
rx_ring->next_to_alloc = (nta < rx_ring->count) ? nta : 0;
/* transfer page from old buffer to new buffer */
- new_bi->dma = old_bi->dma & mask;
- new_bi->dma += hr;
-
- new_bi->addr = (void *)((unsigned long)old_bi->addr & mask);
- new_bi->addr += hr;
-
- new_bi->handle = old_bi->handle & mask;
- new_bi->handle += rx_ring->xsk_umem->headroom;
+ new_bi->dma = old_bi->dma;
+ new_bi->addr = old_bi->addr;
+ new_bi->handle = old_bi->handle;
old_bi->addr = NULL;
}
@@ -471,7 +469,8 @@ void i40e_zca_free(struct zero_copy_allocator *alloc, unsigned long handle)
bi->addr = xdp_umem_get_data(rx_ring->xsk_umem, handle);
bi->addr += hr;
- bi->handle = (u64)handle + rx_ring->xsk_umem->headroom;
+ bi->handle = xsk_umem_adjust_offset(rx_ring->xsk_umem, (u64)handle,
+ rx_ring->xsk_umem->headroom);
}
/**
@@ -626,6 +625,15 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget)
i40e_finalize_xdp_rx(rx_ring, xdp_xmit);
i40e_update_rx_stats(rx_ring, total_rx_bytes, total_rx_packets);
+
+ if (xsk_umem_uses_need_wakeup(rx_ring->xsk_umem)) {
+ if (failure || rx_ring->next_to_clean == rx_ring->next_to_use)
+ xsk_set_rx_need_wakeup(rx_ring->xsk_umem);
+ else
+ xsk_clear_rx_need_wakeup(rx_ring->xsk_umem);
+
+ return (int)total_rx_packets;
+ }
return failure ? budget : (int)total_rx_packets;
}
@@ -681,6 +689,8 @@ static bool i40e_xmit_zc(struct i40e_ring *xdp_ring, unsigned int budget)
i40e_xdp_ring_update_tail(xdp_ring);
xsk_umem_consume_tx_done(xdp_ring->xsk_umem);
+ if (xsk_umem_uses_need_wakeup(xdp_ring->xsk_umem))
+ xsk_clear_tx_need_wakeup(xdp_ring->xsk_umem);
}
return !!budget && work_done;
@@ -759,19 +769,27 @@ bool i40e_clean_xdp_tx_irq(struct i40e_vsi *vsi,
i40e_update_tx_stats(tx_ring, completed_frames, total_bytes);
out_xmit:
+ if (xsk_umem_uses_need_wakeup(tx_ring->xsk_umem)) {
+ if (tx_ring->next_to_clean == tx_ring->next_to_use)
+ xsk_set_tx_need_wakeup(tx_ring->xsk_umem);
+ else
+ xsk_clear_tx_need_wakeup(tx_ring->xsk_umem);
+ }
+
xmit_done = i40e_xmit_zc(tx_ring, budget);
return work_done && xmit_done;
}
/**
- * i40e_xsk_async_xmit - Implements the ndo_xsk_async_xmit
+ * i40e_xsk_wakeup - Implements the ndo_xsk_wakeup
* @dev: the netdevice
* @queue_id: queue id to wake up
+ * @flags: ignored in our case since we have Rx and Tx in the same NAPI.
*
* Returns <0 for errors, 0 otherwise.
**/
-int i40e_xsk_async_xmit(struct net_device *dev, u32 queue_id)
+int i40e_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags)
{
struct i40e_netdev_priv *np = netdev_priv(dev);
struct i40e_vsi *vsi = np->vsi;
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.h b/drivers/net/ethernet/intel/i40e/i40e_xsk.h
index 8cc0a2e..9ed59c1 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.h
@@ -18,6 +18,6 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget);
bool i40e_clean_xdp_tx_irq(struct i40e_vsi *vsi,
struct i40e_ring *tx_ring, int napi_budget);
-int i40e_xsk_async_xmit(struct net_device *dev, u32 queue_id);
+int i40e_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags);
#endif /* _I40E_XSK_H_ */
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 9fc635d..29de3ae 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -253,7 +253,6 @@ struct iavf_adapter {
#define IAVF_FLAG_RESET_PENDING BIT(4)
#define IAVF_FLAG_RESET_NEEDED BIT(5)
#define IAVF_FLAG_WB_ON_ITR_CAPABLE BIT(6)
-#define IAVF_FLAG_ADDR_SET_BY_PF BIT(8)
#define IAVF_FLAG_SERVICE_CLIENT_REQUESTED BIT(9)
#define IAVF_FLAG_CLIENT_NEEDS_OPEN BIT(10)
#define IAVF_FLAG_CLIENT_NEEDS_CLOSE BIT(11)
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 9d2b509..07f5541 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -143,28 +143,6 @@ enum iavf_status iavf_free_virt_mem_d(struct iavf_hw *hw,
}
/**
- * iavf_debug_d - OS dependent version of debug printing
- * @hw: pointer to the HW structure
- * @mask: debug level mask
- * @fmt_str: printf-type format description
- **/
-void iavf_debug_d(void *hw, u32 mask, char *fmt_str, ...)
-{
- char buf[512];
- va_list argptr;
-
- if (!(mask & ((struct iavf_hw *)hw)->debug_mask))
- return;
-
- va_start(argptr, fmt_str);
- vsnprintf(buf, sizeof(buf), fmt_str, argptr);
- va_end(argptr);
-
- /* the debug string is already formatted with a newline */
- pr_info("%s", buf);
-}
-
-/**
* iavf_schedule_reset - Set the flags and schedule a reset event
* @adapter: board private structure
**/
@@ -812,9 +790,6 @@ static int iavf_set_mac(struct net_device *netdev, void *p)
if (ether_addr_equal(netdev->dev_addr, addr->sa_data))
return 0;
- if (adapter->flags & IAVF_FLAG_ADDR_SET_BY_PF)
- return -EPERM;
-
spin_lock_bh(&adapter->mac_vlan_list_lock);
f = iavf_find_filter(adapter, hw->mac.addr);
@@ -1833,7 +1808,6 @@ static int iavf_init_get_resources(struct iavf_adapter *adapter)
eth_hw_addr_random(netdev);
ether_addr_copy(adapter->hw.mac.addr, netdev->dev_addr);
} else {
- adapter->flags |= IAVF_FLAG_ADDR_SET_BY_PF;
ether_addr_copy(netdev->dev_addr, adapter->hw.mac.addr);
ether_addr_copy(netdev->perm_addr, adapter->hw.mac.addr);
}
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 9f9c30d..b36e1cf0 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -47,23 +47,8 @@ extern const char ice_drv_ver[];
#define ICE_MIN_NUM_DESC 64
#define ICE_MAX_NUM_DESC 8160
#define ICE_DFLT_MIN_RX_DESC 512
-/* if the default number of Rx descriptors between ICE_MAX_NUM_DESC and the
- * number of descriptors to fill up an entire page is greater than or equal to
- * ICE_DFLT_MIN_RX_DESC set it based on page size, otherwise set it to
- * ICE_DFLT_MIN_RX_DESC
- */
-#define ICE_DFLT_NUM_RX_DESC \
- min_t(u16, ICE_MAX_NUM_DESC, \
- max_t(u16, ALIGN(PAGE_SIZE / sizeof(union ice_32byte_rx_desc), \
- ICE_REQ_DESC_MULTIPLE), \
- ICE_DFLT_MIN_RX_DESC))
-/* set default number of Tx descriptors to the minimum between ICE_MAX_NUM_DESC
- * and the number of descriptors to fill up an entire page
- */
-#define ICE_DFLT_NUM_TX_DESC min_t(u16, ICE_MAX_NUM_DESC, \
- ALIGN(PAGE_SIZE / \
- sizeof(struct ice_tx_desc), \
- ICE_REQ_DESC_MULTIPLE))
+#define ICE_DFLT_NUM_TX_DESC 256
+#define ICE_DFLT_NUM_RX_DESC 2048
#define ICE_DFLT_TRAFFIC_CLASS BIT(0)
#define ICE_INT_NAME_STR_LEN (IFNAMSIZ + 16)
@@ -73,8 +58,6 @@ extern const char ice_drv_ver[];
#define ICE_MBXRQ_LEN 512
#define ICE_MIN_MSIX 2
#define ICE_NO_VSI 0xffff
-#define ICE_MAX_TXQS 2048
-#define ICE_MAX_RXQS 2048
#define ICE_VSI_MAP_CONTIG 0
#define ICE_VSI_MAP_SCATTER 1
#define ICE_MAX_SCATTER_TXQS 16
@@ -249,9 +232,6 @@ struct ice_vsi {
u16 vsi_num; /* HW (absolute) index of this VSI */
u16 idx; /* software index in pf->vsi[] */
- /* Interrupt thresholds */
- u16 work_lmt;
-
s16 vf_id; /* VF ID for SR-IOV VSIs */
u16 ethtype; /* Ethernet protocol for pause frame */
@@ -284,8 +264,8 @@ struct ice_vsi {
/* queue information */
u8 tx_mapping_mode; /* ICE_MAP_MODE_[CONTIG|SCATTER] */
u8 rx_mapping_mode; /* ICE_MAP_MODE_[CONTIG|SCATTER] */
- u16 txq_map[ICE_MAX_TXQS]; /* index in pf->avail_txqs */
- u16 rxq_map[ICE_MAX_RXQS]; /* index in pf->avail_rxqs */
+ u16 *txq_map; /* index in pf->avail_txqs */
+ u16 *rxq_map; /* index in pf->avail_rxqs */
u16 alloc_txq; /* Allocated Tx queues */
u16 num_txq; /* Used Tx queues */
u16 alloc_rxq; /* Allocated Rx queues */
@@ -329,7 +309,7 @@ enum ice_pf_flags {
ICE_FLAG_DCB_ENA,
ICE_FLAG_LINK_DOWN_ON_CLOSE_ENA,
ICE_FLAG_NO_MEDIA,
- ICE_FLAG_ENABLE_FW_LLDP,
+ ICE_FLAG_FW_LLDP_AGENT,
ICE_FLAG_ETHTOOL_CTXT, /* set when ethtool holds RTNL lock */
ICE_PF_FLAGS_NBITS /* must be last */
};
@@ -355,9 +335,9 @@ struct ice_pf {
u16 num_vf_qps; /* num queue pairs per VF */
u16 num_vf_msix; /* num vectors per VF */
DECLARE_BITMAP(state, __ICE_STATE_NBITS);
- DECLARE_BITMAP(avail_txqs, ICE_MAX_TXQS);
- DECLARE_BITMAP(avail_rxqs, ICE_MAX_RXQS);
DECLARE_BITMAP(flags, ICE_PF_FLAGS_NBITS);
+ unsigned long *avail_txqs; /* bitmap to track PF Tx queue usage */
+ unsigned long *avail_rxqs; /* bitmap to track PF Rx queue usage */
unsigned long serv_tmr_period;
unsigned long serv_tmr_prev;
struct timer_list serv_tmr;
@@ -368,11 +348,11 @@ struct ice_pf {
u32 hw_csum_rx_error;
u32 oicr_idx; /* Other interrupt cause MSIX vector index */
u32 num_avail_sw_msix; /* remaining MSIX SW vectors left unclaimed */
+ u16 max_pf_txqs; /* Total Tx queues PF wide */
+ u16 max_pf_rxqs; /* Total Rx queues PF wide */
u32 num_lan_msix; /* Total MSIX vectors for base driver */
u16 num_lan_tx; /* num LAN Tx queues setup */
u16 num_lan_rx; /* num LAN Rx queues setup */
- u16 q_left_tx; /* remaining num Tx queues left unclaimed */
- u16 q_left_rx; /* remaining num Rx queues left unclaimed */
u16 next_vsi; /* Next free slot in pf->vsi[] - 0-based! */
u16 num_alloc_vsi;
u16 corer_count; /* Core reset count */
@@ -425,21 +405,15 @@ ice_irq_dynamic_ena(struct ice_hw *hw, struct ice_vsi *vsi,
}
/**
- * ice_find_vsi_by_type - Find and return VSI of a given type
- * @pf: PF to search for VSI
- * @type: Value indicating type of VSI we are looking for
+ * ice_get_main_vsi - Get the PF VSI
+ * @pf: PF instance
+ *
+ * returns pf->vsi[0], which by definition is the PF VSI
*/
-static inline struct ice_vsi *
-ice_find_vsi_by_type(struct ice_pf *pf, enum ice_vsi_type type)
+static inline struct ice_vsi *ice_get_main_vsi(struct ice_pf *pf)
{
- int i;
-
- for (i = 0; i < pf->num_alloc_vsi; i++) {
- struct ice_vsi *vsi = pf->vsi[i];
-
- if (vsi && vsi->type == type)
- return vsi;
- }
+ if (pf->vsi)
+ return pf->vsi[0];
return NULL;
}
@@ -447,6 +421,10 @@ ice_find_vsi_by_type(struct ice_pf *pf, enum ice_vsi_type type)
int ice_vsi_setup_tx_rings(struct ice_vsi *vsi);
int ice_vsi_setup_rx_rings(struct ice_vsi *vsi);
void ice_set_ethtool_ops(struct net_device *netdev);
+u16 ice_get_avail_txq_count(struct ice_pf *pf);
+u16 ice_get_avail_rxq_count(struct ice_pf *pf);
+void ice_update_vsi_stats(struct ice_vsi *vsi);
+void ice_update_pf_stats(struct ice_pf *pf);
int ice_up(struct ice_vsi *vsi);
int ice_down(struct ice_vsi *vsi);
int ice_vsi_cfg(struct ice_vsi *vsi);
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 765e3c2..4da0cde 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -35,9 +35,9 @@ struct ice_aqc_get_ver {
/* Queue Shutdown (direct 0x0003) */
struct ice_aqc_q_shutdown {
- __le32 driver_unloading;
+ u8 driver_unloading;
#define ICE_AQC_DRIVER_UNLOADING BIT(0)
- u8 reserved[12];
+ u8 reserved[15];
};
/* Request resource ownership (direct 0x0008)
@@ -91,6 +91,7 @@ struct ice_aqc_list_caps_elem {
#define ICE_AQC_CAPS_SRIOV 0x0012
#define ICE_AQC_CAPS_VF 0x0013
#define ICE_AQC_CAPS_VSI 0x0017
+#define ICE_AQC_CAPS_DCB 0x0018
#define ICE_AQC_CAPS_RSS 0x0040
#define ICE_AQC_CAPS_RXQS 0x0041
#define ICE_AQC_CAPS_TXQS 0x0042
@@ -1610,6 +1611,7 @@ enum ice_aq_err {
ICE_AQ_RC_EBUSY = 12, /* Device or resource busy */
ICE_AQ_RC_EEXIST = 13, /* Object already exists */
ICE_AQ_RC_ENOSPC = 16, /* No space left or allocation failure */
+ ICE_AQ_RC_ENOSYS = 17, /* Function not implemented */
};
/* Admin Queue command opcodes */
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 5f9dc76..8b2c466 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -263,21 +263,23 @@ enum ice_status
ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
struct ice_link_status *link, struct ice_sq_cd *cd)
{
- struct ice_link_status *hw_link_info_old, *hw_link_info;
struct ice_aqc_get_link_status_data link_data = { 0 };
struct ice_aqc_get_link_status *resp;
+ struct ice_link_status *li_old, *li;
enum ice_media_type *hw_media_type;
struct ice_fc_info *hw_fc_info;
bool tx_pause, rx_pause;
struct ice_aq_desc desc;
enum ice_status status;
+ struct ice_hw *hw;
u16 cmd_flags;
if (!pi)
return ICE_ERR_PARAM;
- hw_link_info_old = &pi->phy.link_info_old;
+ hw = pi->hw;
+ li_old = &pi->phy.link_info_old;
hw_media_type = &pi->phy.media_type;
- hw_link_info = &pi->phy.link_info;
+ li = &pi->phy.link_info;
hw_fc_info = &pi->fc;
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_link_status);
@@ -286,27 +288,27 @@ ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
resp->cmd_flags = cpu_to_le16(cmd_flags);
resp->lport_num = pi->lport;
- status = ice_aq_send_cmd(pi->hw, &desc, &link_data, sizeof(link_data),
- cd);
+ status = ice_aq_send_cmd(hw, &desc, &link_data, sizeof(link_data), cd);
if (status)
return status;
/* save off old link status information */
- *hw_link_info_old = *hw_link_info;
+ *li_old = *li;
/* update current link status information */
- hw_link_info->link_speed = le16_to_cpu(link_data.link_speed);
- hw_link_info->phy_type_low = le64_to_cpu(link_data.phy_type_low);
- hw_link_info->phy_type_high = le64_to_cpu(link_data.phy_type_high);
+ li->link_speed = le16_to_cpu(link_data.link_speed);
+ li->phy_type_low = le64_to_cpu(link_data.phy_type_low);
+ li->phy_type_high = le64_to_cpu(link_data.phy_type_high);
*hw_media_type = ice_get_media_type(pi);
- hw_link_info->link_info = link_data.link_info;
- hw_link_info->an_info = link_data.an_info;
- hw_link_info->ext_info = link_data.ext_info;
- hw_link_info->max_frame_size = le16_to_cpu(link_data.max_frame_size);
- hw_link_info->fec_info = link_data.cfg & ICE_AQ_FEC_MASK;
- hw_link_info->topo_media_conflict = link_data.topo_media_conflict;
- hw_link_info->pacing = link_data.cfg & ICE_AQ_CFG_PACING_M;
+ li->link_info = link_data.link_info;
+ li->an_info = link_data.an_info;
+ li->ext_info = link_data.ext_info;
+ li->max_frame_size = le16_to_cpu(link_data.max_frame_size);
+ li->fec_info = link_data.cfg & ICE_AQ_FEC_MASK;
+ li->topo_media_conflict = link_data.topo_media_conflict;
+ li->pacing = link_data.cfg & (ICE_AQ_CFG_PACING_M |
+ ICE_AQ_CFG_PACING_TYPE_M);
/* update fc info */
tx_pause = !!(link_data.an_info & ICE_AQ_LINK_PAUSE_TX);
@@ -320,12 +322,24 @@ ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
else
hw_fc_info->current_mode = ICE_FC_NONE;
- hw_link_info->lse_ena =
- !!(resp->cmd_flags & cpu_to_le16(ICE_AQ_LSE_IS_ENABLED));
+ li->lse_ena = !!(resp->cmd_flags & cpu_to_le16(ICE_AQ_LSE_IS_ENABLED));
+
+ ice_debug(hw, ICE_DBG_LINK, "link_speed = 0x%x\n", li->link_speed);
+ ice_debug(hw, ICE_DBG_LINK, "phy_type_low = 0x%llx\n",
+ (unsigned long long)li->phy_type_low);
+ ice_debug(hw, ICE_DBG_LINK, "phy_type_high = 0x%llx\n",
+ (unsigned long long)li->phy_type_high);
+ ice_debug(hw, ICE_DBG_LINK, "media_type = 0x%x\n", *hw_media_type);
+ ice_debug(hw, ICE_DBG_LINK, "link_info = 0x%x\n", li->link_info);
+ ice_debug(hw, ICE_DBG_LINK, "an_info = 0x%x\n", li->an_info);
+ ice_debug(hw, ICE_DBG_LINK, "ext_info = 0x%x\n", li->ext_info);
+ ice_debug(hw, ICE_DBG_LINK, "lse_ena = 0x%x\n", li->lse_ena);
+ ice_debug(hw, ICE_DBG_LINK, "max_frame = 0x%x\n", li->max_frame_size);
+ ice_debug(hw, ICE_DBG_LINK, "pacing = 0x%x\n", li->pacing);
/* save link status information */
if (link)
- *link = *hw_link_info;
+ *link = *li;
/* flag cleared so calling functions don't call AQ again */
pi->phy.get_link_info = false;
@@ -1118,6 +1132,7 @@ const struct ice_ctx_ele ice_tlan_ctx_info[] = {
ICE_CTX_STORE(ice_tlan_ctx, vmvf_type, 2, 78),
ICE_CTX_STORE(ice_tlan_ctx, src_vsi, 10, 80),
ICE_CTX_STORE(ice_tlan_ctx, tsyn_ena, 1, 90),
+ ICE_CTX_STORE(ice_tlan_ctx, internal_usage_flag, 1, 91),
ICE_CTX_STORE(ice_tlan_ctx, alt_vlan, 1, 92),
ICE_CTX_STORE(ice_tlan_ctx, cpuid, 8, 93),
ICE_CTX_STORE(ice_tlan_ctx, wb_mode, 1, 101),
@@ -1136,7 +1151,7 @@ const struct ice_ctx_ele ice_tlan_ctx_info[] = {
ICE_CTX_STORE(ice_tlan_ctx, drop_ena, 1, 165),
ICE_CTX_STORE(ice_tlan_ctx, cache_prof_idx, 2, 166),
ICE_CTX_STORE(ice_tlan_ctx, pkt_shaper_prof_idx, 3, 168),
- ICE_CTX_STORE(ice_tlan_ctx, int_q_state, 110, 171),
+ ICE_CTX_STORE(ice_tlan_ctx, int_q_state, 122, 171),
{ 0 }
};
@@ -1261,7 +1276,7 @@ enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading)
ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_q_shutdown);
if (unloading)
- cmd->driver_unloading = cpu_to_le32(ICE_AQC_DRIVER_UNLOADING);
+ cmd->driver_unloading = ICE_AQC_DRIVER_UNLOADING;
return ice_aq_send_cmd(hw, &desc, NULL, 0, NULL);
}
@@ -1536,29 +1551,29 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,
case ICE_AQC_CAPS_VALID_FUNCTIONS:
caps->valid_functions = number;
ice_debug(hw, ICE_DBG_INIT,
- "%s: valid functions = %d\n", prefix,
+ "%s: valid_functions (bitmap) = %d\n", prefix,
caps->valid_functions);
break;
case ICE_AQC_CAPS_SRIOV:
caps->sr_iov_1_1 = (number == 1);
ice_debug(hw, ICE_DBG_INIT,
- "%s: SR-IOV = %d\n", prefix,
+ "%s: sr_iov_1_1 = %d\n", prefix,
caps->sr_iov_1_1);
break;
case ICE_AQC_CAPS_VF:
if (dev_p) {
dev_p->num_vfs_exposed = number;
ice_debug(hw, ICE_DBG_INIT,
- "%s: VFs exposed = %d\n", prefix,
+ "%s: num_vfs_exposed = %d\n", prefix,
dev_p->num_vfs_exposed);
} else if (func_p) {
func_p->num_allocd_vfs = number;
func_p->vf_base_id = logical_id;
ice_debug(hw, ICE_DBG_INIT,
- "%s: VFs allocated = %d\n", prefix,
+ "%s: num_allocd_vfs = %d\n", prefix,
func_p->num_allocd_vfs);
ice_debug(hw, ICE_DBG_INIT,
- "%s: VF base_id = %d\n", prefix,
+ "%s: vf_base_id = %d\n", prefix,
func_p->vf_base_id);
}
break;
@@ -1566,63 +1581,75 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,
if (dev_p) {
dev_p->num_vsi_allocd_to_host = number;
ice_debug(hw, ICE_DBG_INIT,
- "%s: num VSI alloc to host = %d\n",
+ "%s: num_vsi_allocd_to_host = %d\n",
prefix,
dev_p->num_vsi_allocd_to_host);
} else if (func_p) {
func_p->guar_num_vsi =
ice_get_num_per_func(hw, ICE_MAX_VSI);
ice_debug(hw, ICE_DBG_INIT,
- "%s: num guaranteed VSI (fw) = %d\n",
+ "%s: guar_num_vsi (fw) = %d\n",
prefix, number);
ice_debug(hw, ICE_DBG_INIT,
- "%s: num guaranteed VSI = %d\n",
+ "%s: guar_num_vsi = %d\n",
prefix, func_p->guar_num_vsi);
}
break;
+ case ICE_AQC_CAPS_DCB:
+ caps->dcb = (number == 1);
+ caps->active_tc_bitmap = logical_id;
+ caps->maxtc = phys_id;
+ ice_debug(hw, ICE_DBG_INIT,
+ "%s: dcb = %d\n", prefix, caps->dcb);
+ ice_debug(hw, ICE_DBG_INIT,
+ "%s: active_tc_bitmap = %d\n", prefix,
+ caps->active_tc_bitmap);
+ ice_debug(hw, ICE_DBG_INIT,
+ "%s: maxtc = %d\n", prefix, caps->maxtc);
+ break;
case ICE_AQC_CAPS_RSS:
caps->rss_table_size = number;
caps->rss_table_entry_width = logical_id;
ice_debug(hw, ICE_DBG_INIT,
- "%s: RSS table size = %d\n", prefix,
+ "%s: rss_table_size = %d\n", prefix,
caps->rss_table_size);
ice_debug(hw, ICE_DBG_INIT,
- "%s: RSS table width = %d\n", prefix,
+ "%s: rss_table_entry_width = %d\n", prefix,
caps->rss_table_entry_width);
break;
case ICE_AQC_CAPS_RXQS:
caps->num_rxq = number;
caps->rxq_first_id = phys_id;
ice_debug(hw, ICE_DBG_INIT,
- "%s: num Rx queues = %d\n", prefix,
+ "%s: num_rxq = %d\n", prefix,
caps->num_rxq);
ice_debug(hw, ICE_DBG_INIT,
- "%s: Rx first queue ID = %d\n", prefix,
+ "%s: rxq_first_id = %d\n", prefix,
caps->rxq_first_id);
break;
case ICE_AQC_CAPS_TXQS:
caps->num_txq = number;
caps->txq_first_id = phys_id;
ice_debug(hw, ICE_DBG_INIT,
- "%s: num Tx queues = %d\n", prefix,
+ "%s: num_txq = %d\n", prefix,
caps->num_txq);
ice_debug(hw, ICE_DBG_INIT,
- "%s: Tx first queue ID = %d\n", prefix,
+ "%s: txq_first_id = %d\n", prefix,
caps->txq_first_id);
break;
case ICE_AQC_CAPS_MSIX:
caps->num_msix_vectors = number;
caps->msix_vector_first_id = phys_id;
ice_debug(hw, ICE_DBG_INIT,
- "%s: MSIX vector count = %d\n", prefix,
+ "%s: num_msix_vectors = %d\n", prefix,
caps->num_msix_vectors);
ice_debug(hw, ICE_DBG_INIT,
- "%s: MSIX first vector index = %d\n", prefix,
+ "%s: msix_vector_first_id = %d\n", prefix,
caps->msix_vector_first_id);
break;
case ICE_AQC_CAPS_MAX_MTU:
caps->max_mtu = number;
- ice_debug(hw, ICE_DBG_INIT, "%s: max MTU = %d\n",
+ ice_debug(hw, ICE_DBG_INIT, "%s: max_mtu = %d\n",
prefix, caps->max_mtu);
break;
default:
@@ -2000,6 +2027,17 @@ ice_aq_set_phy_cfg(struct ice_hw *hw, u8 lport,
desc.params.set_phy.lport_num = lport;
desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ ice_debug(hw, ICE_DBG_LINK, "phy_type_low = 0x%llx\n",
+ (unsigned long long)le64_to_cpu(cfg->phy_type_low));
+ ice_debug(hw, ICE_DBG_LINK, "phy_type_high = 0x%llx\n",
+ (unsigned long long)le64_to_cpu(cfg->phy_type_high));
+ ice_debug(hw, ICE_DBG_LINK, "caps = 0x%x\n", cfg->caps);
+ ice_debug(hw, ICE_DBG_LINK, "low_power_ctrl = 0x%x\n",
+ cfg->low_power_ctrl);
+ ice_debug(hw, ICE_DBG_LINK, "eee_cap = 0x%x\n", cfg->eee_cap);
+ ice_debug(hw, ICE_DBG_LINK, "eeer_value = 0x%x\n", cfg->eeer_value);
+ ice_debug(hw, ICE_DBG_LINK, "link_fec_opt = 0x%x\n", cfg->link_fec_opt);
+
return ice_aq_send_cmd(hw, &desc, cfg, sizeof(*cfg), cd);
}
@@ -2031,7 +2069,7 @@ enum ice_status ice_update_link_info(struct ice_port_info *pi)
if (!pcaps)
return ICE_ERR_NO_MEMORY;
- status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_SW_CFG,
+ status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_TOPO_CAP,
pcaps, NULL);
if (!status)
memcpy(li->module_type, &pcaps->module_type,
@@ -2181,27 +2219,24 @@ ice_cfg_phy_fec(struct ice_aqc_set_phy_cfg_data *cfg, enum ice_fec_mode fec)
{
switch (fec) {
case ICE_FEC_BASER:
- /* Clear auto FEC and RS bits, and AND BASE-R ability
+ /* Clear RS bits, and AND BASE-R ability
* bits and OR request bits.
*/
- cfg->caps &= ~ICE_AQC_PHY_EN_AUTO_FEC;
cfg->link_fec_opt &= ICE_AQC_PHY_FEC_10G_KR_40G_KR4_EN |
ICE_AQC_PHY_FEC_25G_KR_CLAUSE74_EN;
cfg->link_fec_opt |= ICE_AQC_PHY_FEC_10G_KR_40G_KR4_REQ |
ICE_AQC_PHY_FEC_25G_KR_REQ;
break;
case ICE_FEC_RS:
- /* Clear auto FEC and BASE-R bits, and AND RS ability
+ /* Clear BASE-R bits, and AND RS ability
* bits and OR request bits.
*/
- cfg->caps &= ~ICE_AQC_PHY_EN_AUTO_FEC;
cfg->link_fec_opt &= ICE_AQC_PHY_FEC_25G_RS_CLAUSE91_EN;
cfg->link_fec_opt |= ICE_AQC_PHY_FEC_25G_RS_528_REQ |
ICE_AQC_PHY_FEC_25G_RS_544_REQ;
break;
case ICE_FEC_NONE:
- /* Clear auto FEC and all FEC option bits. */
- cfg->caps &= ~ICE_AQC_PHY_EN_AUTO_FEC;
+ /* Clear all FEC option bits. */
cfg->link_fec_opt &= ~ICE_AQC_PHY_FEC_MASK;
break;
case ICE_FEC_AUTO:
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb.c b/drivers/net/ethernet/intel/ice/ice_dcb.c
index c2002de..dd7efff 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb.c
+++ b/drivers/net/ethernet/intel/ice/ice_dcb.c
@@ -60,7 +60,7 @@ ice_aq_get_lldp_mib(struct ice_hw *hw, u8 bridge_type, u8 mib_type, void *buf,
* Enable or Disable posting of an event on ARQ when LLDP MIB
* associated with the interface changes (0x0A01)
*/
-enum ice_status
+static enum ice_status
ice_aq_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_update,
struct ice_sq_cd *cd)
{
@@ -444,9 +444,15 @@ ice_parse_cee_pgcfg_tlv(struct ice_cee_feat_tlv *tlv,
* |pg0|pg1|pg2|pg3|pg4|pg5|pg6|pg7|
* ---------------------------------
*/
- ice_for_each_traffic_class(i)
+ ice_for_each_traffic_class(i) {
etscfg->tcbwtable[i] = buf[offset++];
+ if (etscfg->prio_table[i] == ICE_CEE_PGID_STRICT)
+ dcbcfg->etscfg.tsatable[i] = ICE_IEEE_TSA_STRICT;
+ else
+ dcbcfg->etscfg.tsatable[i] = ICE_IEEE_TSA_ETS;
+ }
+
/* Number of TCs supported (1 octet) */
etscfg->maxtcs = buf[offset];
}
@@ -937,10 +943,11 @@ enum ice_status ice_get_dcb_cfg(struct ice_port_info *pi)
/**
* ice_init_dcb
* @hw: pointer to the HW struct
+ * @enable_mib_change: enable MIB change event
*
* Update DCB configuration from the Firmware
*/
-enum ice_status ice_init_dcb(struct ice_hw *hw)
+enum ice_status ice_init_dcb(struct ice_hw *hw, bool enable_mib_change)
{
struct ice_port_info *pi = hw->port_info;
enum ice_status ret = 0;
@@ -954,7 +961,8 @@ enum ice_status ice_init_dcb(struct ice_hw *hw)
pi->dcbx_status = ice_get_dcbx_status(hw);
if (pi->dcbx_status == ICE_DCBX_STATUS_DONE ||
- pi->dcbx_status == ICE_DCBX_STATUS_IN_PROGRESS) {
+ pi->dcbx_status == ICE_DCBX_STATUS_IN_PROGRESS ||
+ pi->dcbx_status == ICE_DCBX_STATUS_NOT_STARTED) {
/* Get current DCBX configuration */
ret = ice_get_dcb_cfg(pi);
pi->is_sw_lldp = (hw->adminq.sq_last_status == ICE_AQ_RC_EPERM);
@@ -965,9 +973,39 @@ enum ice_status ice_init_dcb(struct ice_hw *hw)
}
/* Configure the LLDP MIB change event */
- ret = ice_aq_cfg_lldp_mib_change(hw, true, NULL);
+ if (enable_mib_change) {
+ ret = ice_aq_cfg_lldp_mib_change(hw, true, NULL);
+ if (!ret)
+ pi->is_sw_lldp = false;
+ }
+
+ return ret;
+}
+
+/**
+ * ice_cfg_lldp_mib_change
+ * @hw: pointer to the HW struct
+ * @ena_mib: enable/disable MIB change event
+ *
+ * Configure (disable/enable) MIB
+ */
+enum ice_status ice_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_mib)
+{
+ struct ice_port_info *pi = hw->port_info;
+ enum ice_status ret;
+
+ if (!hw->func_caps.common_cap.dcb)
+ return ICE_ERR_NOT_SUPPORTED;
+
+ /* Get DCBX status */
+ pi->dcbx_status = ice_get_dcbx_status(hw);
+
+ if (pi->dcbx_status == ICE_DCBX_STATUS_DIS)
+ return ICE_ERR_NOT_READY;
+
+ ret = ice_aq_cfg_lldp_mib_change(hw, ena_mib, NULL);
if (!ret)
- pi->is_sw_lldp = false;
+ pi->is_sw_lldp = !ena_mib;
return ret;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb.h b/drivers/net/ethernet/intel/ice/ice_dcb.h
index 522e145..ee138f9 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb.h
+++ b/drivers/net/ethernet/intel/ice/ice_dcb.h
@@ -125,7 +125,7 @@ ice_aq_get_dcb_cfg(struct ice_hw *hw, u8 mib_type, u8 bridgetype,
struct ice_dcbx_cfg *dcbcfg);
enum ice_status ice_get_dcb_cfg(struct ice_port_info *pi);
enum ice_status ice_set_dcb_cfg(struct ice_port_info *pi);
-enum ice_status ice_init_dcb(struct ice_hw *hw);
+enum ice_status ice_init_dcb(struct ice_hw *hw, bool enable_mib_change);
enum ice_status
ice_query_port_ets(struct ice_port_info *pi,
struct ice_aqc_port_ets_elem *buf, u16 buf_size,
@@ -139,9 +139,7 @@ ice_aq_start_lldp(struct ice_hw *hw, bool persist, struct ice_sq_cd *cd);
enum ice_status
ice_aq_start_stop_dcbx(struct ice_hw *hw, bool start_dcbx_agent,
bool *dcbx_agent_status, struct ice_sq_cd *cd);
-enum ice_status
-ice_aq_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_update,
- struct ice_sq_cd *cd);
+enum ice_status ice_cfg_lldp_mib_change(struct ice_hw *hw, bool ena_mib);
#else /* CONFIG_DCB */
static inline enum ice_status
ice_aq_stop_lldp(struct ice_hw __always_unused *hw,
@@ -172,9 +170,8 @@ ice_aq_start_stop_dcbx(struct ice_hw __always_unused *hw,
}
static inline enum ice_status
-ice_aq_cfg_lldp_mib_change(struct ice_hw __always_unused *hw,
- bool __always_unused ena_update,
- struct ice_sq_cd __always_unused *cd)
+ice_cfg_lldp_mib_change(struct ice_hw __always_unused *hw,
+ bool __always_unused ena_mib)
{
return 0;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
index bf6cd47..97c22d4 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
@@ -318,7 +318,12 @@ void ice_dcb_rebuild(struct ice_pf *pf)
goto dcb_error;
}
- ice_init_dcb(&pf->hw);
+ ice_init_dcb(&pf->hw, true);
+ if (pf->hw.port_info->dcbx_status == ICE_DCBX_STATUS_DIS)
+ pf->hw.port_info->is_sw_lldp = true;
+ else
+ pf->hw.port_info->is_sw_lldp = false;
+
if (ice_dcb_need_recfg(pf, prev_cfg, local_dcbx_cfg)) {
/* difference in cfg detected - disable DCB till next MIB */
dev_err(&pf->pdev->dev, "Set local MIB not accurate\n");
@@ -329,8 +334,10 @@ void ice_dcb_rebuild(struct ice_pf *pf)
devm_kfree(&pf->pdev->dev, prev_cfg);
/* Set the local desired config */
- memset(&pf->hw.port_info->local_dcbx_cfg, 0, sizeof(*local_dcbx_cfg));
- memcpy(local_dcbx_cfg, desired_dcbx_cfg, sizeof(*local_dcbx_cfg));
+ if (local_dcbx_cfg->dcbx_mode == ICE_DCBX_MODE_CEE)
+ memcpy(local_dcbx_cfg, desired_dcbx_cfg,
+ sizeof(*local_dcbx_cfg));
+
ice_cfg_etsrec_defaults(pf->hw.port_info);
ret = ice_set_dcb_cfg(pf->hw.port_info);
if (ret) {
@@ -406,7 +413,7 @@ static int ice_dcb_sw_dflt_cfg(struct ice_pf *pf, bool locked)
memset(&pi->local_dcbx_cfg, 0, sizeof(*dcbcfg));
dcbcfg->etscfg.willing = 1;
- dcbcfg->etscfg.maxtcs = 8;
+ dcbcfg->etscfg.maxtcs = hw->func_caps.common_cap.maxtc;
dcbcfg->etscfg.tcbwtable[0] = 100;
dcbcfg->etscfg.tsatable[0] = ICE_IEEE_TSA_ETS;
@@ -415,7 +422,7 @@ static int ice_dcb_sw_dflt_cfg(struct ice_pf *pf, bool locked)
dcbcfg->etsrec.willing = 0;
dcbcfg->pfc.willing = 1;
- dcbcfg->pfc.pfccap = IEEE_8021QAZ_MAX_TCS;
+ dcbcfg->pfc.pfccap = hw->func_caps.common_cap.maxtc;
dcbcfg->numapps = 1;
dcbcfg->app[0].selector = ICE_APP_SEL_ETHTYPE;
@@ -440,35 +447,24 @@ int ice_init_pf_dcb(struct ice_pf *pf, bool locked)
struct device *dev = &pf->pdev->dev;
struct ice_port_info *port_info;
struct ice_hw *hw = &pf->hw;
- int sw_default = 0;
int err;
port_info = hw->port_info;
- err = ice_init_dcb(hw);
+ err = ice_init_dcb(hw, false);
+ if (err && !port_info->is_sw_lldp) {
+ dev_err(&pf->pdev->dev, "Error initializing DCB %d\n", err);
+ goto dcb_init_err;
+ }
+
+ dev_info(&pf->pdev->dev,
+ "DCB is enabled in the hardware, max number of TCs supported on this port are %d\n",
+ pf->hw.func_caps.common_cap.maxtc);
if (err) {
- /* FW LLDP is not active, default to SW DCBX/LLDP */
- dev_info(&pf->pdev->dev, "FW LLDP is not active\n");
- hw->port_info->dcbx_status = ICE_DCBX_STATUS_NOT_STARTED;
- hw->port_info->is_sw_lldp = true;
- }
-
- if (port_info->dcbx_status == ICE_DCBX_STATUS_DIS)
- dev_info(&pf->pdev->dev, "DCBX disabled\n");
-
- /* LLDP disabled in FW */
- if (port_info->is_sw_lldp) {
- sw_default = 1;
- dev_info(&pf->pdev->dev, "DCBx/LLDP in SW mode.\n");
- clear_bit(ICE_FLAG_ENABLE_FW_LLDP, pf->flags);
- } else {
- set_bit(ICE_FLAG_ENABLE_FW_LLDP, pf->flags);
- }
-
- if (port_info->dcbx_status == ICE_DCBX_STATUS_NOT_STARTED)
- dev_info(&pf->pdev->dev, "DCBX not started\n");
-
- if (sw_default) {
+ /* FW LLDP is disabled, activate SW DCBX/LLDP mode */
+ dev_info(&pf->pdev->dev,
+ "FW LLDP is disabled, DCBx/LLDP in SW mode.\n");
+ clear_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags);
err = ice_dcb_sw_dflt_cfg(pf, locked);
if (err) {
dev_err(&pf->pdev->dev,
@@ -478,21 +474,18 @@ int ice_init_pf_dcb(struct ice_pf *pf, bool locked)
}
pf->dcbx_cap = DCB_CAP_DCBX_HOST | DCB_CAP_DCBX_VER_IEEE;
- set_bit(ICE_FLAG_DCB_CAPABLE, pf->flags);
- set_bit(ICE_FLAG_DCB_ENA, pf->flags);
return 0;
}
+ set_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags);
+
/* DCBX in FW and LLDP enabled in FW */
pf->dcbx_cap = DCB_CAP_DCBX_LLD_MANAGED | DCB_CAP_DCBX_VER_IEEE;
- set_bit(ICE_FLAG_DCB_CAPABLE, pf->flags);
-
err = ice_dcb_init_cfg(pf, locked);
if (err)
goto dcb_init_err;
- dev_info(&pf->pdev->dev, "DCBX offload supported\n");
return err;
dcb_init_err:
@@ -508,30 +501,31 @@ void ice_update_dcb_stats(struct ice_pf *pf)
{
struct ice_hw_port_stats *prev_ps, *cur_ps;
struct ice_hw *hw = &pf->hw;
- u8 pf_id = hw->pf_id;
+ u8 port;
int i;
+ port = hw->port_info->lport;
prev_ps = &pf->stats_prev;
cur_ps = &pf->stats;
for (i = 0; i < 8; i++) {
- ice_stat_update32(hw, GLPRT_PXOFFRXC(pf_id, i),
+ ice_stat_update32(hw, GLPRT_PXOFFRXC(port, i),
pf->stat_prev_loaded,
&prev_ps->priority_xoff_rx[i],
&cur_ps->priority_xoff_rx[i]);
- ice_stat_update32(hw, GLPRT_PXONRXC(pf_id, i),
+ ice_stat_update32(hw, GLPRT_PXONRXC(port, i),
pf->stat_prev_loaded,
&prev_ps->priority_xon_rx[i],
&cur_ps->priority_xon_rx[i]);
- ice_stat_update32(hw, GLPRT_PXONTXC(pf_id, i),
+ ice_stat_update32(hw, GLPRT_PXONTXC(port, i),
pf->stat_prev_loaded,
&prev_ps->priority_xon_tx[i],
&cur_ps->priority_xon_tx[i]);
- ice_stat_update32(hw, GLPRT_PXOFFTXC(pf_id, i),
+ ice_stat_update32(hw, GLPRT_PXOFFTXC(port, i),
pf->stat_prev_loaded,
&prev_ps->priority_xoff_tx[i],
&cur_ps->priority_xoff_tx[i]);
- ice_stat_update32(hw, GLPRT_RXON2OFFCNT(pf_id, i),
+ ice_stat_update32(hw, GLPRT_RXON2OFFCNT(port, i),
pf->stat_prev_loaded,
&prev_ps->priority_xon_2_xoff[i],
&cur_ps->priority_xon_2_xoff[i]);
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index 6a97ddb..d5db142 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -155,7 +155,7 @@ struct ice_priv_flag {
static const struct ice_priv_flag ice_gstrings_priv_flags[] = {
ICE_PRIV_FLAG("link-down-on-close", ICE_FLAG_LINK_DOWN_ON_CLOSE_ENA),
- ICE_PRIV_FLAG("enable-fw-lldp", ICE_FLAG_ENABLE_FW_LLDP),
+ ICE_PRIV_FLAG("fw-lldp-agent", ICE_FLAG_FW_LLDP_AGENT),
};
#define ICE_PRIV_FLAG_ARRAY_SIZE ARRAY_SIZE(ice_gstrings_priv_flags)
@@ -1201,13 +1201,13 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
bitmap_xor(change_flags, pf->flags, orig_flags, ICE_PF_FLAGS_NBITS);
- if (test_bit(ICE_FLAG_ENABLE_FW_LLDP, change_flags)) {
- if (!test_bit(ICE_FLAG_ENABLE_FW_LLDP, pf->flags)) {
+ if (test_bit(ICE_FLAG_FW_LLDP_AGENT, change_flags)) {
+ if (!test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags)) {
enum ice_status status;
/* Disable FW LLDP engine */
- status = ice_aq_cfg_lldp_mib_change(&pf->hw, false,
- NULL);
+ status = ice_cfg_lldp_mib_change(&pf->hw, false);
+
/* If unregistering for LLDP events fails, this is
* not an error state, as there shouldn't be any
* events to respond to.
@@ -1273,6 +1273,12 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
* The FW LLDP engine will now be consuming them.
*/
ice_cfg_sw_lldp(vsi, false, false);
+
+ /* Register for MIB change events */
+ status = ice_cfg_lldp_mib_change(&pf->hw, true);
+ if (status)
+ dev_dbg(&pf->pdev->dev,
+ "Fail to enable MIB change events\n");
}
}
clear_bit(ICE_FLAG_ETHTOOL_CTXT, pf->flags);
@@ -1319,14 +1325,17 @@ ice_get_ethtool_stats(struct net_device *netdev,
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
struct ice_ring *ring;
- unsigned int j = 0;
+ unsigned int j;
int i = 0;
char *p;
+ ice_update_pf_stats(pf);
+ ice_update_vsi_stats(vsi);
+
for (j = 0; j < ICE_VSI_STATS_LEN; j++) {
p = (char *)vsi + ice_gstrings_vsi_stats[j].stat_offset;
data[i++] = (ice_gstrings_vsi_stats[j].sizeof_stat ==
- sizeof(u64)) ? *(u64 *)p : *(u32 *)p;
+ sizeof(u64)) ? *(u64 *)p : *(u32 *)p;
}
/* populate per queue stats */
@@ -3211,12 +3220,6 @@ __ice_get_coalesce(struct net_device *netdev, struct ethtool_coalesce *ec,
if (ice_get_q_coalesce(vsi, ec, q_num))
return -EINVAL;
- if (q_num < vsi->num_txq)
- ec->tx_max_coalesced_frames_irq = vsi->work_lmt;
-
- if (q_num < vsi->num_rxq)
- ec->rx_max_coalesced_frames_irq = vsi->work_lmt;
-
return 0;
}
@@ -3250,25 +3253,25 @@ static int
ice_set_rc_coalesce(enum ice_container_type c_type, struct ethtool_coalesce *ec,
struct ice_ring_container *rc, struct ice_vsi *vsi)
{
+ const char *c_type_str = (c_type == ICE_RX_CONTAINER) ? "rx" : "tx";
+ u32 use_adaptive_coalesce, coalesce_usecs;
struct ice_pf *pf = vsi->back;
u16 itr_setting;
if (!rc->ring)
return -EINVAL;
- itr_setting = rc->itr_setting & ~ICE_ITR_DYNAMIC;
-
switch (c_type) {
case ICE_RX_CONTAINER:
if (ec->rx_coalesce_usecs_high > ICE_MAX_INTRL ||
(ec->rx_coalesce_usecs_high &&
ec->rx_coalesce_usecs_high < pf->hw.intrl_gran)) {
netdev_info(vsi->netdev,
- "Invalid value, rx-usecs-high valid values are 0 (disabled), %d-%d\n",
- pf->hw.intrl_gran, ICE_MAX_INTRL);
+ "Invalid value, %s-usecs-high valid values are 0 (disabled), %d-%d\n",
+ c_type_str, pf->hw.intrl_gran,
+ ICE_MAX_INTRL);
return -EINVAL;
}
-
if (ec->rx_coalesce_usecs_high != rc->ring->q_vector->intrl) {
rc->ring->q_vector->intrl = ec->rx_coalesce_usecs_high;
wr32(&pf->hw, GLINT_RATE(rc->ring->q_vector->reg_idx),
@@ -3276,60 +3279,60 @@ ice_set_rc_coalesce(enum ice_container_type c_type, struct ethtool_coalesce *ec,
pf->hw.intrl_gran));
}
- if (ec->rx_coalesce_usecs != itr_setting &&
- ec->use_adaptive_rx_coalesce) {
- netdev_info(vsi->netdev,
- "Rx interrupt throttling cannot be changed if adaptive-rx is enabled\n");
- return -EINVAL;
- }
+ use_adaptive_coalesce = ec->use_adaptive_rx_coalesce;
+ coalesce_usecs = ec->rx_coalesce_usecs;
- if (ec->rx_coalesce_usecs > ICE_ITR_MAX) {
- netdev_info(vsi->netdev,
- "Invalid value, rx-usecs range is 0-%d\n",
- ICE_ITR_MAX);
- return -EINVAL;
- }
-
- if (ec->use_adaptive_rx_coalesce) {
- rc->itr_setting |= ICE_ITR_DYNAMIC;
- } else {
- rc->itr_setting = ITR_REG_ALIGN(ec->rx_coalesce_usecs);
- rc->target_itr = ITR_TO_REG(rc->itr_setting);
- }
break;
case ICE_TX_CONTAINER:
if (ec->tx_coalesce_usecs_high) {
netdev_info(vsi->netdev,
- "setting tx-usecs-high is not supported\n");
+ "setting %s-usecs-high is not supported\n",
+ c_type_str);
return -EINVAL;
}
- if (ec->tx_coalesce_usecs != itr_setting &&
- ec->use_adaptive_tx_coalesce) {
- netdev_info(vsi->netdev,
- "Tx interrupt throttling cannot be changed if adaptive-tx is enabled\n");
- return -EINVAL;
- }
+ use_adaptive_coalesce = ec->use_adaptive_tx_coalesce;
+ coalesce_usecs = ec->tx_coalesce_usecs;
- if (ec->tx_coalesce_usecs > ICE_ITR_MAX) {
- netdev_info(vsi->netdev,
- "Invalid value, tx-usecs range is 0-%d\n",
- ICE_ITR_MAX);
- return -EINVAL;
- }
-
- if (ec->use_adaptive_tx_coalesce) {
- rc->itr_setting |= ICE_ITR_DYNAMIC;
- } else {
- rc->itr_setting = ITR_REG_ALIGN(ec->tx_coalesce_usecs);
- rc->target_itr = ITR_TO_REG(rc->itr_setting);
- }
break;
default:
dev_dbg(&pf->pdev->dev, "Invalid container type %d\n", c_type);
return -EINVAL;
}
+ itr_setting = rc->itr_setting & ~ICE_ITR_DYNAMIC;
+ if (coalesce_usecs != itr_setting && use_adaptive_coalesce) {
+ netdev_info(vsi->netdev,
+ "%s interrupt throttling cannot be changed if adaptive-%s is enabled\n",
+ c_type_str, c_type_str);
+ return -EINVAL;
+ }
+
+ if (coalesce_usecs > ICE_ITR_MAX) {
+ netdev_info(vsi->netdev,
+ "Invalid value, %s-usecs range is 0-%d\n",
+ c_type_str, ICE_ITR_MAX);
+ return -EINVAL;
+ }
+
+ /* hardware only supports an ITR granularity of 2us */
+ if (coalesce_usecs % 2 != 0) {
+ netdev_info(vsi->netdev,
+ "Invalid value, %s-usecs must be even\n",
+ c_type_str);
+ return -EINVAL;
+ }
+
+ if (use_adaptive_coalesce) {
+ rc->itr_setting |= ICE_ITR_DYNAMIC;
+ } else {
+ /* store user facing value how it was set */
+ rc->itr_setting = coalesce_usecs;
+ /* set to static and convert to value HW understands */
+ rc->target_itr =
+ ITR_TO_REG(ITR_REG_ALIGN(rc->itr_setting));
+ }
+
return 0;
}
@@ -3396,17 +3399,13 @@ __ice_set_coalesce(struct net_device *netdev, struct ethtool_coalesce *ec,
if (ice_set_q_coalesce(vsi, ec, i))
return -EINVAL;
}
- goto set_work_lmt;
+ goto set_complete;
}
if (ice_set_q_coalesce(vsi, ec, q_num))
return -EINVAL;
-set_work_lmt:
-
- if (ec->tx_max_coalesced_frames_irq || ec->rx_max_coalesced_frames_irq)
- vsi->work_lmt = max(ec->tx_max_coalesced_frames_irq,
- ec->rx_max_coalesced_frames_irq);
+set_complete:
return 0;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h b/drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h
index 57ea681..2aac8f1 100644
--- a/drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h
+++ b/drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h
@@ -428,6 +428,7 @@ struct ice_tlan_ctx {
#define ICE_TLAN_CTX_VMVF_TYPE_PF 2
u16 src_vsi;
u8 tsyn_ena;
+ u8 internal_usage_flag;
u8 alt_vlan;
u16 cpuid; /* bigger than needed, see above for reason */
u8 wb_mode;
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 6e34c40..7cd8c5d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -191,41 +191,58 @@ static int ice_pf_rxq_wait(struct ice_pf *pf, int pf_q, bool ena)
}
/**
+ * ice_vsi_ctrl_rx_ring - Start or stop a VSI's Rx ring
+ * @vsi: the VSI being configured
+ * @ena: start or stop the Rx rings
+ * @rxq_idx: Rx queue index
+ */
+#ifndef CONFIG_PCI_IOV
+static
+#endif /* !CONFIG_PCI_IOV */
+int ice_vsi_ctrl_rx_ring(struct ice_vsi *vsi, bool ena, u16 rxq_idx)
+{
+ int pf_q = vsi->rxq_map[rxq_idx];
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ int ret = 0;
+ u32 rx_reg;
+
+ rx_reg = rd32(hw, QRX_CTRL(pf_q));
+
+ /* Skip if the queue is already in the requested state */
+ if (ena == !!(rx_reg & QRX_CTRL_QENA_STAT_M))
+ return 0;
+
+ /* turn on/off the queue */
+ if (ena)
+ rx_reg |= QRX_CTRL_QENA_REQ_M;
+ else
+ rx_reg &= ~QRX_CTRL_QENA_REQ_M;
+ wr32(hw, QRX_CTRL(pf_q), rx_reg);
+
+ /* wait for the change to finish */
+ ret = ice_pf_rxq_wait(pf, pf_q, ena);
+ if (ret)
+ dev_err(&pf->pdev->dev,
+ "VSI idx %d Rx ring %d %sable timeout\n",
+ vsi->idx, pf_q, (ena ? "en" : "dis"));
+
+ return ret;
+}
+
+/**
* ice_vsi_ctrl_rx_rings - Start or stop a VSI's Rx rings
* @vsi: the VSI being configured
* @ena: start or stop the Rx rings
*/
static int ice_vsi_ctrl_rx_rings(struct ice_vsi *vsi, bool ena)
{
- struct ice_pf *pf = vsi->back;
- struct ice_hw *hw = &pf->hw;
int i, ret = 0;
for (i = 0; i < vsi->num_rxq; i++) {
- int pf_q = vsi->rxq_map[i];
- u32 rx_reg;
-
- rx_reg = rd32(hw, QRX_CTRL(pf_q));
-
- /* Skip if the queue is already in the requested state */
- if (ena == !!(rx_reg & QRX_CTRL_QENA_STAT_M))
- continue;
-
- /* turn on/off the queue */
- if (ena)
- rx_reg |= QRX_CTRL_QENA_REQ_M;
- else
- rx_reg &= ~QRX_CTRL_QENA_REQ_M;
- wr32(hw, QRX_CTRL(pf_q), rx_reg);
-
- /* wait for the change to finish */
- ret = ice_pf_rxq_wait(pf, pf_q, ena);
- if (ret) {
- dev_err(&pf->pdev->dev,
- "VSI idx %d Rx ring %d %sable timeout\n",
- vsi->idx, pf_q, (ena ? "en" : "dis"));
+ ret = ice_vsi_ctrl_rx_ring(vsi, ena, i);
+ if (ret)
break;
- }
}
return ret;
@@ -246,12 +263,24 @@ static int ice_vsi_alloc_arrays(struct ice_vsi *vsi)
vsi->tx_rings = devm_kcalloc(&pf->pdev->dev, vsi->alloc_txq,
sizeof(*vsi->tx_rings), GFP_KERNEL);
if (!vsi->tx_rings)
- goto err_txrings;
+ return -ENOMEM;
vsi->rx_rings = devm_kcalloc(&pf->pdev->dev, vsi->alloc_rxq,
sizeof(*vsi->rx_rings), GFP_KERNEL);
if (!vsi->rx_rings)
- goto err_rxrings;
+ goto err_rings;
+
+ vsi->txq_map = devm_kcalloc(&pf->pdev->dev, vsi->alloc_txq,
+ sizeof(*vsi->txq_map), GFP_KERNEL);
+
+ if (!vsi->txq_map)
+ goto err_txq_map;
+
+ vsi->rxq_map = devm_kcalloc(&pf->pdev->dev, vsi->alloc_rxq,
+ sizeof(*vsi->rxq_map), GFP_KERNEL);
+ if (!vsi->rxq_map)
+ goto err_rxq_map;
+
/* There is no need to allocate q_vectors for a loopback VSI. */
if (vsi->type == ICE_VSI_LB)
@@ -266,10 +295,13 @@ static int ice_vsi_alloc_arrays(struct ice_vsi *vsi)
return 0;
err_vectors:
+ devm_kfree(&pf->pdev->dev, vsi->rxq_map);
+err_rxq_map:
+ devm_kfree(&pf->pdev->dev, vsi->txq_map);
+err_txq_map:
devm_kfree(&pf->pdev->dev, vsi->rx_rings);
-err_rxrings:
+err_rings:
devm_kfree(&pf->pdev->dev, vsi->tx_rings);
-err_txrings:
return -ENOMEM;
}
@@ -311,9 +343,21 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi, u16 vf_id)
switch (vsi->type) {
case ICE_VSI_PF:
- vsi->alloc_txq = pf->num_lan_tx;
- vsi->alloc_rxq = pf->num_lan_rx;
- vsi->num_q_vectors = max_t(int, pf->num_lan_rx, pf->num_lan_tx);
+ vsi->alloc_txq = min_t(int, ice_get_avail_txq_count(pf),
+ num_online_cpus());
+
+ pf->num_lan_tx = vsi->alloc_txq;
+
+ /* only 1 Rx queue unless RSS is enabled */
+ if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags))
+ vsi->alloc_rxq = 1;
+ else
+ vsi->alloc_rxq = min_t(int, ice_get_avail_rxq_count(pf),
+ num_online_cpus());
+
+ pf->num_lan_rx = vsi->alloc_rxq;
+
+ vsi->num_q_vectors = max_t(int, vsi->alloc_rxq, vsi->alloc_txq);
break;
case ICE_VSI_VF:
vf = &pf->vf[vsi->vf_id];
@@ -416,6 +460,14 @@ static void ice_vsi_free_arrays(struct ice_vsi *vsi)
devm_kfree(&pf->pdev->dev, vsi->rx_rings);
vsi->rx_rings = NULL;
}
+ if (vsi->txq_map) {
+ devm_kfree(&pf->pdev->dev, vsi->txq_map);
+ vsi->txq_map = NULL;
+ }
+ if (vsi->rxq_map) {
+ devm_kfree(&pf->pdev->dev, vsi->rxq_map);
+ vsi->rxq_map = NULL;
+ }
}
/**
@@ -508,8 +560,8 @@ ice_vsi_alloc(struct ice_pf *pf, enum ice_vsi_type type, u16 vf_id)
vsi->type = type;
vsi->back = pf;
set_bit(__ICE_DOWN, vsi->state);
+
vsi->idx = pf->next_vsi;
- vsi->work_lmt = ICE_DFLT_IRQ_WORK;
if (type == ICE_VSI_VF)
ice_vsi_set_num_qs(vsi, vf_id);
@@ -647,7 +699,7 @@ static int ice_vsi_get_qs(struct ice_vsi *vsi)
struct ice_qs_cfg tx_qs_cfg = {
.qs_mutex = &pf->avail_q_mutex,
.pf_map = pf->avail_txqs,
- .pf_map_size = ICE_MAX_TXQS,
+ .pf_map_size = pf->max_pf_txqs,
.q_count = vsi->alloc_txq,
.scatter_count = ICE_MAX_SCATTER_TXQS,
.vsi_map = vsi->txq_map,
@@ -657,7 +709,7 @@ static int ice_vsi_get_qs(struct ice_vsi *vsi)
struct ice_qs_cfg rx_qs_cfg = {
.qs_mutex = &pf->avail_q_mutex,
.pf_map = pf->avail_rxqs,
- .pf_map_size = ICE_MAX_RXQS,
+ .pf_map_size = pf->max_pf_rxqs,
.q_count = vsi->alloc_rxq,
.scatter_count = ICE_MAX_SCATTER_RXQS,
.vsi_map = vsi->rxq_map,
@@ -1010,6 +1062,13 @@ static int ice_vsi_init(struct ice_vsi *vsi)
ICE_AQ_VSI_SEC_FLAG_ENA_MAC_ANTI_SPOOF;
}
+ /* Allow control frames out of main VSI */
+ if (vsi->type == ICE_VSI_PF) {
+ ctxt->info.sec_flags |= ICE_AQ_VSI_SEC_FLAG_ALLOW_DEST_OVRD;
+ ctxt->info.valid_sections |=
+ cpu_to_le16(ICE_AQ_VSI_PROP_SECURITY_VALID);
+ }
+
ret = ice_add_vsi(hw, vsi->idx, ctxt, NULL);
if (ret) {
dev_err(&pf->pdev->dev,
@@ -1642,6 +1701,62 @@ int ice_vsi_cfg_rxqs(struct ice_vsi *vsi)
}
/**
+ * ice_vsi_cfg_txq - Configure single Tx queue
+ * @vsi: the VSI that queue belongs to
+ * @ring: Tx ring to be configured
+ * @tc_q_idx: queue index within given TC
+ * @qg_buf: queue group buffer
+ * @tc: TC that Tx ring belongs to
+ */
+static int
+ice_vsi_cfg_txq(struct ice_vsi *vsi, struct ice_ring *ring, u16 tc_q_idx,
+ struct ice_aqc_add_tx_qgrp *qg_buf, u8 tc)
+{
+ struct ice_tlan_ctx tlan_ctx = { 0 };
+ struct ice_aqc_add_txqs_perq *txq;
+ struct ice_pf *pf = vsi->back;
+ u8 buf_len = sizeof(*qg_buf);
+ enum ice_status status;
+ u16 pf_q;
+
+ pf_q = ring->reg_idx;
+ ice_setup_tx_ctx(ring, &tlan_ctx, pf_q);
+ /* copy context contents into the qg_buf */
+ qg_buf->txqs[0].txq_id = cpu_to_le16(pf_q);
+ ice_set_ctx((u8 *)&tlan_ctx, qg_buf->txqs[0].txq_ctx,
+ ice_tlan_ctx_info);
+
+ /* init queue specific tail reg. It is referred as
+ * transmit comm scheduler queue doorbell.
+ */
+ ring->tail = pf->hw.hw_addr + QTX_COMM_DBELL(pf_q);
+
+ /* Add unique software queue handle of the Tx queue per
+ * TC into the VSI Tx ring
+ */
+ ring->q_handle = tc_q_idx;
+
+ status = ice_ena_vsi_txq(vsi->port_info, vsi->idx, tc, ring->q_handle,
+ 1, qg_buf, buf_len, NULL);
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Failed to set LAN Tx queue context, error: %d\n",
+ status);
+ return -ENODEV;
+ }
+
+ /* Add Tx Queue TEID into the VSI Tx ring from the
+ * response. This will complete configuring and
+ * enabling the queue.
+ */
+ txq = &qg_buf->txqs[0];
+ if (pf_q == le16_to_cpu(txq->txq_id))
+ ring->txq_teid = le32_to_cpu(txq->q_teid);
+
+ return 0;
+}
+
+/**
* ice_vsi_cfg_txqs - Configure the VSI for Tx
* @vsi: the VSI being configured
* @rings: Tx ring array to be configured
@@ -1654,20 +1769,16 @@ static int
ice_vsi_cfg_txqs(struct ice_vsi *vsi, struct ice_ring **rings, int offset)
{
struct ice_aqc_add_tx_qgrp *qg_buf;
- struct ice_aqc_add_txqs_perq *txq;
struct ice_pf *pf = vsi->back;
- u8 num_q_grps, q_idx = 0;
- enum ice_status status;
- u16 buf_len, i, pf_q;
- int err = 0, tc;
+ u16 q_idx = 0, i;
+ int err = 0;
+ u8 tc;
- buf_len = sizeof(*qg_buf);
- qg_buf = devm_kzalloc(&pf->pdev->dev, buf_len, GFP_KERNEL);
+ qg_buf = devm_kzalloc(&pf->pdev->dev, sizeof(*qg_buf), GFP_KERNEL);
if (!qg_buf)
return -ENOMEM;
qg_buf->num_txqs = 1;
- num_q_grps = 1;
/* set up and configure the Tx queues for each enabled TC */
ice_for_each_traffic_class(tc) {
@@ -1675,39 +1786,10 @@ ice_vsi_cfg_txqs(struct ice_vsi *vsi, struct ice_ring **rings, int offset)
break;
for (i = 0; i < vsi->tc_cfg.tc_info[tc].qcount_tx; i++) {
- struct ice_tlan_ctx tlan_ctx = { 0 };
-
- pf_q = vsi->txq_map[q_idx + offset];
- ice_setup_tx_ctx(rings[q_idx], &tlan_ctx, pf_q);
- /* copy context contents into the qg_buf */
- qg_buf->txqs[0].txq_id = cpu_to_le16(pf_q);
- ice_set_ctx((u8 *)&tlan_ctx, qg_buf->txqs[0].txq_ctx,
- ice_tlan_ctx_info);
-
- /* init queue specific tail reg. It is referred as
- * transmit comm scheduler queue doorbell.
- */
- rings[q_idx]->tail =
- pf->hw.hw_addr + QTX_COMM_DBELL(pf_q);
- status = ice_ena_vsi_txq(vsi->port_info, vsi->idx, tc,
- i, num_q_grps, qg_buf,
- buf_len, NULL);
- if (status) {
- dev_err(&pf->pdev->dev,
- "Failed to set LAN Tx queue context, error: %d\n",
- status);
- err = -ENODEV;
+ err = ice_vsi_cfg_txq(vsi, rings[q_idx], i + offset,
+ qg_buf, tc);
+ if (err)
goto err_cfg_txqs;
- }
-
- /* Add Tx Queue TEID into the VSI Tx ring from the
- * response. This will complete configuring and
- * enabling the queue.
- */
- txq = &qg_buf->txqs[0];
- if (pf_q == le16_to_cpu(txq->txq_id))
- rings[q_idx]->txq_teid =
- le32_to_cpu(txq->q_teid);
q_idx++;
}
@@ -2054,113 +2136,137 @@ void ice_trigger_sw_intr(struct ice_hw *hw, struct ice_q_vector *q_vector)
}
/**
+ * ice_vsi_stop_tx_ring - Disable single Tx ring
+ * @vsi: the VSI being configured
+ * @rst_src: reset source
+ * @rel_vmvf_num: Relative ID of VF/VM
+ * @ring: Tx ring to be stopped
+ * @txq_meta: Meta data of Tx ring to be stopped
+ */
+#ifndef CONFIG_PCI_IOV
+static
+#endif /* !CONFIG_PCI_IOV */
+int
+ice_vsi_stop_tx_ring(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
+ u16 rel_vmvf_num, struct ice_ring *ring,
+ struct ice_txq_meta *txq_meta)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_q_vector *q_vector;
+ struct ice_hw *hw = &pf->hw;
+ enum ice_status status;
+ u32 val;
+
+ /* clear cause_ena bit for disabled queues */
+ val = rd32(hw, QINT_TQCTL(ring->reg_idx));
+ val &= ~QINT_TQCTL_CAUSE_ENA_M;
+ wr32(hw, QINT_TQCTL(ring->reg_idx), val);
+
+ /* software is expected to wait for 100 ns */
+ ndelay(100);
+
+ /* trigger a software interrupt for the vector
+ * associated to the queue to schedule NAPI handler
+ */
+ q_vector = ring->q_vector;
+ if (q_vector)
+ ice_trigger_sw_intr(hw, q_vector);
+
+ status = ice_dis_vsi_txq(vsi->port_info, txq_meta->vsi_idx,
+ txq_meta->tc, 1, &txq_meta->q_handle,
+ &txq_meta->q_id, &txq_meta->q_teid, rst_src,
+ rel_vmvf_num, NULL);
+
+ /* if the disable queue command was exercised during an
+ * active reset flow, ICE_ERR_RESET_ONGOING is returned.
+ * This is not an error as the reset operation disables
+ * queues at the hardware level anyway.
+ */
+ if (status == ICE_ERR_RESET_ONGOING) {
+ dev_dbg(&vsi->back->pdev->dev,
+ "Reset in progress. LAN Tx queues already disabled\n");
+ } else if (status == ICE_ERR_DOES_NOT_EXIST) {
+ dev_dbg(&vsi->back->pdev->dev,
+ "LAN Tx queues do not exist, nothing to disable\n");
+ } else if (status) {
+ dev_err(&vsi->back->pdev->dev,
+ "Failed to disable LAN Tx queues, error: %d\n", status);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_fill_txq_meta - Prepare the Tx queue's meta data
+ * @vsi: VSI that ring belongs to
+ * @ring: ring that txq_meta will be based on
+ * @txq_meta: a helper struct that wraps Tx queue's information
+ *
+ * Set up a helper struct that will contain all the necessary fields that
+ * are needed for stopping Tx queue
+ */
+#ifndef CONFIG_PCI_IOV
+static
+#endif /* !CONFIG_PCI_IOV */
+void
+ice_fill_txq_meta(struct ice_vsi *vsi, struct ice_ring *ring,
+ struct ice_txq_meta *txq_meta)
+{
+ u8 tc = 0;
+
+#ifdef CONFIG_DCB
+ tc = ring->dcb_tc;
+#endif /* CONFIG_DCB */
+ txq_meta->q_id = ring->reg_idx;
+ txq_meta->q_teid = ring->txq_teid;
+ txq_meta->q_handle = ring->q_handle;
+ txq_meta->vsi_idx = vsi->idx;
+ txq_meta->tc = tc;
+}
+
+/**
* ice_vsi_stop_tx_rings - Disable Tx rings
* @vsi: the VSI being configured
* @rst_src: reset source
* @rel_vmvf_num: Relative ID of VF/VM
* @rings: Tx ring array to be stopped
- * @offset: offset within vsi->txq_map
*/
static int
ice_vsi_stop_tx_rings(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
- u16 rel_vmvf_num, struct ice_ring **rings, int offset)
+ u16 rel_vmvf_num, struct ice_ring **rings)
{
- struct ice_pf *pf = vsi->back;
- struct ice_hw *hw = &pf->hw;
- int tc, q_idx = 0, err = 0;
- u16 *q_ids, *q_handles, i;
- enum ice_status status;
- u32 *q_teids, val;
+ u16 i, q_idx = 0;
+ int status;
+ u8 tc;
if (vsi->num_txq > ICE_LAN_TXQ_MAX_QDIS)
return -EINVAL;
- q_teids = devm_kcalloc(&pf->pdev->dev, vsi->num_txq, sizeof(*q_teids),
- GFP_KERNEL);
- if (!q_teids)
- return -ENOMEM;
-
- q_ids = devm_kcalloc(&pf->pdev->dev, vsi->num_txq, sizeof(*q_ids),
- GFP_KERNEL);
- if (!q_ids) {
- err = -ENOMEM;
- goto err_alloc_q_ids;
- }
-
- q_handles = devm_kcalloc(&pf->pdev->dev, vsi->num_txq,
- sizeof(*q_handles), GFP_KERNEL);
- if (!q_handles) {
- err = -ENOMEM;
- goto err_alloc_q_handles;
- }
-
/* set up the Tx queue list to be disabled for each enabled TC */
ice_for_each_traffic_class(tc) {
if (!(vsi->tc_cfg.ena_tc & BIT(tc)))
break;
for (i = 0; i < vsi->tc_cfg.tc_info[tc].qcount_tx; i++) {
- struct ice_q_vector *q_vector;
+ struct ice_txq_meta txq_meta = { };
- if (!rings || !rings[q_idx]) {
- err = -EINVAL;
- goto err_out;
- }
+ if (!rings || !rings[q_idx])
+ return -EINVAL;
- q_ids[i] = vsi->txq_map[q_idx + offset];
- q_teids[i] = rings[q_idx]->txq_teid;
- q_handles[i] = i;
+ ice_fill_txq_meta(vsi, rings[q_idx], &txq_meta);
+ status = ice_vsi_stop_tx_ring(vsi, rst_src,
+ rel_vmvf_num,
+ rings[q_idx], &txq_meta);
- /* clear cause_ena bit for disabled queues */
- val = rd32(hw, QINT_TQCTL(rings[i]->reg_idx));
- val &= ~QINT_TQCTL_CAUSE_ENA_M;
- wr32(hw, QINT_TQCTL(rings[i]->reg_idx), val);
-
- /* software is expected to wait for 100 ns */
- ndelay(100);
-
- /* trigger a software interrupt for the vector
- * associated to the queue to schedule NAPI handler
- */
- q_vector = rings[i]->q_vector;
- if (q_vector)
- ice_trigger_sw_intr(hw, q_vector);
+ if (status)
+ return status;
q_idx++;
}
- status = ice_dis_vsi_txq(vsi->port_info, vsi->idx, tc,
- vsi->num_txq, q_handles, q_ids,
- q_teids, rst_src, rel_vmvf_num, NULL);
-
- /* if the disable queue command was exercised during an active
- * reset flow, ICE_ERR_RESET_ONGOING is returned. This is not
- * an error as the reset operation disables queues at the
- * hardware level anyway.
- */
- if (status == ICE_ERR_RESET_ONGOING) {
- dev_dbg(&pf->pdev->dev,
- "Reset in progress. LAN Tx queues already disabled\n");
- } else if (status == ICE_ERR_DOES_NOT_EXIST) {
- dev_dbg(&pf->pdev->dev,
- "LAN Tx queues does not exist, nothing to disabled\n");
- } else if (status) {
- dev_err(&pf->pdev->dev,
- "Failed to disable LAN Tx queues, error: %d\n",
- status);
- err = -ENODEV;
- }
}
-err_out:
- devm_kfree(&pf->pdev->dev, q_handles);
-
-err_alloc_q_handles:
- devm_kfree(&pf->pdev->dev, q_ids);
-
-err_alloc_q_ids:
- devm_kfree(&pf->pdev->dev, q_teids);
-
- return err;
+ return 0;
}
/**
@@ -2173,8 +2279,7 @@ int
ice_vsi_stop_lan_tx_rings(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
u16 rel_vmvf_num)
{
- return ice_vsi_stop_tx_rings(vsi, rst_src, rel_vmvf_num, vsi->tx_rings,
- 0);
+ return ice_vsi_stop_tx_rings(vsi, rst_src, rel_vmvf_num, vsi->tx_rings);
}
/**
@@ -2484,9 +2589,6 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,
if (ret)
goto unroll_vector_base;
- pf->q_left_tx -= vsi->alloc_txq;
- pf->q_left_rx -= vsi->alloc_rxq;
-
/* Do not exit if configuring RSS had an issue, at least
* receive traffic on first queue. Hence no need to capture
* return value
@@ -2534,7 +2636,7 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,
ice_cfg_sw_lldp(vsi, true, true);
/* Rx LLDP packets */
- if (!test_bit(ICE_FLAG_ENABLE_FW_LLDP, pf->flags))
+ if (!test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags))
ice_cfg_sw_lldp(vsi, false, true);
}
@@ -2550,8 +2652,6 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,
ice_vsi_delete(vsi);
unroll_get_qs:
ice_vsi_put_qs(vsi);
- pf->q_left_tx += vsi->alloc_txq;
- pf->q_left_rx += vsi->alloc_rxq;
ice_vsi_clear(vsi);
return NULL;
@@ -2810,6 +2910,10 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)
ice_flush(hw);
+ /* don't call synchronize_irq() for VF's from the host */
+ if (vsi->type == ICE_VSI_VF)
+ return;
+
ice_for_each_q_vector(vsi, i)
synchronize_irq(pf->msix_entries[i + base].vector);
}
@@ -2877,7 +2981,7 @@ int ice_vsi_release(struct ice_vsi *vsi)
/* The Rx rule will only exist to remove if the LLDP FW
* engine is currently stopped
*/
- if (!test_bit(ICE_FLAG_ENABLE_FW_LLDP, pf->flags))
+ if (!test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags))
ice_cfg_sw_lldp(vsi, false, false);
}
@@ -2895,8 +2999,6 @@ int ice_vsi_release(struct ice_vsi *vsi)
ice_vsi_clear_rings(vsi);
ice_vsi_put_qs(vsi);
- pf->q_left_tx += vsi->alloc_txq;
- pf->q_left_rx += vsi->alloc_rxq;
/* retain SW VSI data structure since it is needed to unregister and
* free VSI netdev when PF is not in reset recovery pending state,\
@@ -2944,6 +3046,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)
vsi->base_vector = 0;
}
+ ice_vsi_put_qs(vsi);
ice_vsi_clear_rings(vsi);
ice_vsi_free_arrays(vsi);
ice_dev_onetime_setup(&pf->hw);
@@ -2951,6 +3054,12 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)
ice_vsi_set_num_qs(vsi, vf->vf_id);
else
ice_vsi_set_num_qs(vsi, ICE_INVAL_VFID);
+
+ ret = ice_vsi_alloc_arrays(vsi);
+ if (ret < 0)
+ goto err_vsi;
+
+ ice_vsi_get_qs(vsi);
ice_vsi_set_tc_cfg(vsi);
/* Initialize VSI struct elements and create VSI in FW */
@@ -2958,9 +3067,6 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)
if (ret < 0)
goto err_vsi;
- ret = ice_vsi_alloc_arrays(vsi);
- if (ret < 0)
- goto err_vsi;
switch (vsi->type) {
case ICE_VSI_PF:
@@ -3001,8 +3107,6 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)
if (ret)
goto err_vectors;
- pf->q_left_tx -= vsi->alloc_txq;
- pf->q_left_rx -= vsi->alloc_rxq;
break;
default:
break;
@@ -3170,3 +3274,33 @@ int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc)
return ret;
}
#endif /* CONFIG_DCB */
+
+/**
+ * ice_vsi_cfg_mac_fltr - Add or remove a MAC address filter for a VSI
+ * @vsi: the VSI being configured MAC filter
+ * @macaddr: the MAC address to be added.
+ * @set: Add or delete a MAC filter
+ *
+ * Adds or removes MAC address filter entry for VF VSI
+ */
+enum ice_status
+ice_vsi_cfg_mac_fltr(struct ice_vsi *vsi, const u8 *macaddr, bool set)
+{
+ LIST_HEAD(tmp_add_list);
+ enum ice_status status;
+
+ /* Update MAC filter list to be added or removed for a VSI */
+ if (ice_add_mac_to_list(vsi, &tmp_add_list, macaddr)) {
+ status = ICE_ERR_NO_MEMORY;
+ goto cfg_mac_fltr_exit;
+ }
+
+ if (set)
+ status = ice_add_mac(&vsi->back->hw, &tmp_add_list);
+ else
+ status = ice_remove_mac(&vsi->back->hw, &tmp_add_list);
+
+cfg_mac_fltr_exit:
+ ice_free_fltr_list(&vsi->back->pdev->dev, &tmp_add_list);
+ return status;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 6e43ef0..7faf8db 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -6,8 +6,22 @@
#include "ice.h"
-int ice_add_mac_to_list(struct ice_vsi *vsi, struct list_head *add_list,
- const u8 *macaddr);
+struct ice_txq_meta {
+ /* Tx-scheduler element identifier */
+ u32 q_teid;
+ /* Entry in VSI's txq_map bitmap */
+ u16 q_id;
+ /* Relative index of Tx queue within TC */
+ u16 q_handle;
+ /* VSI index that Tx queue belongs to */
+ u16 vsi_idx;
+ /* TC number that Tx queue belongs to */
+ u8 tc;
+};
+
+int
+ice_add_mac_to_list(struct ice_vsi *vsi, struct list_head *add_list,
+ const u8 *macaddr);
void ice_free_fltr_list(struct device *dev, struct list_head *h);
@@ -25,6 +39,16 @@ ice_cfg_txq_interrupt(struct ice_vsi *vsi, u16 txq, u16 msix_idx, u16 itr_idx);
void
ice_cfg_rxq_interrupt(struct ice_vsi *vsi, u16 rxq, u16 msix_idx, u16 itr_idx);
+
+int
+ice_vsi_stop_tx_ring(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
+ u16 rel_vmvf_num, struct ice_ring *ring,
+ struct ice_txq_meta *txq_meta);
+
+void ice_fill_txq_meta(struct ice_vsi *vsi, struct ice_ring *ring,
+ struct ice_txq_meta *txq_meta);
+
+int ice_vsi_ctrl_rx_ring(struct ice_vsi *vsi, bool ena, u16 rxq_idx);
#endif /* CONFIG_PCI_IOV */
int ice_vsi_add_vlan(struct ice_vsi *vsi, u16 vid);
@@ -95,4 +119,8 @@ void ice_vsi_free_tx_rings(struct ice_vsi *vsi);
int ice_vsi_manage_rss_lut(struct ice_vsi *vsi, bool ena);
u32 ice_intrl_usec_to_reg(u8 intrl, u8 gran);
+
+enum ice_status
+ice_vsi_cfg_mac_fltr(struct ice_vsi *vsi, const u8 *macaddr, bool set);
+
#endif /* !_ICE_LIB_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index a0d148f..f8be9ad 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -34,8 +34,6 @@ static const struct net_device_ops ice_netdev_ops;
static void ice_rebuild(struct ice_pf *pf);
static void ice_vsi_release_all(struct ice_pf *pf);
-static void ice_update_vsi_stats(struct ice_vsi *vsi);
-static void ice_update_pf_stats(struct ice_pf *pf);
/**
* ice_get_tx_pending - returns number of Tx descriptors not processed
@@ -118,12 +116,11 @@ static void ice_check_for_hang_subtask(struct ice_pf *pf)
*/
static int ice_init_mac_fltr(struct ice_pf *pf)
{
- LIST_HEAD(tmp_add_list);
+ enum ice_status status;
u8 broadcast[ETH_ALEN];
struct ice_vsi *vsi;
- int status;
- vsi = ice_find_vsi_by_type(pf, ICE_VSI_PF);
+ vsi = ice_get_main_vsi(pf);
if (!vsi)
return -EINVAL;
@@ -132,8 +129,7 @@ static int ice_init_mac_fltr(struct ice_pf *pf)
*/
/* Add a unicast MAC filter so the VSI can get its packets */
- status = ice_add_mac_to_list(vsi, &tmp_add_list,
- vsi->port_info->mac.perm_addr);
+ status = ice_vsi_cfg_mac_fltr(vsi, vsi->port_info->mac.perm_addr, true);
if (status)
goto unregister;
@@ -141,18 +137,11 @@ static int ice_init_mac_fltr(struct ice_pf *pf)
* MAC address to the list as well.
*/
eth_broadcast_addr(broadcast);
- status = ice_add_mac_to_list(vsi, &tmp_add_list, broadcast);
+ status = ice_vsi_cfg_mac_fltr(vsi, broadcast, true);
if (status)
- goto free_mac_list;
+ goto unregister;
- /* Program MAC filters for entries in tmp_add_list */
- status = ice_add_mac(&pf->hw, &tmp_add_list);
- if (status)
- status = -ENOMEM;
-
-free_mac_list:
- ice_free_fltr_list(&pf->pdev->dev, &tmp_add_list);
-
+ return 0;
unregister:
/* We aren't useful with no MAC filters, so unregister if we
* had an error
@@ -166,7 +155,7 @@ static int ice_init_mac_fltr(struct ice_pf *pf)
vsi->netdev = NULL;
}
- return status;
+ return -EIO;
}
/**
@@ -447,13 +436,13 @@ static void ice_dis_vsi(struct ice_vsi *vsi, bool locked)
if (vsi->type == ICE_VSI_PF && vsi->netdev) {
if (netif_running(vsi->netdev)) {
- if (!locked) {
+ if (!locked)
rtnl_lock();
- vsi->netdev->netdev_ops->ndo_stop(vsi->netdev);
+
+ ice_stop(vsi->netdev);
+
+ if (!locked)
rtnl_unlock();
- } else {
- vsi->netdev->netdev_ops->ndo_stop(vsi->netdev);
- }
} else {
ice_vsi_close(vsi);
}
@@ -488,7 +477,7 @@ static void
ice_prepare_for_reset(struct ice_pf *pf)
{
struct ice_hw *hw = &pf->hw;
- u8 i;
+ int i;
/* already prepared for reset */
if (test_bit(__ICE_PREPARED_FOR_RESET, pf->state))
@@ -500,7 +489,7 @@ ice_prepare_for_reset(struct ice_pf *pf)
/* Disable VFs until reset is completed */
for (i = 0; i < pf->num_alloc_vfs; i++)
- clear_bit(ICE_VF_STATE_ENA, pf->vf[i].vf_states);
+ ice_set_vf_state_qs_dis(&pf->vf[i]);
/* disable the VSIs and their queues that are not already DOWN */
ice_pf_dis_all_vsi(pf, false);
@@ -578,6 +567,8 @@ static void ice_reset_subtask(struct ice_pf *pf)
reset_type = ICE_RESET_CORER;
if (test_and_clear_bit(__ICE_GLOBR_RECV, pf->state))
reset_type = ICE_RESET_GLOBR;
+ if (test_and_clear_bit(__ICE_EMPR_RECV, pf->state))
+ reset_type = ICE_RESET_EMPR;
/* return if no valid reset type requested */
if (reset_type == ICE_RESET_INVAL)
return;
@@ -623,6 +614,22 @@ static void ice_reset_subtask(struct ice_pf *pf)
}
/**
+ * ice_print_topo_conflict - print topology conflict message
+ * @vsi: the VSI whose topology status is being checked
+ */
+static void ice_print_topo_conflict(struct ice_vsi *vsi)
+{
+ switch (vsi->port_info->phy.link_info.topo_media_conflict) {
+ case ICE_AQ_LINK_TOPO_CONFLICT:
+ case ICE_AQ_LINK_MEDIA_CONFLICT:
+ netdev_info(vsi->netdev, "Possible mis-configuration of the Ethernet port detected, please use the Intel(R) Ethernet Port Configuration Tool application to address the issue.\n");
+ break;
+ default:
+ break;
+ }
+}
+
+/**
* ice_print_link_msg - print link up or down message
* @vsi: the VSI whose link status is being queried
* @isup: boolean for if the link is now up or down
@@ -635,6 +642,7 @@ void ice_print_link_msg(struct ice_vsi *vsi, bool isup)
const char *speed;
const char *fec;
const char *fc;
+ const char *an;
if (!vsi)
return;
@@ -718,6 +726,12 @@ void ice_print_link_msg(struct ice_vsi *vsi, bool isup)
break;
}
+ /* check if autoneg completed, might be false due to not supported */
+ if (vsi->port_info->phy.link_info.an_info & ICE_AQ_AN_COMPLETED)
+ an = "True";
+ else
+ an = "False";
+
/* Get FEC mode requested based on PHY caps last SW configuration */
caps = devm_kzalloc(&vsi->back->pdev->dev, sizeof(*caps), GFP_KERNEL);
if (!caps) {
@@ -742,8 +756,9 @@ void ice_print_link_msg(struct ice_vsi *vsi, bool isup)
devm_kfree(&vsi->back->pdev->dev, caps);
done:
- netdev_info(vsi->netdev, "NIC Link is up %sbps, Requested FEC: %s, FEC: %s, Flow Control: %s\n",
- speed, fec_req, fec, fc);
+ netdev_info(vsi->netdev, "NIC Link is up %sbps, Requested FEC: %s, FEC: %s, Autoneg: %s, Flow Control: %s\n",
+ speed, fec_req, fec, an, fc);
+ ice_print_topo_conflict(vsi);
}
/**
@@ -811,7 +826,7 @@ ice_link_event(struct ice_pf *pf, struct ice_port_info *pi, bool link_up,
if (link_up == old_link && link_speed == old_link_speed)
return result;
- vsi = ice_find_vsi_by_type(pf, ICE_VSI_PF);
+ vsi = ice_get_main_vsi(pf);
if (!vsi || !vsi->port_info)
return -EINVAL;
@@ -1326,8 +1341,10 @@ static void ice_handle_mdd_event(struct ice_pf *pf)
if (vf_mdd_detected) {
vf->num_mdd_events++;
- if (vf->num_mdd_events > 1)
- dev_info(&pf->pdev->dev, "VF %d has had %llu MDD events since last boot\n",
+ if (vf->num_mdd_events &&
+ vf->num_mdd_events <= ICE_MDD_EVENTS_THRESHOLD)
+ dev_info(&pf->pdev->dev,
+ "VF %d has had %llu MDD events since last boot, Admin might need to reload AVF driver with this number of events\n",
i, vf->num_mdd_events);
}
}
@@ -1422,7 +1439,7 @@ static void ice_check_media_subtask(struct ice_pf *pf)
struct ice_vsi *vsi;
int err;
- vsi = ice_find_vsi_by_type(pf, ICE_VSI_PF);
+ vsi = ice_get_main_vsi(pf);
if (!vsi)
return;
@@ -2175,36 +2192,48 @@ static int ice_setup_pf_sw(struct ice_pf *pf)
ice_vsi_free_q_vectors(vsi);
ice_vsi_delete(vsi);
ice_vsi_put_qs(vsi);
- pf->q_left_tx += vsi->alloc_txq;
- pf->q_left_rx += vsi->alloc_rxq;
ice_vsi_clear(vsi);
}
return status;
}
/**
- * ice_determine_q_usage - Calculate queue distribution
- * @pf: board private structure
- *
- * Return -ENOMEM if we don't get enough queues for all ports
+ * ice_get_avail_q_count - Get count of queues in use
+ * @pf_qmap: bitmap to get queue use count from
+ * @lock: pointer to a mutex that protects access to pf_qmap
+ * @size: size of the bitmap
*/
-static void ice_determine_q_usage(struct ice_pf *pf)
+static u16
+ice_get_avail_q_count(unsigned long *pf_qmap, struct mutex *lock, u16 size)
{
- u16 q_left_tx, q_left_rx;
+ u16 count = 0, bit;
- q_left_tx = pf->hw.func_caps.common_cap.num_txq;
- q_left_rx = pf->hw.func_caps.common_cap.num_rxq;
+ mutex_lock(lock);
+ for_each_clear_bit(bit, pf_qmap, size)
+ count++;
+ mutex_unlock(lock);
- pf->num_lan_tx = min_t(int, q_left_tx, num_online_cpus());
+ return count;
+}
- /* only 1 Rx queue unless RSS is enabled */
- if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags))
- pf->num_lan_rx = 1;
- else
- pf->num_lan_rx = min_t(int, q_left_rx, num_online_cpus());
+/**
+ * ice_get_avail_txq_count - Get count of Tx queues in use
+ * @pf: pointer to an ice_pf instance
+ */
+u16 ice_get_avail_txq_count(struct ice_pf *pf)
+{
+ return ice_get_avail_q_count(pf->avail_txqs, &pf->avail_q_mutex,
+ pf->max_pf_txqs);
+}
- pf->q_left_tx = q_left_tx - pf->num_lan_tx;
- pf->q_left_rx = q_left_rx - pf->num_lan_rx;
+/**
+ * ice_get_avail_rxq_count - Get count of Rx queues in use
+ * @pf: pointer to an ice_pf instance
+ */
+u16 ice_get_avail_rxq_count(struct ice_pf *pf)
+{
+ return ice_get_avail_q_count(pf->avail_rxqs, &pf->avail_q_mutex,
+ pf->max_pf_rxqs);
}
/**
@@ -2216,15 +2245,27 @@ static void ice_deinit_pf(struct ice_pf *pf)
ice_service_task_stop(pf);
mutex_destroy(&pf->sw_mutex);
mutex_destroy(&pf->avail_q_mutex);
+
+ if (pf->avail_txqs) {
+ bitmap_free(pf->avail_txqs);
+ pf->avail_txqs = NULL;
+ }
+
+ if (pf->avail_rxqs) {
+ bitmap_free(pf->avail_rxqs);
+ pf->avail_rxqs = NULL;
+ }
}
/**
* ice_init_pf - Initialize general software structures (struct ice_pf)
* @pf: board private structure to initialize
*/
-static void ice_init_pf(struct ice_pf *pf)
+static int ice_init_pf(struct ice_pf *pf)
{
bitmap_zero(pf->flags, ICE_PF_FLAGS_NBITS);
+ if (pf->hw.func_caps.common_cap.dcb)
+ set_bit(ICE_FLAG_DCB_CAPABLE, pf->flags);
#ifdef CONFIG_PCI_IOV
if (pf->hw.func_caps.common_cap.sr_iov_1_1) {
struct ice_hw *hw = &pf->hw;
@@ -2238,12 +2279,6 @@ static void ice_init_pf(struct ice_pf *pf)
mutex_init(&pf->sw_mutex);
mutex_init(&pf->avail_q_mutex);
- /* Clear avail_[t|r]x_qs bitmaps (set all to avail) */
- mutex_lock(&pf->avail_q_mutex);
- bitmap_zero(pf->avail_txqs, ICE_MAX_TXQS);
- bitmap_zero(pf->avail_rxqs, ICE_MAX_RXQS);
- mutex_unlock(&pf->avail_q_mutex);
-
if (pf->hw.func_caps.common_cap.rss_table_size)
set_bit(ICE_FLAG_RSS_ENA, pf->flags);
@@ -2252,6 +2287,22 @@ static void ice_init_pf(struct ice_pf *pf)
pf->serv_tmr_period = HZ;
INIT_WORK(&pf->serv_task, ice_service_task);
clear_bit(__ICE_SERVICE_SCHED, pf->state);
+
+ pf->max_pf_txqs = pf->hw.func_caps.common_cap.num_txq;
+ pf->max_pf_rxqs = pf->hw.func_caps.common_cap.num_rxq;
+
+ pf->avail_txqs = bitmap_zalloc(pf->max_pf_txqs, GFP_KERNEL);
+ if (!pf->avail_txqs)
+ return -ENOMEM;
+
+ pf->avail_rxqs = bitmap_zalloc(pf->max_pf_rxqs, GFP_KERNEL);
+ if (!pf->avail_rxqs) {
+ devm_kfree(&pf->pdev->dev, pf->avail_txqs);
+ pf->avail_txqs = NULL;
+ return -ENOMEM;
+ }
+
+ return 0;
}
/**
@@ -2270,13 +2321,18 @@ static int ice_ena_msix_range(struct ice_pf *pf)
/* reserve one vector for miscellaneous handler */
needed = 1;
+ if (v_left < needed)
+ goto no_hw_vecs_left_err;
v_budget += needed;
v_left -= needed;
/* reserve vectors for LAN traffic */
- pf->num_lan_msix = min_t(int, num_online_cpus(), v_left);
- v_budget += pf->num_lan_msix;
- v_left -= pf->num_lan_msix;
+ needed = min_t(int, num_online_cpus(), v_left);
+ if (v_left < needed)
+ goto no_hw_vecs_left_err;
+ pf->num_lan_msix = needed;
+ v_budget += needed;
+ v_left -= needed;
pf->msix_entries = devm_kcalloc(&pf->pdev->dev, v_budget,
sizeof(*pf->msix_entries), GFP_KERNEL);
@@ -2301,18 +2357,18 @@ static int ice_ena_msix_range(struct ice_pf *pf)
if (v_actual < v_budget) {
dev_warn(&pf->pdev->dev,
- "not enough vectors. requested = %d, obtained = %d\n",
+ "not enough OS MSI-X vectors. requested = %d, obtained = %d\n",
v_budget, v_actual);
- if (v_actual >= (pf->num_lan_msix + 1)) {
- pf->num_avail_sw_msix = v_actual -
- (pf->num_lan_msix + 1);
- } else if (v_actual >= 2) {
- pf->num_lan_msix = 1;
- pf->num_avail_sw_msix = v_actual - 2;
- } else {
+/* 2 vectors for LAN (traffic + OICR) */
+#define ICE_MIN_LAN_VECS 2
+
+ if (v_actual < ICE_MIN_LAN_VECS) {
+ /* error if we can't get minimum vectors */
pci_disable_msix(pf->pdev);
err = -ERANGE;
goto msix_err;
+ } else {
+ pf->num_lan_msix = ICE_MIN_LAN_VECS;
}
}
@@ -2322,6 +2378,11 @@ static int ice_ena_msix_range(struct ice_pf *pf)
devm_kfree(&pf->pdev->dev, pf->msix_entries);
goto exit_err;
+no_hw_vecs_left_err:
+ dev_err(&pf->pdev->dev,
+ "not enough device MSI-X vectors. requested = %d, available = %d\n",
+ needed, v_left);
+ err = -ERANGE;
exit_err:
pf->num_lan_msix = 0;
return err;
@@ -2476,18 +2537,21 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
hw->fw_maj_ver, hw->fw_min_ver, hw->fw_build,
hw->api_maj_ver, hw->api_min_ver);
- ice_init_pf(pf);
-
- err = ice_init_pf_dcb(pf, false);
+ err = ice_init_pf(pf);
if (err) {
- clear_bit(ICE_FLAG_DCB_CAPABLE, pf->flags);
- clear_bit(ICE_FLAG_DCB_ENA, pf->flags);
-
- /* do not fail overall init if DCB init fails */
- err = 0;
+ dev_err(dev, "ice_init_pf failed: %d\n", err);
+ goto err_init_pf_unroll;
}
- ice_determine_q_usage(pf);
+ if (test_bit(ICE_FLAG_DCB_CAPABLE, pf->flags)) {
+ /* Note: DCB init failure is non-fatal to load */
+ if (ice_init_pf_dcb(pf, false)) {
+ clear_bit(ICE_FLAG_DCB_CAPABLE, pf->flags);
+ clear_bit(ICE_FLAG_DCB_ENA, pf->flags);
+ } else {
+ ice_cfg_lldp_mib_change(&pf->hw, true);
+ }
+ }
pf->num_alloc_vsi = hw->func_caps.guar_num_vsi;
if (!pf->num_alloc_vsi) {
@@ -2608,9 +2672,14 @@ static void ice_remove(struct pci_dev *pdev)
continue;
ice_vsi_free_q_vectors(pf->vsi[i]);
}
- ice_clear_interrupt_scheme(pf);
ice_deinit_pf(pf);
ice_deinit_hw(&pf->hw);
+ ice_clear_interrupt_scheme(pf);
+ /* Issue a PFR as part of the prescribed driver unload flow. Do not
+ * do it via ice_schedule_reset() since there is no need to rebuild
+ * and the service task is already stopped.
+ */
+ ice_reset(&pf->hw, ICE_RESET_PFR);
pci_disable_pcie_error_reporting(pdev);
}
@@ -2836,10 +2905,8 @@ static int ice_set_mac_address(struct net_device *netdev, void *pi)
struct ice_hw *hw = &pf->hw;
struct sockaddr *addr = pi;
enum ice_status status;
- LIST_HEAD(a_mac_list);
- LIST_HEAD(r_mac_list);
u8 flags = 0;
- int err;
+ int err = 0;
u8 *mac;
mac = (u8 *)addr->sa_data;
@@ -2862,42 +2929,23 @@ static int ice_set_mac_address(struct net_device *netdev, void *pi)
/* When we change the MAC address we also have to change the MAC address
* based filter rules that were created previously for the old MAC
* address. So first, we remove the old filter rule using ice_remove_mac
- * and then create a new filter rule using ice_add_mac. Note that for
- * both these operations, we first need to form a "list" of MAC
- * addresses (even though in this case, we have only 1 MAC address to be
- * added/removed) and this done using ice_add_mac_to_list. Depending on
- * the ensuing operation this "list" of MAC addresses is either to be
- * added or removed from the filter.
+ * and then create a new filter rule using ice_add_mac via
+ * ice_vsi_cfg_mac_fltr function call for both add and/or remove
+ * filters.
*/
- err = ice_add_mac_to_list(vsi, &r_mac_list, netdev->dev_addr);
- if (err) {
- err = -EADDRNOTAVAIL;
- goto free_lists;
- }
-
- status = ice_remove_mac(hw, &r_mac_list);
+ status = ice_vsi_cfg_mac_fltr(vsi, netdev->dev_addr, false);
if (status) {
err = -EADDRNOTAVAIL;
- goto free_lists;
+ goto err_update_filters;
}
- err = ice_add_mac_to_list(vsi, &a_mac_list, mac);
- if (err) {
- err = -EADDRNOTAVAIL;
- goto free_lists;
- }
-
- status = ice_add_mac(hw, &a_mac_list);
+ status = ice_vsi_cfg_mac_fltr(vsi, mac, true);
if (status) {
err = -EADDRNOTAVAIL;
- goto free_lists;
+ goto err_update_filters;
}
-free_lists:
- /* free list entries */
- ice_free_fltr_list(&pf->pdev->dev, &r_mac_list);
- ice_free_fltr_list(&pf->pdev->dev, &a_mac_list);
-
+err_update_filters:
if (err) {
netdev_err(netdev, "can't set MAC %pM. filter update failed\n",
mac);
@@ -2913,8 +2961,8 @@ static int ice_set_mac_address(struct net_device *netdev, void *pi)
flags = ICE_AQC_MAN_MAC_UPDATE_LAA_WOL;
status = ice_aq_manage_mac_write(hw, mac, flags, NULL);
if (status) {
- netdev_err(netdev, "can't set MAC %pM. write to firmware failed.\n",
- mac);
+ netdev_err(netdev, "can't set MAC %pM. write to firmware failed error %d\n",
+ mac, status);
}
return 0;
}
@@ -3254,7 +3302,7 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)
* ice_update_vsi_stats - Update VSI stats counters
* @vsi: the VSI to be updated
*/
-static void ice_update_vsi_stats(struct ice_vsi *vsi)
+void ice_update_vsi_stats(struct ice_vsi *vsi)
{
struct rtnl_link_stats64 *cur_ns = &vsi->net_stats;
struct ice_eth_stats *cur_es = &vsi->eth_stats;
@@ -3290,29 +3338,29 @@ static void ice_update_vsi_stats(struct ice_vsi *vsi)
* ice_update_pf_stats - Update PF port stats counters
* @pf: PF whose stats needs to be updated
*/
-static void ice_update_pf_stats(struct ice_pf *pf)
+void ice_update_pf_stats(struct ice_pf *pf)
{
struct ice_hw_port_stats *prev_ps, *cur_ps;
struct ice_hw *hw = &pf->hw;
- u8 pf_id;
+ u8 port;
+ port = hw->port_info->lport;
prev_ps = &pf->stats_prev;
cur_ps = &pf->stats;
- pf_id = hw->pf_id;
- ice_stat_update40(hw, GLPRT_GORCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_GORCL(port), pf->stat_prev_loaded,
&prev_ps->eth.rx_bytes,
&cur_ps->eth.rx_bytes);
- ice_stat_update40(hw, GLPRT_UPRCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_UPRCL(port), pf->stat_prev_loaded,
&prev_ps->eth.rx_unicast,
&cur_ps->eth.rx_unicast);
- ice_stat_update40(hw, GLPRT_MPRCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_MPRCL(port), pf->stat_prev_loaded,
&prev_ps->eth.rx_multicast,
&cur_ps->eth.rx_multicast);
- ice_stat_update40(hw, GLPRT_BPRCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_BPRCL(port), pf->stat_prev_loaded,
&prev_ps->eth.rx_broadcast,
&cur_ps->eth.rx_broadcast);
@@ -3320,109 +3368,109 @@ static void ice_update_pf_stats(struct ice_pf *pf)
&prev_ps->eth.rx_discards,
&cur_ps->eth.rx_discards);
- ice_stat_update40(hw, GLPRT_GOTCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_GOTCL(port), pf->stat_prev_loaded,
&prev_ps->eth.tx_bytes,
&cur_ps->eth.tx_bytes);
- ice_stat_update40(hw, GLPRT_UPTCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_UPTCL(port), pf->stat_prev_loaded,
&prev_ps->eth.tx_unicast,
&cur_ps->eth.tx_unicast);
- ice_stat_update40(hw, GLPRT_MPTCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_MPTCL(port), pf->stat_prev_loaded,
&prev_ps->eth.tx_multicast,
&cur_ps->eth.tx_multicast);
- ice_stat_update40(hw, GLPRT_BPTCL(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_BPTCL(port), pf->stat_prev_loaded,
&prev_ps->eth.tx_broadcast,
&cur_ps->eth.tx_broadcast);
- ice_stat_update32(hw, GLPRT_TDOLD(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_TDOLD(port), pf->stat_prev_loaded,
&prev_ps->tx_dropped_link_down,
&cur_ps->tx_dropped_link_down);
- ice_stat_update40(hw, GLPRT_PRC64L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC64L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_64, &cur_ps->rx_size_64);
- ice_stat_update40(hw, GLPRT_PRC127L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC127L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_127, &cur_ps->rx_size_127);
- ice_stat_update40(hw, GLPRT_PRC255L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC255L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_255, &cur_ps->rx_size_255);
- ice_stat_update40(hw, GLPRT_PRC511L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC511L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_511, &cur_ps->rx_size_511);
- ice_stat_update40(hw, GLPRT_PRC1023L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC1023L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_1023, &cur_ps->rx_size_1023);
- ice_stat_update40(hw, GLPRT_PRC1522L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC1522L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_1522, &cur_ps->rx_size_1522);
- ice_stat_update40(hw, GLPRT_PRC9522L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PRC9522L(port), pf->stat_prev_loaded,
&prev_ps->rx_size_big, &cur_ps->rx_size_big);
- ice_stat_update40(hw, GLPRT_PTC64L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC64L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_64, &cur_ps->tx_size_64);
- ice_stat_update40(hw, GLPRT_PTC127L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC127L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_127, &cur_ps->tx_size_127);
- ice_stat_update40(hw, GLPRT_PTC255L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC255L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_255, &cur_ps->tx_size_255);
- ice_stat_update40(hw, GLPRT_PTC511L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC511L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_511, &cur_ps->tx_size_511);
- ice_stat_update40(hw, GLPRT_PTC1023L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC1023L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_1023, &cur_ps->tx_size_1023);
- ice_stat_update40(hw, GLPRT_PTC1522L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC1522L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_1522, &cur_ps->tx_size_1522);
- ice_stat_update40(hw, GLPRT_PTC9522L(pf_id), pf->stat_prev_loaded,
+ ice_stat_update40(hw, GLPRT_PTC9522L(port), pf->stat_prev_loaded,
&prev_ps->tx_size_big, &cur_ps->tx_size_big);
- ice_stat_update32(hw, GLPRT_LXONRXC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_LXONRXC(port), pf->stat_prev_loaded,
&prev_ps->link_xon_rx, &cur_ps->link_xon_rx);
- ice_stat_update32(hw, GLPRT_LXOFFRXC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_LXOFFRXC(port), pf->stat_prev_loaded,
&prev_ps->link_xoff_rx, &cur_ps->link_xoff_rx);
- ice_stat_update32(hw, GLPRT_LXONTXC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_LXONTXC(port), pf->stat_prev_loaded,
&prev_ps->link_xon_tx, &cur_ps->link_xon_tx);
- ice_stat_update32(hw, GLPRT_LXOFFTXC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_LXOFFTXC(port), pf->stat_prev_loaded,
&prev_ps->link_xoff_tx, &cur_ps->link_xoff_tx);
ice_update_dcb_stats(pf);
- ice_stat_update32(hw, GLPRT_CRCERRS(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_CRCERRS(port), pf->stat_prev_loaded,
&prev_ps->crc_errors, &cur_ps->crc_errors);
- ice_stat_update32(hw, GLPRT_ILLERRC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_ILLERRC(port), pf->stat_prev_loaded,
&prev_ps->illegal_bytes, &cur_ps->illegal_bytes);
- ice_stat_update32(hw, GLPRT_MLFC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_MLFC(port), pf->stat_prev_loaded,
&prev_ps->mac_local_faults,
&cur_ps->mac_local_faults);
- ice_stat_update32(hw, GLPRT_MRFC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_MRFC(port), pf->stat_prev_loaded,
&prev_ps->mac_remote_faults,
&cur_ps->mac_remote_faults);
- ice_stat_update32(hw, GLPRT_RLEC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_RLEC(port), pf->stat_prev_loaded,
&prev_ps->rx_len_errors, &cur_ps->rx_len_errors);
- ice_stat_update32(hw, GLPRT_RUC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_RUC(port), pf->stat_prev_loaded,
&prev_ps->rx_undersize, &cur_ps->rx_undersize);
- ice_stat_update32(hw, GLPRT_RFC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_RFC(port), pf->stat_prev_loaded,
&prev_ps->rx_fragments, &cur_ps->rx_fragments);
- ice_stat_update32(hw, GLPRT_ROC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_ROC(port), pf->stat_prev_loaded,
&prev_ps->rx_oversize, &cur_ps->rx_oversize);
- ice_stat_update32(hw, GLPRT_RJC(pf_id), pf->stat_prev_loaded,
+ ice_stat_update32(hw, GLPRT_RJC(port), pf->stat_prev_loaded,
&prev_ps->rx_jabber, &cur_ps->rx_jabber);
pf->stat_prev_loaded = true;
@@ -3442,12 +3490,16 @@ void ice_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *stats)
vsi_stats = &vsi->net_stats;
- if (test_bit(__ICE_DOWN, vsi->state) || !vsi->num_txq || !vsi->num_rxq)
+ if (!vsi->num_txq || !vsi->num_rxq)
return;
+
/* netdev packet/byte stats come from ring counter. These are obtained
* by summing up ring counters (done by ice_update_vsi_ring_stats).
+ * But, only call the update routine and read the registers if VSI is
+ * not down.
*/
- ice_update_vsi_ring_stats(vsi);
+ if (!test_bit(__ICE_DOWN, vsi->state))
+ ice_update_vsi_ring_stats(vsi);
stats->tx_packets = vsi_stats->tx_packets;
stats->tx_bytes = vsi_stats->tx_bytes;
stats->rx_packets = vsi_stats->rx_packets;
@@ -3686,21 +3738,19 @@ static int ice_ena_vsi(struct ice_vsi *vsi, bool locked)
int err = 0;
if (!test_bit(__ICE_NEEDS_RESTART, vsi->state))
- return err;
+ return 0;
clear_bit(__ICE_NEEDS_RESTART, vsi->state);
if (vsi->netdev && vsi->type == ICE_VSI_PF) {
- struct net_device *netd = vsi->netdev;
-
if (netif_running(vsi->netdev)) {
- if (locked) {
- err = netd->netdev_ops->ndo_open(netd);
- } else {
+ if (!locked)
rtnl_lock();
- err = netd->netdev_ops->ndo_open(netd);
+
+ err = ice_open(vsi->netdev);
+
+ if (!locked)
rtnl_unlock();
- }
}
}
@@ -3738,22 +3788,23 @@ static int ice_vsi_rebuild_all(struct ice_pf *pf)
/* loop through pf->vsi array and reinit the VSI if found */
ice_for_each_vsi(pf, i) {
+ struct ice_vsi *vsi = pf->vsi[i];
int err;
- if (!pf->vsi[i])
+ if (!vsi)
continue;
- err = ice_vsi_rebuild(pf->vsi[i]);
+ err = ice_vsi_rebuild(vsi);
if (err) {
dev_err(&pf->pdev->dev,
"VSI at index %d rebuild failed\n",
- pf->vsi[i]->idx);
+ vsi->idx);
return err;
}
dev_info(&pf->pdev->dev,
"VSI at index %d rebuilt. vsi_num = 0x%x\n",
- pf->vsi[i]->idx, pf->vsi[i]->vsi_num);
+ vsi->idx, vsi->vsi_num);
}
return 0;
@@ -3771,25 +3822,27 @@ static int ice_vsi_replay_all(struct ice_pf *pf)
/* loop through pf->vsi array and replay the VSI if found */
ice_for_each_vsi(pf, i) {
- if (!pf->vsi[i])
+ struct ice_vsi *vsi = pf->vsi[i];
+
+ if (!vsi)
continue;
- ret = ice_replay_vsi(hw, pf->vsi[i]->idx);
+ ret = ice_replay_vsi(hw, vsi->idx);
if (ret) {
dev_err(&pf->pdev->dev,
"VSI at index %d replay failed %d\n",
- pf->vsi[i]->idx, ret);
+ vsi->idx, ret);
return -EIO;
}
/* Re-map HW VSI number, using VSI handle that has been
* previously validated in ice_replay_vsi() call above
*/
- pf->vsi[i]->vsi_num = ice_get_hw_vsi_num(hw, pf->vsi[i]->idx);
+ vsi->vsi_num = ice_get_hw_vsi_num(hw, vsi->idx);
dev_info(&pf->pdev->dev,
"VSI at index %d filter replayed successfully - vsi_num %i\n",
- pf->vsi[i]->idx, pf->vsi[i]->vsi_num);
+ vsi->idx, vsi->vsi_num);
}
/* Clean up replay filter after successful re-configuration */
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.c b/drivers/net/ethernet/intel/ice/ice_sched.c
index 2a23250..fc624b7 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.c
+++ b/drivers/net/ethernet/intel/ice/ice_sched.c
@@ -260,33 +260,17 @@ ice_sched_remove_elems(struct ice_hw *hw, struct ice_sched_node *parent,
/**
* ice_sched_get_first_node - get the first node of the given layer
- * @hw: pointer to the HW struct
+ * @pi: port information structure
* @parent: pointer the base node of the subtree
* @layer: layer number
*
* This function retrieves the first node of the given layer from the subtree
*/
static struct ice_sched_node *
-ice_sched_get_first_node(struct ice_hw *hw, struct ice_sched_node *parent,
- u8 layer)
+ice_sched_get_first_node(struct ice_port_info *pi,
+ struct ice_sched_node *parent, u8 layer)
{
- u8 i;
-
- if (layer < hw->sw_entry_point_layer)
- return NULL;
- for (i = 0; i < parent->num_children; i++) {
- struct ice_sched_node *node = parent->children[i];
-
- if (node) {
- if (node->tx_sched_layer == layer)
- return node;
- /* this recursion is intentional, and wouldn't
- * go more than 9 calls
- */
- return ice_sched_get_first_node(hw, node, layer);
- }
- }
- return NULL;
+ return pi->sib_head[parent->tc_num][layer];
}
/**
@@ -300,7 +284,7 @@ struct ice_sched_node *ice_sched_get_tc_node(struct ice_port_info *pi, u8 tc)
{
u8 i;
- if (!pi)
+ if (!pi || !pi->root)
return NULL;
for (i = 0; i < pi->root->num_children; i++)
if (pi->root->children[i]->tc_num == tc)
@@ -342,7 +326,7 @@ void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node)
parent = node->parent;
/* root has no parent */
if (parent) {
- struct ice_sched_node *p, *tc_node;
+ struct ice_sched_node *p;
/* update the parent */
for (i = 0; i < parent->num_children; i++)
@@ -354,16 +338,7 @@ void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node)
break;
}
- /* search for previous sibling that points to this node and
- * remove the reference
- */
- tc_node = ice_sched_get_tc_node(pi, node->tc_num);
- if (!tc_node) {
- ice_debug(hw, ICE_DBG_SCHED,
- "Invalid TC number %d\n", node->tc_num);
- goto err_exit;
- }
- p = ice_sched_get_first_node(hw, tc_node, node->tx_sched_layer);
+ p = ice_sched_get_first_node(pi, node, node->tx_sched_layer);
while (p) {
if (p->sibling == node) {
p->sibling = node->sibling;
@@ -371,8 +346,13 @@ void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node)
}
p = p->sibling;
}
+
+ /* update the sibling head if head is getting removed */
+ if (pi->sib_head[node->tc_num][node->tx_sched_layer] == node)
+ pi->sib_head[node->tc_num][node->tx_sched_layer] =
+ node->sibling;
}
-err_exit:
+
/* leaf nodes have no children */
if (node->children)
devm_kfree(ice_hw_to_dev(hw), node->children);
@@ -743,13 +723,17 @@ ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
/* add it to previous node sibling pointer */
/* Note: siblings are not linked across branches */
- prev = ice_sched_get_first_node(hw, tc_node, layer);
+ prev = ice_sched_get_first_node(pi, tc_node, layer);
if (prev && prev != new_node) {
while (prev->sibling)
prev = prev->sibling;
prev->sibling = new_node;
}
+ /* initialize the sibling head */
+ if (!pi->sib_head[tc_node->tc_num][layer])
+ pi->sib_head[tc_node->tc_num][layer] = new_node;
+
if (i == 0)
*first_node_teid = teid;
}
@@ -1160,7 +1144,7 @@ ice_sched_get_free_qparent(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
goto lan_q_exit;
/* get the first queue group node from VSI sub-tree */
- qgrp_node = ice_sched_get_first_node(pi->hw, vsi_node, qgrp_layer);
+ qgrp_node = ice_sched_get_first_node(pi, vsi_node, qgrp_layer);
while (qgrp_node) {
/* make sure the qgroup node is part of the VSI subtree */
if (ice_sched_find_node_in_subtree(pi->hw, vsi_node, qgrp_node))
@@ -1191,7 +1175,7 @@ ice_sched_get_vsi_node(struct ice_hw *hw, struct ice_sched_node *tc_node,
u8 vsi_layer;
vsi_layer = ice_sched_get_vsi_layer(hw);
- node = ice_sched_get_first_node(hw, tc_node, vsi_layer);
+ node = ice_sched_get_first_node(hw->port_info, tc_node, vsi_layer);
/* Check whether it already exists */
while (node) {
@@ -1316,7 +1300,8 @@ ice_sched_calc_vsi_support_nodes(struct ice_hw *hw,
/* If intermediate nodes are reached max children
* then add a new one.
*/
- node = ice_sched_get_first_node(hw, tc_node, (u8)i);
+ node = ice_sched_get_first_node(hw->port_info, tc_node,
+ (u8)i);
/* scan all the siblings */
while (node) {
if (node->num_children < hw->max_children[i])
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.c b/drivers/net/ethernet/intel/ice/ice_switch.c
index 8271fd6..1acdd43 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.c
+++ b/drivers/net/ethernet/intel/ice/ice_switch.c
@@ -1623,12 +1623,13 @@ ice_remove_rule_internal(struct ice_hw *hw, u8 recp_id,
status = ice_aq_sw_rules(hw, s_rule,
ICE_SW_RULE_RX_TX_NO_HDR_SIZE, 1,
ice_aqc_opc_remove_sw_rules, NULL);
- if (status)
- goto exit;
/* Remove a book keeping from the list */
devm_kfree(ice_hw_to_dev(hw), s_rule);
+ if (status)
+ goto exit;
+
list_del(&list_elem->list_entry);
devm_kfree(ice_hw_to_dev(hw), list_elem);
}
@@ -2137,6 +2138,38 @@ ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_handle, bool set, u8 direction)
}
/**
+ * ice_find_ucast_rule_entry - Search for a unicast MAC filter rule entry
+ * @hw: pointer to the hardware structure
+ * @recp_id: lookup type for which the specified rule needs to be searched
+ * @f_info: rule information
+ *
+ * Helper function to search for a unicast rule entry - this is to be used
+ * to remove unicast MAC filter that is not shared with other VSIs on the
+ * PF switch.
+ *
+ * Returns pointer to entry storing the rule if found
+ */
+static struct ice_fltr_mgmt_list_entry *
+ice_find_ucast_rule_entry(struct ice_hw *hw, u8 recp_id,
+ struct ice_fltr_info *f_info)
+{
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_fltr_mgmt_list_entry *list_itr;
+ struct list_head *list_head;
+
+ list_head = &sw->recp_list[recp_id].filt_rules;
+ list_for_each_entry(list_itr, list_head, list_entry) {
+ if (!memcmp(&f_info->l_data, &list_itr->fltr_info.l_data,
+ sizeof(f_info->l_data)) &&
+ f_info->fwd_id.hw_vsi_id ==
+ list_itr->fltr_info.fwd_id.hw_vsi_id &&
+ f_info->flag == list_itr->fltr_info.flag)
+ return list_itr;
+ }
+ return NULL;
+}
+
+/**
* ice_remove_mac - remove a MAC address based filter rule
* @hw: pointer to the hardware structure
* @m_list: list of MAC addresses and forwarding information
@@ -2153,15 +2186,39 @@ enum ice_status
ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
{
struct ice_fltr_list_entry *list_itr, *tmp;
+ struct mutex *rule_lock; /* Lock to protect filter rule list */
if (!m_list)
return ICE_ERR_PARAM;
+ rule_lock = &hw->switch_info->recp_list[ICE_SW_LKUP_MAC].filt_rule_lock;
list_for_each_entry_safe(list_itr, tmp, m_list, list_entry) {
enum ice_sw_lkup_type l_type = list_itr->fltr_info.lkup_type;
+ u8 *add = &list_itr->fltr_info.l_data.mac.mac_addr[0];
+ u16 vsi_handle;
if (l_type != ICE_SW_LKUP_MAC)
return ICE_ERR_PARAM;
+
+ vsi_handle = list_itr->fltr_info.vsi_handle;
+ if (!ice_is_vsi_valid(hw, vsi_handle))
+ return ICE_ERR_PARAM;
+
+ list_itr->fltr_info.fwd_id.hw_vsi_id =
+ ice_get_hw_vsi_num(hw, vsi_handle);
+ if (is_unicast_ether_addr(add) && !hw->ucast_shared) {
+ /* Don't remove the unicast address that belongs to
+ * another VSI on the switch, since it is not being
+ * shared...
+ */
+ mutex_lock(rule_lock);
+ if (!ice_find_ucast_rule_entry(hw, ICE_SW_LKUP_MAC,
+ &list_itr->fltr_info)) {
+ mutex_unlock(rule_lock);
+ return ICE_ERR_DOES_NOT_EXIST;
+ }
+ mutex_unlock(rule_lock);
+ }
list_itr->status = ice_remove_rule_internal(hw,
ICE_SW_LKUP_MAC,
list_itr);
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index e5c4c91..33dd103 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -95,17 +95,16 @@ void ice_free_tx_ring(struct ice_ring *tx_ring)
/**
* ice_clean_tx_irq - Reclaim resources after transmit completes
- * @vsi: the VSI we care about
* @tx_ring: Tx ring to clean
* @napi_budget: Used to determine if we are in netpoll
*
* Returns true if there's any budget left (e.g. the clean is finished)
*/
-static bool
-ice_clean_tx_irq(struct ice_vsi *vsi, struct ice_ring *tx_ring, int napi_budget)
+static bool ice_clean_tx_irq(struct ice_ring *tx_ring, int napi_budget)
{
unsigned int total_bytes = 0, total_pkts = 0;
- unsigned int budget = vsi->work_lmt;
+ unsigned int budget = ICE_DFLT_IRQ_WORK;
+ struct ice_vsi *vsi = tx_ring->vsi;
s16 i = tx_ring->next_to_clean;
struct ice_tx_desc *tx_desc;
struct ice_tx_buf *tx_buf;
@@ -114,6 +113,8 @@ ice_clean_tx_irq(struct ice_vsi *vsi, struct ice_ring *tx_ring, int napi_budget)
tx_desc = ICE_TX_DESC(tx_ring, i);
i -= tx_ring->count;
+ prefetch(&vsi->state);
+
do {
struct ice_tx_desc *eop_desc = tx_buf->next_to_watch;
@@ -206,7 +207,7 @@ ice_clean_tx_irq(struct ice_vsi *vsi, struct ice_ring *tx_ring, int napi_budget)
smp_mb();
if (__netif_subqueue_stopped(tx_ring->netdev,
tx_ring->q_index) &&
- !test_bit(__ICE_DOWN, vsi->state)) {
+ !test_bit(__ICE_DOWN, vsi->state)) {
netif_wake_subqueue(tx_ring->netdev,
tx_ring->q_index);
++tx_ring->tx_stats.restart_q;
@@ -879,7 +880,7 @@ ice_rx_hash(struct ice_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc,
/**
* ice_rx_csum - Indicate in skb if checksum is good
- * @vsi: the VSI we care about
+ * @ring: the ring we care about
* @skb: skb currently being received and modified
* @rx_desc: the receive descriptor
* @ptype: the packet type decoded by hardware
@@ -887,7 +888,7 @@ ice_rx_hash(struct ice_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc,
* skb->protocol must be set before this function is called
*/
static void
-ice_rx_csum(struct ice_vsi *vsi, struct sk_buff *skb,
+ice_rx_csum(struct ice_ring *ring, struct sk_buff *skb,
union ice_32b_rx_flex_desc *rx_desc, u8 ptype)
{
struct ice_rx_ptype_decoded decoded;
@@ -904,7 +905,7 @@ ice_rx_csum(struct ice_vsi *vsi, struct sk_buff *skb,
skb_checksum_none_assert(skb);
/* check if Rx checksum is enabled */
- if (!(vsi->netdev->features & NETIF_F_RXCSUM))
+ if (!(ring->netdev->features & NETIF_F_RXCSUM))
return;
/* check if HW has decoded the packet and checksum */
@@ -944,7 +945,7 @@ ice_rx_csum(struct ice_vsi *vsi, struct sk_buff *skb,
return;
checksum_fail:
- vsi->back->hw_csum_rx_error++;
+ ring->vsi->back->hw_csum_rx_error++;
}
/**
@@ -968,7 +969,7 @@ ice_process_skb_fields(struct ice_ring *rx_ring,
/* modifies the skb - consumes the enet header */
skb->protocol = eth_type_trans(skb, rx_ring->netdev);
- ice_rx_csum(rx_ring->vsi, skb, rx_desc, ptype);
+ ice_rx_csum(rx_ring, skb, rx_desc, ptype);
}
/**
@@ -1067,9 +1068,6 @@ static int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
continue;
}
- rx_ptype = le16_to_cpu(rx_desc->wb.ptype_flex_flags0) &
- ICE_RX_FLEX_DESC_PTYPE_M;
-
stat_err_bits = BIT(ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S);
if (ice_test_staterr(rx_desc, stat_err_bits))
vlan_tag = le16_to_cpu(rx_desc->wb.l2tag1);
@@ -1086,6 +1084,9 @@ static int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
total_rx_bytes += skb->len;
/* populate checksum, VLAN, and protocol */
+ rx_ptype = le16_to_cpu(rx_desc->wb.ptype_flex_flags0) &
+ ICE_RX_FLEX_DESC_PTYPE_M;
+
ice_process_skb_fields(rx_ring, rx_desc, skb, rx_ptype);
/* send completed skb up the stack */
@@ -1225,6 +1226,8 @@ ice_update_itr(struct ice_q_vector *q_vector, struct ice_ring_container *rc)
if (time_after(next_update, rc->next_update))
goto clear_counts;
+ prefetch(q_vector->vsi->port_info);
+
packets = rc->total_pkts;
bytes = rc->total_bytes;
@@ -1354,14 +1357,13 @@ static u32 ice_buildreg_itr(u16 itr_idx, u16 itr)
/**
* ice_update_ena_itr - Update ITR and re-enable MSIX interrupt
- * @vsi: the VSI associated with the q_vector
* @q_vector: q_vector for which ITR is being updated and interrupt enabled
*/
-static void
-ice_update_ena_itr(struct ice_vsi *vsi, struct ice_q_vector *q_vector)
+static void ice_update_ena_itr(struct ice_q_vector *q_vector)
{
struct ice_ring_container *tx = &q_vector->tx;
struct ice_ring_container *rx = &q_vector->rx;
+ struct ice_vsi *vsi = q_vector->vsi;
u32 itr_val;
/* when exiting WB_ON_ITR lets set a low ITR value and trigger
@@ -1419,15 +1421,14 @@ ice_update_ena_itr(struct ice_vsi *vsi, struct ice_q_vector *q_vector)
q_vector->itr_countdown--;
}
- if (!test_bit(__ICE_DOWN, vsi->state))
- wr32(&vsi->back->hw,
+ if (!test_bit(__ICE_DOWN, q_vector->vsi->state))
+ wr32(&q_vector->vsi->back->hw,
GLINT_DYN_CTL(q_vector->reg_idx),
itr_val);
}
/**
* ice_set_wb_on_itr - set WB_ON_ITR for this q_vector
- * @vsi: pointer to the VSI structure
* @q_vector: q_vector to set WB_ON_ITR on
*
* We need to tell hardware to write-back completed descriptors even when
@@ -1440,9 +1441,10 @@ ice_update_ena_itr(struct ice_vsi *vsi, struct ice_q_vector *q_vector)
* value that's not 0 due to ITR granularity. Also, set the INTENA_MSK bit to
* make sure hardware knows we aren't meddling with the INTENA_M bit.
*/
-static void
-ice_set_wb_on_itr(struct ice_vsi *vsi, struct ice_q_vector *q_vector)
+static void ice_set_wb_on_itr(struct ice_q_vector *q_vector)
{
+ struct ice_vsi *vsi = q_vector->vsi;
+
/* already in WB_ON_ITR mode no need to change it */
if (q_vector->itr_countdown == ICE_IN_WB_ON_ITR_MODE)
return;
@@ -1473,7 +1475,6 @@ int ice_napi_poll(struct napi_struct *napi, int budget)
{
struct ice_q_vector *q_vector =
container_of(napi, struct ice_q_vector, napi);
- struct ice_vsi *vsi = q_vector->vsi;
bool clean_complete = true;
struct ice_ring *ring;
int budget_per_ring;
@@ -1483,11 +1484,11 @@ int ice_napi_poll(struct napi_struct *napi, int budget)
* budget and be more aggressive about cleaning up the Tx descriptors.
*/
ice_for_each_ring(ring, q_vector->tx)
- if (!ice_clean_tx_irq(vsi, ring, budget))
+ if (!ice_clean_tx_irq(ring, budget))
clean_complete = false;
/* Handle case where we are called by netpoll with a budget of 0 */
- if (budget <= 0)
+ if (unlikely(budget <= 0))
return budget;
/* normally we have 1 Rx ring per q_vector */
@@ -1519,9 +1520,9 @@ int ice_napi_poll(struct napi_struct *napi, int budget)
* poll us due to busy-polling
*/
if (likely(napi_complete_done(napi, work_done)))
- ice_update_ena_itr(vsi, q_vector);
+ ice_update_ena_itr(q_vector);
else
- ice_set_wb_on_itr(vsi, q_vector);
+ ice_set_wb_on_itr(q_vector);
return min_t(int, work_done, budget - 1);
}
@@ -2106,6 +2107,7 @@ static netdev_tx_t
ice_xmit_frame_ring(struct sk_buff *skb, struct ice_ring *tx_ring)
{
struct ice_tx_offload_params offload = { 0 };
+ struct ice_vsi *vsi = tx_ring->vsi;
struct ice_tx_buf *first;
unsigned int count;
int tso, csum;
@@ -2153,7 +2155,15 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_ring *tx_ring)
if (csum < 0)
goto out_drop;
- if (tso || offload.cd_tunnel_params) {
+ /* allow CONTROL frames egress from main VSI if FW LLDP disabled */
+ if (unlikely(skb->priority == TC_PRIO_CONTROL &&
+ vsi->type == ICE_VSI_PF &&
+ vsi->port_info->is_sw_lldp))
+ offload.cd_qw1 |= (u64)(ICE_TX_DESC_DTYPE_CTX |
+ ICE_TX_CTX_DESC_SWTCH_UPLINK <<
+ ICE_TXD_CTX_QW1_CMD_S);
+
+ if (offload.cd_qw1 & ICE_TX_DESC_DTYPE_CTX) {
struct ice_tx_ctx_desc *cdesc;
int i = tx_ring->next_to_use;
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index 24bbef8..4501d50 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -4,18 +4,18 @@
#ifndef _ICE_TYPE_H_
#define _ICE_TYPE_H_
+#define ICE_BYTES_PER_WORD 2
+#define ICE_BYTES_PER_DWORD 4
+
#include "ice_status.h"
#include "ice_hw_autogen.h"
#include "ice_osdep.h"
#include "ice_controlq.h"
#include "ice_lan_tx_rx.h"
-#define ICE_BYTES_PER_WORD 2
-#define ICE_BYTES_PER_DWORD 4
-
-static inline bool ice_is_tc_ena(u8 bitmap, u8 tc)
+static inline bool ice_is_tc_ena(unsigned long bitmap, u8 tc)
{
- return test_bit(tc, (unsigned long *)&bitmap);
+ return test_bit(tc, &bitmap);
}
/* Driver always calls main vsi_handle first */
@@ -139,6 +139,9 @@ struct ice_phy_info {
/* Common HW capabilities for SW use */
struct ice_hw_common_caps {
u32 valid_functions;
+ /* DCB capabilities */
+ u32 active_tc_bitmap;
+ u32 maxtc;
/* Tx/Rx queues */
u16 num_rxq; /* Number/Total Rx queues */
@@ -347,6 +350,8 @@ struct ice_port_info {
struct ice_mac_info mac;
struct ice_phy_info phy;
struct mutex sched_lock; /* protect access to TXSched tree */
+ struct ice_sched_node *
+ sib_head[ICE_MAX_TRAFFIC_CLASS][ICE_AQC_TOPO_MAX_LEVEL_NUM];
struct ice_dcbx_cfg local_dcbx_cfg; /* Oper/Local Cfg */
/* DCBX info */
struct ice_dcbx_cfg remote_dcbx_cfg; /* Peer Cfg */
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
index 1b1d1ea..64de05c 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
@@ -129,7 +129,10 @@ static void ice_vc_notify_vf_link_state(struct ice_vf *vf)
pfe.event = VIRTCHNL_EVENT_LINK_CHANGE;
pfe.severity = PF_EVENT_SEVERITY_INFO;
- if (vf->link_forced)
+ /* Always report link is down if the VF queues aren't enabled */
+ if (!vf->num_qs_ena)
+ ice_set_pfe_link(vf, &pfe, ICE_AQ_LINK_SPEED_UNKNOWN, false);
+ else if (vf->link_forced)
ice_set_pfe_link_forced(vf, &pfe, vf->link_up);
else
ice_set_pfe_link(vf, &pfe, ls->link_speed, ls->link_info &
@@ -252,6 +255,35 @@ static int ice_sriov_free_msix_res(struct ice_pf *pf)
}
/**
+ * ice_set_vf_state_qs_dis - Set VF queues state to disabled
+ * @vf: pointer to the VF structure
+ */
+void ice_set_vf_state_qs_dis(struct ice_vf *vf)
+{
+ /* Clear Rx/Tx enabled queues flag */
+ bitmap_zero(vf->txq_ena, ICE_MAX_BASE_QS_PER_VF);
+ bitmap_zero(vf->rxq_ena, ICE_MAX_BASE_QS_PER_VF);
+ vf->num_qs_ena = 0;
+ clear_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
+}
+
+/**
+ * ice_dis_vf_qs - Disable the VF queues
+ * @vf: pointer to the VF structure
+ */
+static void ice_dis_vf_qs(struct ice_vf *vf)
+{
+ struct ice_pf *pf = vf->pf;
+ struct ice_vsi *vsi;
+
+ vsi = pf->vsi[vf->lan_vsi_idx];
+
+ ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, vf->vf_id);
+ ice_vsi_stop_rx_rings(vsi);
+ ice_set_vf_state_qs_dis(vf);
+}
+
+/**
* ice_free_vfs - Free all VFs
* @pf: pointer to the PF structure
*/
@@ -267,19 +299,9 @@ void ice_free_vfs(struct ice_pf *pf)
usleep_range(1000, 2000);
/* Avoid wait time by stopping all VFs at the same time */
- for (i = 0; i < pf->num_alloc_vfs; i++) {
- struct ice_vsi *vsi;
-
- if (!test_bit(ICE_VF_STATE_ENA, pf->vf[i].vf_states))
- continue;
-
- vsi = pf->vsi[pf->vf[i].lan_vsi_idx];
- /* stop rings without wait time */
- ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, i);
- ice_vsi_stop_rx_rings(vsi);
-
- clear_bit(ICE_VF_STATE_ENA, pf->vf[i].vf_states);
- }
+ for (i = 0; i < pf->num_alloc_vfs; i++)
+ if (test_bit(ICE_VF_STATE_QS_ENA, pf->vf[i].vf_states))
+ ice_dis_vf_qs(&pf->vf[i]);
/* Disable IOV before freeing resources. This lets any VF drivers
* running in the host get themselves cleaned up before we yank
@@ -334,12 +356,13 @@ void ice_free_vfs(struct ice_pf *pf)
* ice_trigger_vf_reset - Reset a VF on HW
* @vf: pointer to the VF structure
* @is_vflr: true if VFLR was issued, false if not
+ * @is_pfr: true if the reset was triggered due to a previous PFR
*
* Trigger hardware to start a reset for a particular VF. Expects the caller
* to wait the proper amount of time to allow hardware to reset the VF before
* it cleans up and restores VF functionality.
*/
-static void ice_trigger_vf_reset(struct ice_vf *vf, bool is_vflr)
+static void ice_trigger_vf_reset(struct ice_vf *vf, bool is_vflr, bool is_pfr)
{
struct ice_pf *pf = vf->pf;
u32 reg, reg_idx, bit_idx;
@@ -360,10 +383,13 @@ static void ice_trigger_vf_reset(struct ice_vf *vf, bool is_vflr)
*/
clear_bit(ICE_VF_STATE_INIT, vf->vf_states);
- /* Clear the VF's ARQLEN register. This is how the VF detects reset,
- * since the VFGEN_RSTAT register doesn't stick at 0 after reset.
+ /* VF_MBX_ARQLEN is cleared by PFR, so the driver needs to clear it
+ * in the case of VFR. If this is done for PFR, it can mess up VF
+ * resets because the VF driver may already have started cleanup
+ * by the time we get here.
*/
- wr32(hw, VF_MBX_ARQLEN(vf_abs_id), 0);
+ if (!is_pfr)
+ wr32(hw, VF_MBX_ARQLEN(vf_abs_id), 0);
/* In the case of a VFLR, the HW has already reset the VF and we
* just need to clean up, so don't hit the VFRTRIG register.
@@ -540,7 +566,10 @@ static int ice_alloc_vsi_res(struct ice_vf *vf)
status = ice_add_mac(&pf->hw, &tmp_add_list);
if (status)
- dev_err(&pf->pdev->dev, "could not add mac filters\n");
+ dev_err(&pf->pdev->dev,
+ "could not add mac filters error %d\n", status);
+ else
+ vf->num_mac = 1;
/* Clear this bit after VF initialization since we shouldn't reclaim
* and reassign interrupts for synchronous or asynchronous VFR events.
@@ -566,7 +595,8 @@ static int ice_alloc_vf_res(struct ice_vf *vf)
/* Update number of VF queues, in case VF had requested for queue
* changes
*/
- tx_rx_queue_left = min_t(int, pf->q_left_tx, pf->q_left_rx);
+ tx_rx_queue_left = min_t(int, ice_get_avail_txq_count(pf),
+ ice_get_avail_rxq_count(pf));
tx_rx_queue_left += ICE_DFLT_QS_PER_VF;
if (vf->num_req_qs && vf->num_req_qs <= tx_rx_queue_left &&
vf->num_req_qs != vf->num_vf_qs)
@@ -869,11 +899,11 @@ static int ice_check_avail_res(struct ice_pf *pf)
* at runtime through Virtchnl, that is the reason we start by reserving
* few queues.
*/
- num_txq = ice_determine_res(pf, pf->q_left_tx, ICE_DFLT_QS_PER_VF,
- ICE_MIN_QS_PER_VF);
+ num_txq = ice_determine_res(pf, ice_get_avail_txq_count(pf),
+ ICE_DFLT_QS_PER_VF, ICE_MIN_QS_PER_VF);
- num_rxq = ice_determine_res(pf, pf->q_left_rx, ICE_DFLT_QS_PER_VF,
- ICE_MIN_QS_PER_VF);
+ num_rxq = ice_determine_res(pf, ice_get_avail_rxq_count(pf),
+ ICE_DFLT_QS_PER_VF, ICE_MIN_QS_PER_VF);
if (!num_txq || !num_rxq)
return -EIO;
@@ -1050,18 +1080,17 @@ bool ice_reset_all_vfs(struct ice_pf *pf, bool is_vflr)
/* Begin reset on all VFs at once */
for (v = 0; v < pf->num_alloc_vfs; v++)
- ice_trigger_vf_reset(&pf->vf[v], is_vflr);
+ ice_trigger_vf_reset(&pf->vf[v], is_vflr, true);
for (v = 0; v < pf->num_alloc_vfs; v++) {
struct ice_vsi *vsi;
vf = &pf->vf[v];
vsi = pf->vsi[vf->lan_vsi_idx];
- if (test_bit(ICE_VF_STATE_ENA, vf->vf_states)) {
- ice_vsi_stop_lan_tx_rings(vsi, ICE_VF_RESET, vf->vf_id);
- ice_vsi_stop_rx_rings(vsi);
- clear_bit(ICE_VF_STATE_ENA, vf->vf_states);
- }
+ if (test_bit(ICE_VF_STATE_QS_ENA, vf->vf_states))
+ ice_dis_vf_qs(vf);
+ ice_dis_vsi_txq(vsi->port_info, vsi->idx, 0, 0, NULL, NULL,
+ NULL, ICE_VF_RESET, vf->vf_id, NULL);
}
/* HW requires some time to make sure it can flush the FIFO for a VF
@@ -1138,27 +1167,31 @@ static bool ice_reset_vf(struct ice_vf *vf, bool is_vflr)
u32 reg;
int i;
- /* If the VFs have been disabled, this means something else is
- * resetting the VF, so we shouldn't continue.
+ /* If the PF has been disabled, there is no need resetting VF until
+ * PF is active again.
*/
- if (test_and_set_bit(__ICE_VF_DIS, pf->state))
+ if (test_bit(__ICE_VF_DIS, pf->state))
return false;
- ice_trigger_vf_reset(vf, is_vflr);
+ /* If the VF has been disabled, this means something else is
+ * resetting the VF, so we shouldn't continue. Otherwise, set
+ * disable VF state bit for actual reset, and continue.
+ */
+ if (test_and_set_bit(ICE_VF_STATE_DIS, vf->vf_states))
+ return false;
+
+ ice_trigger_vf_reset(vf, is_vflr, false);
vsi = pf->vsi[vf->lan_vsi_idx];
- if (test_bit(ICE_VF_STATE_ENA, vf->vf_states)) {
- ice_vsi_stop_lan_tx_rings(vsi, ICE_VF_RESET, vf->vf_id);
- ice_vsi_stop_rx_rings(vsi);
- clear_bit(ICE_VF_STATE_ENA, vf->vf_states);
- } else {
- /* Call Disable LAN Tx queue AQ call even when queues are not
- * enabled. This is needed for successful completiom of VFR
- */
- ice_dis_vsi_txq(vsi->port_info, vsi->idx, 0, 0, NULL, NULL,
- NULL, ICE_VF_RESET, vf->vf_id, NULL);
- }
+ if (test_bit(ICE_VF_STATE_QS_ENA, vf->vf_states))
+ ice_dis_vf_qs(vf);
+
+ /* Call Disable LAN Tx queue AQ whether or not queues are
+ * enabled. This is needed for successful completion of VFR.
+ */
+ ice_dis_vsi_txq(vsi->port_info, vsi->idx, 0, 0, NULL, NULL,
+ NULL, ICE_VF_RESET, vf->vf_id, NULL);
hw = &pf->hw;
/* poll VPGEN_VFRSTAT reg to make sure
@@ -1207,7 +1240,6 @@ static bool ice_reset_vf(struct ice_vf *vf, bool is_vflr)
ice_cleanup_and_realloc_vf(vf);
ice_flush(hw);
- clear_bit(__ICE_VF_DIS, pf->state);
return true;
}
@@ -1512,10 +1544,10 @@ ice_vc_send_msg_to_vf(struct ice_vf *vf, u32 v_opcode,
aq_ret = ice_aq_send_msg_to_vf(&pf->hw, vf->vf_id, v_opcode, v_retval,
msg, msglen, NULL);
- if (aq_ret) {
+ if (aq_ret && pf->hw.mailboxq.sq_last_status != ICE_AQ_RC_ENOSYS) {
dev_info(&pf->pdev->dev,
- "Unable to send the message to VF %d aq_err %d\n",
- vf->vf_id, pf->hw.mailboxq.sq_last_status);
+ "Unable to send the message to VF %d ret %d aq_err %d\n",
+ vf->vf_id, aq_ret, pf->hw.mailboxq.sq_last_status);
return -EIO;
}
@@ -1710,6 +1742,21 @@ static bool ice_vc_isvalid_q_id(struct ice_vf *vf, u16 vsi_id, u8 qid)
}
/**
+ * ice_vc_isvalid_ring_len
+ * @ring_len: length of ring
+ *
+ * check for the valid ring count, should be multiple of ICE_REQ_DESC_MULTIPLE
+ * or zero
+ */
+static bool ice_vc_isvalid_ring_len(u16 ring_len)
+{
+ return ring_len == 0 ||
+ (ring_len >= ICE_MIN_NUM_DESC &&
+ ring_len <= ICE_MAX_NUM_DESC &&
+ !(ring_len % ICE_REQ_DESC_MULTIPLE));
+}
+
+/**
* ice_vc_config_rss_key
* @vf: pointer to the VF info
* @msg: pointer to the msg buffer
@@ -1734,12 +1781,6 @@ static int ice_vc_config_rss_key(struct ice_vf *vf, u8 *msg)
goto error_param;
}
- vsi = pf->vsi[vf->lan_vsi_idx];
- if (!vsi) {
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
- goto error_param;
- }
-
if (vrk->key_len != ICE_VSIQF_HKEY_ARRAY_SIZE) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
@@ -1750,6 +1791,12 @@ static int ice_vc_config_rss_key(struct ice_vf *vf, u8 *msg)
goto error_param;
}
+ vsi = pf->vsi[vf->lan_vsi_idx];
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
if (ice_set_rss(vsi, vrk->key, NULL, 0))
v_ret = VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR;
error_param:
@@ -1781,12 +1828,6 @@ static int ice_vc_config_rss_lut(struct ice_vf *vf, u8 *msg)
goto error_param;
}
- vsi = pf->vsi[vf->lan_vsi_idx];
- if (!vsi) {
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
- goto error_param;
- }
-
if (vrl->lut_entries != ICE_VSIQF_HLUT_ARRAY_SIZE) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
@@ -1797,6 +1838,12 @@ static int ice_vc_config_rss_lut(struct ice_vf *vf, u8 *msg)
goto error_param;
}
+ vsi = pf->vsi[vf->lan_vsi_idx];
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
if (ice_set_rss(vsi, NULL, vrl->lut, ICE_VSIQF_HLUT_ARRAY_SIZE))
v_ret = VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR;
error_param:
@@ -1861,6 +1908,8 @@ static int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)
(struct virtchnl_queue_select *)msg;
struct ice_pf *pf = vf->pf;
struct ice_vsi *vsi;
+ unsigned long q_map;
+ u16 vf_q_id;
if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
@@ -1877,6 +1926,12 @@ static int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)
goto error_param;
}
+ if (vqs->rx_queues > ICE_MAX_BASE_QS_PER_VF ||
+ vqs->tx_queues > ICE_MAX_BASE_QS_PER_VF) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
vsi = pf->vsi[vf->lan_vsi_idx];
if (!vsi) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
@@ -1887,12 +1942,48 @@ static int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)
* Tx queue group list was configured and the context bits were
* programmed using ice_vsi_cfg_txqs
*/
- if (ice_vsi_start_rx_rings(vsi))
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ q_map = vqs->rx_queues;
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_BASE_QS_PER_VF) {
+ if (!ice_vc_isvalid_q_id(vf, vqs->vsi_id, vf_q_id)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* Skip queue if enabled */
+ if (test_bit(vf_q_id, vf->rxq_ena))
+ continue;
+
+ if (ice_vsi_ctrl_rx_ring(vsi, true, vf_q_id)) {
+ dev_err(&vsi->back->pdev->dev,
+ "Failed to enable Rx ring %d on VSI %d\n",
+ vf_q_id, vsi->vsi_num);
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ set_bit(vf_q_id, vf->rxq_ena);
+ vf->num_qs_ena++;
+ }
+
+ vsi = pf->vsi[vf->lan_vsi_idx];
+ q_map = vqs->tx_queues;
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_BASE_QS_PER_VF) {
+ if (!ice_vc_isvalid_q_id(vf, vqs->vsi_id, vf_q_id)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* Skip queue if enabled */
+ if (test_bit(vf_q_id, vf->txq_ena))
+ continue;
+
+ set_bit(vf_q_id, vf->txq_ena);
+ vf->num_qs_ena++;
+ }
/* Set flag to indicate that queues are enabled */
if (v_ret == VIRTCHNL_STATUS_SUCCESS)
- set_bit(ICE_VF_STATE_ENA, vf->vf_states);
+ set_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
error_param:
/* send the response to the VF */
@@ -1915,9 +2006,11 @@ static int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg)
(struct virtchnl_queue_select *)msg;
struct ice_pf *pf = vf->pf;
struct ice_vsi *vsi;
+ unsigned long q_map;
+ u16 vf_q_id;
if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states) &&
- !test_bit(ICE_VF_STATE_ENA, vf->vf_states)) {
+ !test_bit(ICE_VF_STATE_QS_ENA, vf->vf_states)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
@@ -1932,29 +2025,81 @@ static int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg)
goto error_param;
}
+ if (vqs->rx_queues > ICE_MAX_BASE_QS_PER_VF ||
+ vqs->tx_queues > ICE_MAX_BASE_QS_PER_VF) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
vsi = pf->vsi[vf->lan_vsi_idx];
if (!vsi) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
- if (ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, vf->vf_id)) {
- dev_err(&vsi->back->pdev->dev,
- "Failed to stop tx rings on VSI %d\n",
- vsi->vsi_num);
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ if (vqs->tx_queues) {
+ q_map = vqs->tx_queues;
+
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_BASE_QS_PER_VF) {
+ struct ice_ring *ring = vsi->tx_rings[vf_q_id];
+ struct ice_txq_meta txq_meta = { 0 };
+
+ if (!ice_vc_isvalid_q_id(vf, vqs->vsi_id, vf_q_id)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* Skip queue if not enabled */
+ if (!test_bit(vf_q_id, vf->txq_ena))
+ continue;
+
+ ice_fill_txq_meta(vsi, ring, &txq_meta);
+
+ if (ice_vsi_stop_tx_ring(vsi, ICE_NO_RESET, vf->vf_id,
+ ring, &txq_meta)) {
+ dev_err(&vsi->back->pdev->dev,
+ "Failed to stop Tx ring %d on VSI %d\n",
+ vf_q_id, vsi->vsi_num);
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* Clear enabled queues flag */
+ clear_bit(vf_q_id, vf->txq_ena);
+ vf->num_qs_ena--;
+ }
}
- if (ice_vsi_stop_rx_rings(vsi)) {
- dev_err(&vsi->back->pdev->dev,
- "Failed to stop rx rings on VSI %d\n",
- vsi->vsi_num);
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ if (vqs->rx_queues) {
+ q_map = vqs->rx_queues;
+
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_BASE_QS_PER_VF) {
+ if (!ice_vc_isvalid_q_id(vf, vqs->vsi_id, vf_q_id)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* Skip queue if not enabled */
+ if (!test_bit(vf_q_id, vf->rxq_ena))
+ continue;
+
+ if (ice_vsi_ctrl_rx_ring(vsi, false, vf_q_id)) {
+ dev_err(&vsi->back->pdev->dev,
+ "Failed to stop Rx ring %d on VSI %d\n",
+ vf_q_id, vsi->vsi_num);
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* Clear enabled queues flag */
+ clear_bit(vf_q_id, vf->rxq_ena);
+ vf->num_qs_ena--;
+ }
}
/* Clear enabled queues flag */
- if (v_ret == VIRTCHNL_STATUS_SUCCESS)
- clear_bit(ICE_VF_STATE_ENA, vf->vf_states);
+ if (v_ret == VIRTCHNL_STATUS_SUCCESS && !vf->num_qs_ena)
+ clear_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
error_param:
/* send the response to the VF */
@@ -1984,12 +2129,6 @@ static int ice_vc_cfg_irq_map_msg(struct ice_vf *vf, u8 *msg)
irqmap_info = (struct virtchnl_irq_map_info *)msg;
num_q_vectors_mapped = irqmap_info->num_vectors;
- vsi = pf->vsi[vf->lan_vsi_idx];
- if (!vsi) {
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
- goto error_param;
- }
-
/* Check to make sure number of VF vectors mapped is not greater than
* number of VF vectors originally allocated, and check that
* there is actually at least a single VF queue vector mapped
@@ -2001,6 +2140,12 @@ static int ice_vc_cfg_irq_map_msg(struct ice_vf *vf, u8 *msg)
goto error_param;
}
+ vsi = pf->vsi[vf->lan_vsi_idx];
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
for (i = 0; i < num_q_vectors_mapped; i++) {
struct ice_q_vector *q_vector;
@@ -2078,6 +2223,7 @@ static int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
struct virtchnl_vsi_queue_config_info *qci =
(struct virtchnl_vsi_queue_config_info *)msg;
struct virtchnl_queue_pair_info *qpi;
+ u16 num_rxq = 0, num_txq = 0;
struct ice_pf *pf = vf->pf;
struct ice_vsi *vsi;
int i;
@@ -2093,13 +2239,16 @@ static int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
}
vsi = pf->vsi[vf->lan_vsi_idx];
- if (!vsi)
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
+ }
- if (qci->num_queue_pairs > ICE_MAX_BASE_QS_PER_VF) {
+ if (qci->num_queue_pairs > ICE_MAX_BASE_QS_PER_VF ||
+ qci->num_queue_pairs > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(&pf->pdev->dev,
"VF-%d requesting more than supported number of queues: %d\n",
- vf->vf_id, qci->num_queue_pairs);
+ vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
@@ -2109,37 +2258,52 @@ static int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
if (qpi->txq.vsi_id != qci->vsi_id ||
qpi->rxq.vsi_id != qci->vsi_id ||
qpi->rxq.queue_id != qpi->txq.queue_id ||
+ qpi->txq.headwb_enabled ||
+ !ice_vc_isvalid_ring_len(qpi->txq.ring_len) ||
+ !ice_vc_isvalid_ring_len(qpi->rxq.ring_len) ||
!ice_vc_isvalid_q_id(vf, qci->vsi_id, qpi->txq.queue_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
/* copy Tx queue info from VF into VSI */
- vsi->tx_rings[i]->dma = qpi->txq.dma_ring_addr;
- vsi->tx_rings[i]->count = qpi->txq.ring_len;
+ if (qpi->txq.ring_len > 0) {
+ num_txq++;
+ vsi->tx_rings[i]->dma = qpi->txq.dma_ring_addr;
+ vsi->tx_rings[i]->count = qpi->txq.ring_len;
+ }
+
/* copy Rx queue info from VF into VSI */
- vsi->rx_rings[i]->dma = qpi->rxq.dma_ring_addr;
- vsi->rx_rings[i]->count = qpi->rxq.ring_len;
- if (qpi->rxq.databuffer_size > ((16 * 1024) - 128)) {
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
- goto error_param;
+ if (qpi->rxq.ring_len > 0) {
+ num_rxq++;
+ vsi->rx_rings[i]->dma = qpi->rxq.dma_ring_addr;
+ vsi->rx_rings[i]->count = qpi->rxq.ring_len;
+
+ if (qpi->rxq.databuffer_size != 0 &&
+ (qpi->rxq.databuffer_size > ((16 * 1024) - 128) ||
+ qpi->rxq.databuffer_size < 1024)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+ vsi->rx_buf_len = qpi->rxq.databuffer_size;
+ vsi->rx_rings[i]->rx_buf_len = vsi->rx_buf_len;
+ if (qpi->rxq.max_pkt_size >= (16 * 1024) ||
+ qpi->rxq.max_pkt_size < 64) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
}
- vsi->rx_buf_len = qpi->rxq.databuffer_size;
- if (qpi->rxq.max_pkt_size >= (16 * 1024) ||
- qpi->rxq.max_pkt_size < 64) {
- v_ret = VIRTCHNL_STATUS_ERR_PARAM;
- goto error_param;
- }
+
vsi->max_frame = qpi->rxq.max_pkt_size;
}
/* VF can request to configure less than allocated queues
* or default allocated queues. So update the VSI with new number
*/
- vsi->num_txq = qci->num_queue_pairs;
- vsi->num_rxq = qci->num_queue_pairs;
+ vsi->num_txq = num_txq;
+ vsi->num_rxq = num_rxq;
/* All queues of VF VSI are in TC 0 */
- vsi->tc_cfg.tc_info[0].qcount_tx = qci->num_queue_pairs;
- vsi->tc_cfg.tc_info[0].qcount_rx = qci->num_queue_pairs;
+ vsi->tc_cfg.tc_info[0].qcount_tx = num_txq;
+ vsi->tc_cfg.tc_info[0].qcount_rx = num_rxq;
if (ice_vsi_cfg_lan_txqs(vsi) || ice_vsi_cfg_rxqs(vsi))
v_ret = VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR;
@@ -2193,7 +2357,7 @@ ice_vc_handle_mac_addr_msg(struct ice_vf *vf, u8 *msg, bool set)
(struct virtchnl_ether_addr_list *)msg;
struct ice_pf *pf = vf->pf;
enum virtchnl_ops vc_op;
- LIST_HEAD(mac_list);
+ enum ice_status status;
struct ice_vsi *vsi;
int mac_count = 0;
int i;
@@ -2267,33 +2431,32 @@ ice_vc_handle_mac_addr_msg(struct ice_vf *vf, u8 *msg, bool set)
goto handle_mac_exit;
}
- /* get here if maddr is multicast or if VF can change MAC */
- if (ice_add_mac_to_list(vsi, &mac_list, al->list[i].addr)) {
- v_ret = VIRTCHNL_STATUS_ERR_NO_MEMORY;
+ /* program the updated filter list */
+ status = ice_vsi_cfg_mac_fltr(vsi, maddr, set);
+ if (status == ICE_ERR_DOES_NOT_EXIST ||
+ status == ICE_ERR_ALREADY_EXISTS) {
+ dev_info(&pf->pdev->dev,
+ "can't %s MAC filters %pM for VF %d, error %d\n",
+ set ? "add" : "remove", maddr, vf->vf_id,
+ status);
+ } else if (status) {
+ dev_err(&pf->pdev->dev,
+ "can't %s MAC filters for VF %d, error %d\n",
+ set ? "add" : "remove", vf->vf_id, status);
+ v_ret = ice_err_to_virt_err(status);
goto handle_mac_exit;
}
+
mac_count++;
}
- /* program the updated filter list */
+ /* Track number of MAC filters programmed for the VF VSI */
if (set)
- v_ret = ice_err_to_virt_err(ice_add_mac(&pf->hw, &mac_list));
+ vf->num_mac += mac_count;
else
- v_ret = ice_err_to_virt_err(ice_remove_mac(&pf->hw, &mac_list));
-
- if (v_ret) {
- dev_err(&pf->pdev->dev,
- "can't %s MAC filters for VF %d, error %d\n",
- set ? "add" : "remove", vf->vf_id, v_ret);
- } else {
- if (set)
- vf->num_mac += mac_count;
- else
- vf->num_mac -= mac_count;
- }
+ vf->num_mac -= mac_count;
handle_mac_exit:
- ice_free_fltr_list(&pf->pdev->dev, &mac_list);
/* send the response to the VF */
return ice_vc_send_msg_to_vf(vf, vc_op, v_ret, NULL, 0);
}
@@ -2349,7 +2512,8 @@ static int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
}
cur_queues = vf->num_vf_qs;
- tx_rx_queue_left = min_t(u16, pf->q_left_tx, pf->q_left_rx);
+ tx_rx_queue_left = min_t(u16, ice_get_avail_txq_count(pf),
+ ice_get_avail_rxq_count(pf));
max_allowed_vf_queues = tx_rx_queue_left + cur_queues;
if (!req_queues) {
dev_err(&pf->pdev->dev,
@@ -2612,8 +2776,9 @@ static int ice_vc_process_vlan_msg(struct ice_vf *vf, u8 *msg, bool add_v)
}
vf->num_vlan--;
- /* Disable VLAN pruning when removing VLAN */
- ice_cfg_vlan_pruning(vsi, false, false);
+ /* Disable VLAN pruning when the last VLAN is removed */
+ if (!vf->num_vlan)
+ ice_cfg_vlan_pruning(vsi, false, false);
/* Disable Unicast/Multicast VLAN promiscuous mode */
if (vlan_promisc) {
@@ -2754,20 +2919,6 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event)
err = -EPERM;
else
err = -EINVAL;
- goto error_handler;
- }
-
- /* Perform additional checks specific to RSS and Virtchnl */
- if (v_opcode == VIRTCHNL_OP_CONFIG_RSS_KEY) {
- struct virtchnl_rss_key *vrk = (struct virtchnl_rss_key *)msg;
-
- if (vrk->key_len != ICE_VSIQF_HKEY_ARRAY_SIZE)
- err = -EINVAL;
- } else if (v_opcode == VIRTCHNL_OP_CONFIG_RSS_LUT) {
- struct virtchnl_rss_lut *vrl = (struct virtchnl_rss_lut *)msg;
-
- if (vrl->lut_entries != ICE_VSIQF_HLUT_ARRAY_SIZE)
- err = -EINVAL;
}
error_handler:
@@ -2785,6 +2936,7 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event)
break;
case VIRTCHNL_OP_GET_VF_RESOURCES:
err = ice_vc_get_vf_res_msg(vf, msg);
+ ice_vc_notify_vf_link_state(vf);
break;
case VIRTCHNL_OP_RESET_VF:
ice_vc_reset_vf_msg(vf);
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h
index 4d94853..0d9880c 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h
@@ -15,8 +15,8 @@
#define ICE_MAX_MACADDR_PER_VF 12
/* Malicious Driver Detection */
-#define ICE_DFLT_NUM_MDD_EVENTS_ALLOWED 3
#define ICE_DFLT_NUM_INVAL_MSGS_ALLOWED 10
+#define ICE_MDD_EVENTS_THRESHOLD 30
/* Static VF transaction/status register def */
#define VF_DEVICE_STATUS 0xAA
@@ -41,9 +41,9 @@
/* Specific VF states */
enum ice_vf_states {
- ICE_VF_STATE_INIT = 0,
- ICE_VF_STATE_ACTIVE,
- ICE_VF_STATE_ENA,
+ ICE_VF_STATE_INIT = 0, /* PF is initializing VF */
+ ICE_VF_STATE_ACTIVE, /* VF resources are allocated for use */
+ ICE_VF_STATE_QS_ENA, /* VF queue(s) enabled */
ICE_VF_STATE_DIS,
ICE_VF_STATE_MC_PROMISC,
ICE_VF_STATE_UC_PROMISC,
@@ -68,6 +68,8 @@ struct ice_vf {
struct virtchnl_version_info vf_ver;
u32 driver_caps; /* reported by VF driver */
struct virtchnl_ether_addr dflt_lan_addr;
+ DECLARE_BITMAP(txq_ena, ICE_MAX_BASE_QS_PER_VF);
+ DECLARE_BITMAP(rxq_ena, ICE_MAX_BASE_QS_PER_VF);
u16 port_vlan_id;
u8 pf_set_mac:1; /* VF MAC address set by VMM admin */
u8 trusted:1;
@@ -90,6 +92,7 @@ struct ice_vf {
u16 num_mac;
u16 num_vlan;
u16 num_vf_qs; /* num of queue configured per VF */
+ u16 num_qs_ena; /* total num of Tx/Rx queue enabled */
};
#ifdef CONFIG_PCI_IOV
@@ -116,12 +119,15 @@ int ice_set_vf_link_state(struct net_device *netdev, int vf_id, int link_state);
int ice_set_vf_spoofchk(struct net_device *netdev, int vf_id, bool ena);
int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector);
+
+void ice_set_vf_state_qs_dis(struct ice_vf *vf);
#else /* CONFIG_PCI_IOV */
#define ice_process_vflr_event(pf) do {} while (0)
#define ice_free_vfs(pf) do {} while (0)
#define ice_vc_process_vf_msg(pf, event) do {} while (0)
#define ice_vc_notify_link_state(pf) do {} while (0)
#define ice_vc_notify_reset(pf) do {} while (0)
+#define ice_set_vf_state_qs_dis(vf) do {} while (0)
static inline bool
ice_reset_all_vfs(struct ice_pf __always_unused *pf,
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index b63e775..105b062 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -4731,8 +4731,7 @@ static void igb_clean_rx_ring(struct igb_ring *rx_ring)
{
u16 i = rx_ring->next_to_clean;
- if (rx_ring->skb)
- dev_kfree_skb(rx_ring->skb);
+ dev_kfree_skb(rx_ring->skb);
rx_ring->skb = NULL;
/* Free all the Rx ring sk_buffs */
diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h
index 0f5534c..7e16345 100644
--- a/drivers/net/ethernet/intel/igc/igc.h
+++ b/drivers/net/ethernet/intel/igc/igc.h
@@ -135,6 +135,9 @@ extern char igc_driver_version[];
/* How many Rx Buffers do we bundle into one write to the hardware ? */
#define IGC_RX_BUFFER_WRITE 16 /* Must be power of 2 */
+/* VLAN info */
+#define IGC_TX_FLAGS_VLAN_MASK 0xffff0000
+
/* igc_test_staterr - tests bits within Rx descriptor status and error fields */
static inline __le32 igc_test_staterr(union igc_adv_rx_desc *rx_desc,
const u32 stat_err_bits)
@@ -254,6 +257,7 @@ struct igc_ring {
u16 count; /* number of desc. in the ring */
u8 queue_index; /* logical index of the ring*/
u8 reg_idx; /* physical index of the ring */
+ bool launchtime_enable; /* true if LaunchTime is enabled */
/* everything past this point are written often */
u16 next_to_clean;
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
index 58d1109..ea627ce 100644
--- a/drivers/net/ethernet/intel/igc/igc_base.h
+++ b/drivers/net/ethernet/intel/igc/igc_base.h
@@ -22,6 +22,14 @@ union igc_adv_tx_desc {
} wb;
};
+/* Context descriptors */
+struct igc_adv_tx_context_desc {
+ __le32 vlan_macip_lens;
+ __le32 launch_time;
+ __le32 type_tucmd_mlhl;
+ __le32 mss_l4len_idx;
+};
+
/* Adv Transmit Descriptor Config Masks */
#define IGC_ADVTXD_MAC_TSTAMP 0x00080000 /* IEEE1588 Timestamp packet */
#define IGC_ADVTXD_DTYP_CTXT 0x00200000 /* Advanced Context Descriptor */
diff --git a/drivers/net/ethernet/intel/igc/igc_defines.h b/drivers/net/ethernet/intel/igc/igc_defines.h
index 11b99ac..f3f2325 100644
--- a/drivers/net/ethernet/intel/igc/igc_defines.h
+++ b/drivers/net/ethernet/intel/igc/igc_defines.h
@@ -10,10 +10,6 @@
#define IGC_CTRL_EXT_DRV_LOAD 0x10000000 /* Drv loaded bit for FW */
-/* PCI Bus Info */
-#define PCIE_DEVICE_CONTROL2 0x28
-#define PCIE_DEVICE_CONTROL2_16ms 0x0005
-
/* Physical Func Reset Done Indication */
#define IGC_CTRL_EXT_LINK_MODE_MASK 0x00C00000
@@ -401,4 +397,9 @@
#define IGC_VLAPQF_P_VALID(_n) (0x1 << (3 + (_n) * 4))
#define IGC_VLAPQF_QUEUE_MASK 0x03
+#define IGC_ADVTXD_MACLEN_SHIFT 9 /* Adv ctxt desc mac len shift */
+#define IGC_ADVTXD_TUCMD_IPV4 0x00000400 /* IP Packet Type:1=IPv4 */
+#define IGC_ADVTXD_TUCMD_L4T_TCP 0x00000800 /* L4 Packet Type of TCP */
+#define IGC_ADVTXD_TUCMD_L4T_SCTP 0x00001000 /* L4 packet TYPE of SCTP */
+
#endif /* _IGC_DEFINES_H_ */
diff --git a/drivers/net/ethernet/intel/igc/igc_mac.c b/drivers/net/ethernet/intel/igc/igc_mac.c
index ba46467..5eeb4c8 100644
--- a/drivers/net/ethernet/intel/igc/igc_mac.c
+++ b/drivers/net/ethernet/intel/igc/igc_mac.c
@@ -7,9 +7,6 @@
#include "igc_mac.h"
#include "igc_hw.h"
-/* forward declaration */
-static s32 igc_set_fc_watermarks(struct igc_hw *hw);
-
/**
* igc_disable_pcie_master - Disables PCI-express master access
* @hw: pointer to the HW structure
@@ -75,6 +72,41 @@ void igc_init_rx_addrs(struct igc_hw *hw, u16 rar_count)
}
/**
+ * igc_set_fc_watermarks - Set flow control high/low watermarks
+ * @hw: pointer to the HW structure
+ *
+ * Sets the flow control high/low threshold (watermark) registers. If
+ * flow control XON frame transmission is enabled, then set XON frame
+ * transmission as well.
+ */
+static s32 igc_set_fc_watermarks(struct igc_hw *hw)
+{
+ u32 fcrtl = 0, fcrth = 0;
+
+ /* Set the flow control receive threshold registers. Normally,
+ * these registers will be set to a default threshold that may be
+ * adjusted later by the driver's runtime code. However, if the
+ * ability to transmit pause frames is not enabled, then these
+ * registers will be set to 0.
+ */
+ if (hw->fc.current_mode & igc_fc_tx_pause) {
+ /* We need to set up the Receive Threshold high and low water
+ * marks as well as (optionally) enabling the transmission of
+ * XON frames.
+ */
+ fcrtl = hw->fc.low_water;
+ if (hw->fc.send_xon)
+ fcrtl |= IGC_FCRTL_XONE;
+
+ fcrth = hw->fc.high_water;
+ }
+ wr32(IGC_FCRTL, fcrtl);
+ wr32(IGC_FCRTH, fcrth);
+
+ return 0;
+}
+
+/**
* igc_setup_link - Setup flow control and link settings
* @hw: pointer to the HW structure
*
@@ -195,41 +227,6 @@ s32 igc_force_mac_fc(struct igc_hw *hw)
}
/**
- * igc_set_fc_watermarks - Set flow control high/low watermarks
- * @hw: pointer to the HW structure
- *
- * Sets the flow control high/low threshold (watermark) registers. If
- * flow control XON frame transmission is enabled, then set XON frame
- * transmission as well.
- */
-static s32 igc_set_fc_watermarks(struct igc_hw *hw)
-{
- u32 fcrtl = 0, fcrth = 0;
-
- /* Set the flow control receive threshold registers. Normally,
- * these registers will be set to a default threshold that may be
- * adjusted later by the driver's runtime code. However, if the
- * ability to transmit pause frames is not enabled, then these
- * registers will be set to 0.
- */
- if (hw->fc.current_mode & igc_fc_tx_pause) {
- /* We need to set up the Receive Threshold high and low water
- * marks as well as (optionally) enabling the transmission of
- * XON frames.
- */
- fcrtl = hw->fc.low_water;
- if (hw->fc.send_xon)
- fcrtl |= IGC_FCRTL_XONE;
-
- fcrth = hw->fc.high_water;
- }
- wr32(IGC_FCRTL, fcrtl);
- wr32(IGC_FCRTH, fcrth);
-
- return 0;
-}
-
-/**
* igc_clear_hw_cntrs_base - Clear base hardware counters
* @hw: pointer to the HW structure
*
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index e5114be..63b62d7 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -5,6 +5,11 @@
#include <linux/types.h>
#include <linux/if_vlan.h>
#include <linux/aer.h>
+#include <linux/tcp.h>
+#include <linux/udp.h>
+#include <linux/ip.h>
+
+#include <net/ipv6.h>
#include "igc.h"
#include "igc_hw.h"
@@ -352,8 +357,7 @@ static void igc_clean_rx_ring(struct igc_ring *rx_ring)
{
u16 i = rx_ring->next_to_clean;
- if (rx_ring->skb)
- dev_kfree_skb(rx_ring->skb);
+ dev_kfree_skb(rx_ring->skb);
rx_ring->skb = NULL;
/* Free all the Rx ring sk_buffs */
@@ -791,8 +795,96 @@ static int igc_set_mac(struct net_device *netdev, void *p)
return 0;
}
+static void igc_tx_ctxtdesc(struct igc_ring *tx_ring,
+ struct igc_tx_buffer *first,
+ u32 vlan_macip_lens, u32 type_tucmd,
+ u32 mss_l4len_idx)
+{
+ struct igc_adv_tx_context_desc *context_desc;
+ u16 i = tx_ring->next_to_use;
+ struct timespec64 ts;
+
+ context_desc = IGC_TX_CTXTDESC(tx_ring, i);
+
+ i++;
+ tx_ring->next_to_use = (i < tx_ring->count) ? i : 0;
+
+ /* set bits to identify this as an advanced context descriptor */
+ type_tucmd |= IGC_TXD_CMD_DEXT | IGC_ADVTXD_DTYP_CTXT;
+
+ /* For 82575, context index must be unique per ring. */
+ if (test_bit(IGC_RING_FLAG_TX_CTX_IDX, &tx_ring->flags))
+ mss_l4len_idx |= tx_ring->reg_idx << 4;
+
+ context_desc->vlan_macip_lens = cpu_to_le32(vlan_macip_lens);
+ context_desc->type_tucmd_mlhl = cpu_to_le32(type_tucmd);
+ context_desc->mss_l4len_idx = cpu_to_le32(mss_l4len_idx);
+
+ /* We assume there is always a valid Tx time available. Invalid times
+ * should have been handled by the upper layers.
+ */
+ if (tx_ring->launchtime_enable) {
+ ts = ns_to_timespec64(first->skb->tstamp);
+ first->skb->tstamp = 0;
+ context_desc->launch_time = cpu_to_le32(ts.tv_nsec / 32);
+ } else {
+ context_desc->launch_time = 0;
+ }
+}
+
+static inline bool igc_ipv6_csum_is_sctp(struct sk_buff *skb)
+{
+ unsigned int offset = 0;
+
+ ipv6_find_hdr(skb, &offset, IPPROTO_SCTP, NULL, NULL);
+
+ return offset == skb_checksum_start_offset(skb);
+}
+
static void igc_tx_csum(struct igc_ring *tx_ring, struct igc_tx_buffer *first)
{
+ struct sk_buff *skb = first->skb;
+ u32 vlan_macip_lens = 0;
+ u32 type_tucmd = 0;
+
+ if (skb->ip_summed != CHECKSUM_PARTIAL) {
+csum_failed:
+ if (!(first->tx_flags & IGC_TX_FLAGS_VLAN) &&
+ !tx_ring->launchtime_enable)
+ return;
+ goto no_csum;
+ }
+
+ switch (skb->csum_offset) {
+ case offsetof(struct tcphdr, check):
+ type_tucmd = IGC_ADVTXD_TUCMD_L4T_TCP;
+ /* fall through */
+ case offsetof(struct udphdr, check):
+ break;
+ case offsetof(struct sctphdr, checksum):
+ /* validate that this is actually an SCTP request */
+ if ((first->protocol == htons(ETH_P_IP) &&
+ (ip_hdr(skb)->protocol == IPPROTO_SCTP)) ||
+ (first->protocol == htons(ETH_P_IPV6) &&
+ igc_ipv6_csum_is_sctp(skb))) {
+ type_tucmd = IGC_ADVTXD_TUCMD_L4T_SCTP;
+ break;
+ }
+ /* fall through */
+ default:
+ skb_checksum_help(skb);
+ goto csum_failed;
+ }
+
+ /* update TX checksum flag */
+ first->tx_flags |= IGC_TX_FLAGS_CSUM;
+ vlan_macip_lens = skb_checksum_start_offset(skb) -
+ skb_network_offset(skb);
+no_csum:
+ vlan_macip_lens |= skb_network_offset(skb) << IGC_ADVTXD_MACLEN_SHIFT;
+ vlan_macip_lens |= first->tx_flags & IGC_TX_FLAGS_VLAN_MASK;
+
+ igc_tx_ctxtdesc(tx_ring, first, vlan_macip_lens, type_tucmd, 0);
}
static int __igc_maybe_stop_tx(struct igc_ring *tx_ring, const u16 size)
@@ -4117,6 +4209,9 @@ static int igc_probe(struct pci_dev *pdev,
if (err)
goto err_sw_init;
+ /* Add supported features to the features list*/
+ netdev->features |= NETIF_F_HW_CSUM;
+
/* setup the private structure */
err = igc_sw_init(adapter);
if (err)
@@ -4124,6 +4219,7 @@ static int igc_probe(struct pci_dev *pdev,
/* copy netdev features into list of user selectable features */
netdev->hw_features |= NETIF_F_NTUPLE;
+ netdev->hw_features |= netdev->features;
/* MTU range: 68 - 9216 */
netdev->min_mtu = ETH_MIN_MTU;
@@ -4134,6 +4230,15 @@ static int igc_probe(struct pci_dev *pdev,
*/
hw->mac.ops.reset_hw(hw);
+ if (igc_get_flash_presence_i225(hw)) {
+ if (hw->nvm.ops.validate(hw) < 0) {
+ dev_err(&pdev->dev,
+ "The NVM Checksum Is Not Valid\n");
+ err = -EIO;
+ goto err_eeprom;
+ }
+ }
+
if (eth_platform_get_mac_address(&pdev->dev, hw->mac.addr)) {
/* copy the MAC address out of the NVM */
if (hw->mac.ops.read_mac_addr(hw))
diff --git a/drivers/net/ethernet/intel/igc/igc_phy.c b/drivers/net/ethernet/intel/igc/igc_phy.c
index 4c8f96a9..f4b05af 100644
--- a/drivers/net/ethernet/intel/igc/igc_phy.c
+++ b/drivers/net/ethernet/intel/igc/igc_phy.c
@@ -3,10 +3,6 @@
#include "igc_phy.h"
-/* forward declaration */
-static s32 igc_phy_setup_autoneg(struct igc_hw *hw);
-static s32 igc_wait_autoneg(struct igc_hw *hw);
-
/**
* igc_check_reset_block - Check if PHY reset is blocked
* @hw: pointer to the HW structure
@@ -208,100 +204,6 @@ s32 igc_phy_hw_reset(struct igc_hw *hw)
}
/**
- * igc_copper_link_autoneg - Setup/Enable autoneg for copper link
- * @hw: pointer to the HW structure
- *
- * Performs initial bounds checking on autoneg advertisement parameter, then
- * configure to advertise the full capability. Setup the PHY to autoneg
- * and restart the negotiation process between the link partner. If
- * autoneg_wait_to_complete, then wait for autoneg to complete before exiting.
- */
-static s32 igc_copper_link_autoneg(struct igc_hw *hw)
-{
- struct igc_phy_info *phy = &hw->phy;
- u16 phy_ctrl;
- s32 ret_val;
-
- /* Perform some bounds checking on the autoneg advertisement
- * parameter.
- */
- phy->autoneg_advertised &= phy->autoneg_mask;
-
- /* If autoneg_advertised is zero, we assume it was not defaulted
- * by the calling code so we set to advertise full capability.
- */
- if (phy->autoneg_advertised == 0)
- phy->autoneg_advertised = phy->autoneg_mask;
-
- hw_dbg("Reconfiguring auto-neg advertisement params\n");
- ret_val = igc_phy_setup_autoneg(hw);
- if (ret_val) {
- hw_dbg("Error Setting up Auto-Negotiation\n");
- goto out;
- }
- hw_dbg("Restarting Auto-Neg\n");
-
- /* Restart auto-negotiation by setting the Auto Neg Enable bit and
- * the Auto Neg Restart bit in the PHY control register.
- */
- ret_val = phy->ops.read_reg(hw, PHY_CONTROL, &phy_ctrl);
- if (ret_val)
- goto out;
-
- phy_ctrl |= (MII_CR_AUTO_NEG_EN | MII_CR_RESTART_AUTO_NEG);
- ret_val = phy->ops.write_reg(hw, PHY_CONTROL, phy_ctrl);
- if (ret_val)
- goto out;
-
- /* Does the user want to wait for Auto-Neg to complete here, or
- * check at a later time (for example, callback routine).
- */
- if (phy->autoneg_wait_to_complete) {
- ret_val = igc_wait_autoneg(hw);
- if (ret_val) {
- hw_dbg("Error while waiting for autoneg to complete\n");
- goto out;
- }
- }
-
- hw->mac.get_link_status = true;
-
-out:
- return ret_val;
-}
-
-/**
- * igc_wait_autoneg - Wait for auto-neg completion
- * @hw: pointer to the HW structure
- *
- * Waits for auto-negotiation to complete or for the auto-negotiation time
- * limit to expire, which ever happens first.
- */
-static s32 igc_wait_autoneg(struct igc_hw *hw)
-{
- u16 i, phy_status;
- s32 ret_val = 0;
-
- /* Break after autoneg completes or PHY_AUTO_NEG_LIMIT expires. */
- for (i = PHY_AUTO_NEG_LIMIT; i > 0; i--) {
- ret_val = hw->phy.ops.read_reg(hw, PHY_STATUS, &phy_status);
- if (ret_val)
- break;
- ret_val = hw->phy.ops.read_reg(hw, PHY_STATUS, &phy_status);
- if (ret_val)
- break;
- if (phy_status & MII_SR_AUTONEG_COMPLETE)
- break;
- msleep(100);
- }
-
- /* PHY_AUTO_NEG_TIME expiration doesn't guarantee auto-negotiation
- * has completed.
- */
- return ret_val;
-}
-
-/**
* igc_phy_setup_autoneg - Configure PHY for auto-negotiation
* @hw: pointer to the HW structure
*
@@ -486,6 +388,100 @@ static s32 igc_phy_setup_autoneg(struct igc_hw *hw)
}
/**
+ * igc_wait_autoneg - Wait for auto-neg completion
+ * @hw: pointer to the HW structure
+ *
+ * Waits for auto-negotiation to complete or for the auto-negotiation time
+ * limit to expire, which ever happens first.
+ */
+static s32 igc_wait_autoneg(struct igc_hw *hw)
+{
+ u16 i, phy_status;
+ s32 ret_val = 0;
+
+ /* Break after autoneg completes or PHY_AUTO_NEG_LIMIT expires. */
+ for (i = PHY_AUTO_NEG_LIMIT; i > 0; i--) {
+ ret_val = hw->phy.ops.read_reg(hw, PHY_STATUS, &phy_status);
+ if (ret_val)
+ break;
+ ret_val = hw->phy.ops.read_reg(hw, PHY_STATUS, &phy_status);
+ if (ret_val)
+ break;
+ if (phy_status & MII_SR_AUTONEG_COMPLETE)
+ break;
+ msleep(100);
+ }
+
+ /* PHY_AUTO_NEG_TIME expiration doesn't guarantee auto-negotiation
+ * has completed.
+ */
+ return ret_val;
+}
+
+/**
+ * igc_copper_link_autoneg - Setup/Enable autoneg for copper link
+ * @hw: pointer to the HW structure
+ *
+ * Performs initial bounds checking on autoneg advertisement parameter, then
+ * configure to advertise the full capability. Setup the PHY to autoneg
+ * and restart the negotiation process between the link partner. If
+ * autoneg_wait_to_complete, then wait for autoneg to complete before exiting.
+ */
+static s32 igc_copper_link_autoneg(struct igc_hw *hw)
+{
+ struct igc_phy_info *phy = &hw->phy;
+ u16 phy_ctrl;
+ s32 ret_val;
+
+ /* Perform some bounds checking on the autoneg advertisement
+ * parameter.
+ */
+ phy->autoneg_advertised &= phy->autoneg_mask;
+
+ /* If autoneg_advertised is zero, we assume it was not defaulted
+ * by the calling code so we set to advertise full capability.
+ */
+ if (phy->autoneg_advertised == 0)
+ phy->autoneg_advertised = phy->autoneg_mask;
+
+ hw_dbg("Reconfiguring auto-neg advertisement params\n");
+ ret_val = igc_phy_setup_autoneg(hw);
+ if (ret_val) {
+ hw_dbg("Error Setting up Auto-Negotiation\n");
+ goto out;
+ }
+ hw_dbg("Restarting Auto-Neg\n");
+
+ /* Restart auto-negotiation by setting the Auto Neg Enable bit and
+ * the Auto Neg Restart bit in the PHY control register.
+ */
+ ret_val = phy->ops.read_reg(hw, PHY_CONTROL, &phy_ctrl);
+ if (ret_val)
+ goto out;
+
+ phy_ctrl |= (MII_CR_AUTO_NEG_EN | MII_CR_RESTART_AUTO_NEG);
+ ret_val = phy->ops.write_reg(hw, PHY_CONTROL, phy_ctrl);
+ if (ret_val)
+ goto out;
+
+ /* Does the user want to wait for Auto-Neg to complete here, or
+ * check at a later time (for example, callback routine).
+ */
+ if (phy->autoneg_wait_to_complete) {
+ ret_val = igc_wait_autoneg(hw);
+ if (ret_val) {
+ hw_dbg("Error while waiting for autoneg to complete\n");
+ goto out;
+ }
+ }
+
+ hw->mac.get_link_status = true;
+
+out:
+ return ret_val;
+}
+
+/**
* igc_setup_copper_link - Configure copper link settings
* @hw: pointer to the HW structure
*
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
index 31629fc..113f608 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
@@ -960,11 +960,9 @@ int ixgbe_ipsec_vf_add_sa(struct ixgbe_adapter *adapter, u32 *msgbuf, u32 vf)
return 0;
err_aead:
- memset(xs->aead, 0, sizeof(*xs->aead));
- kfree(xs->aead);
+ kzfree(xs->aead);
err_xs:
- memset(xs, 0, sizeof(*xs));
- kfree(xs);
+ kzfree(xs);
err_out:
msgbuf[1] = err;
return err;
@@ -1049,8 +1047,7 @@ int ixgbe_ipsec_vf_del_sa(struct ixgbe_adapter *adapter, u32 *msgbuf, u32 vf)
ixgbe_ipsec_del_sa(xs);
/* remove the xs that was made-up in the add request */
- memset(xs, 0, sizeof(*xs));
- kfree(xs);
+ kzfree(xs);
return 0;
}
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 17b7ae9..99df595 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -1825,13 +1825,7 @@ static void ixgbe_pull_tail(struct ixgbe_ring *rx_ring,
static void ixgbe_dma_sync_frag(struct ixgbe_ring *rx_ring,
struct sk_buff *skb)
{
- /* if the page was released unmap it, else just sync our portion */
- if (unlikely(IXGBE_CB(skb)->page_released)) {
- dma_unmap_page_attrs(rx_ring->dev, IXGBE_CB(skb)->dma,
- ixgbe_rx_pg_size(rx_ring),
- DMA_FROM_DEVICE,
- IXGBE_RX_DMA_ATTR);
- } else if (ring_uses_build_skb(rx_ring)) {
+ if (ring_uses_build_skb(rx_ring)) {
unsigned long offset = (unsigned long)(skb->data) & ~PAGE_MASK;
dma_sync_single_range_for_cpu(rx_ring->dev,
@@ -1848,6 +1842,14 @@ static void ixgbe_dma_sync_frag(struct ixgbe_ring *rx_ring,
skb_frag_size(frag),
DMA_FROM_DEVICE);
}
+
+ /* If the page was released, just unmap it. */
+ if (unlikely(IXGBE_CB(skb)->page_released)) {
+ dma_unmap_page_attrs(rx_ring->dev, IXGBE_CB(skb)->dma,
+ ixgbe_rx_pg_size(rx_ring),
+ DMA_FROM_DEVICE,
+ IXGBE_RX_DMA_ATTR);
+ }
}
/**
@@ -10260,7 +10262,8 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
if (need_reset && prog)
for (i = 0; i < adapter->num_rx_queues; i++)
if (adapter->xdp_ring[i]->xsk_umem)
- (void)ixgbe_xsk_async_xmit(adapter->netdev, i);
+ (void)ixgbe_xsk_wakeup(adapter->netdev, i,
+ XDP_WAKEUP_RX);
return 0;
}
@@ -10379,7 +10382,7 @@ static const struct net_device_ops ixgbe_netdev_ops = {
.ndo_features_check = ixgbe_features_check,
.ndo_bpf = ixgbe_xdp,
.ndo_xdp_xmit = ixgbe_xdp_xmit,
- .ndo_xsk_async_xmit = ixgbe_xsk_async_xmit,
+ .ndo_xsk_wakeup = ixgbe_xsk_wakeup,
};
static void ixgbe_disable_txr_hw(struct ixgbe_adapter *adapter,
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h
index d93a690..6d01700 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_txrx_common.h
@@ -42,7 +42,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
void ixgbe_xsk_clean_rx_ring(struct ixgbe_ring *rx_ring);
bool ixgbe_clean_xdp_tx_irq(struct ixgbe_q_vector *q_vector,
struct ixgbe_ring *tx_ring, int napi_budget);
-int ixgbe_xsk_async_xmit(struct net_device *dev, u32 queue_id);
+int ixgbe_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags);
void ixgbe_xsk_clean_tx_ring(struct ixgbe_ring *tx_ring);
#endif /* #define _IXGBE_TXRX_COMMON_H_ */
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index 6b60955..ad802a8 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -100,7 +100,7 @@ static int ixgbe_xsk_umem_enable(struct ixgbe_adapter *adapter,
ixgbe_txrx_ring_enable(adapter, qid);
/* Kick start the NAPI context so that receiving will start */
- err = ixgbe_xsk_async_xmit(adapter->netdev, qid);
+ err = ixgbe_xsk_wakeup(adapter->netdev, qid, XDP_WAKEUP_RX);
if (err)
return err;
}
@@ -143,7 +143,9 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
struct ixgbe_ring *rx_ring,
struct xdp_buff *xdp)
{
+ struct xdp_umem *umem = rx_ring->xsk_umem;
int err, result = IXGBE_XDP_PASS;
+ u64 offset = umem->headroom;
struct bpf_prog *xdp_prog;
struct xdp_frame *xdpf;
u32 act;
@@ -151,7 +153,10 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
rcu_read_lock();
xdp_prog = READ_ONCE(rx_ring->xdp_prog);
act = bpf_prog_run_xdp(xdp_prog, xdp);
- xdp->handle += xdp->data - xdp->data_hard_start;
+ offset += xdp->data - xdp->data_hard_start;
+
+ xdp->handle = xsk_umem_adjust_offset(umem, xdp->handle, offset);
+
switch (act) {
case XDP_PASS:
break;
@@ -201,8 +206,6 @@ ixgbe_rx_buffer *ixgbe_get_rx_buffer_zc(struct ixgbe_ring *rx_ring,
static void ixgbe_reuse_rx_buffer_zc(struct ixgbe_ring *rx_ring,
struct ixgbe_rx_buffer *obi)
{
- unsigned long mask = (unsigned long)rx_ring->xsk_umem->chunk_mask;
- u64 hr = rx_ring->xsk_umem->headroom + XDP_PACKET_HEADROOM;
u16 nta = rx_ring->next_to_alloc;
struct ixgbe_rx_buffer *nbi;
@@ -212,14 +215,9 @@ static void ixgbe_reuse_rx_buffer_zc(struct ixgbe_ring *rx_ring,
rx_ring->next_to_alloc = (nta < rx_ring->count) ? nta : 0;
/* transfer page from old buffer to new buffer */
- nbi->dma = obi->dma & mask;
- nbi->dma += hr;
-
- nbi->addr = (void *)((unsigned long)obi->addr & mask);
- nbi->addr += hr;
-
- nbi->handle = obi->handle & mask;
- nbi->handle += rx_ring->xsk_umem->headroom;
+ nbi->dma = obi->dma;
+ nbi->addr = obi->addr;
+ nbi->handle = obi->handle;
obi->addr = NULL;
obi->skb = NULL;
@@ -250,7 +248,8 @@ void ixgbe_zca_free(struct zero_copy_allocator *alloc, unsigned long handle)
bi->addr = xdp_umem_get_data(rx_ring->xsk_umem, handle);
bi->addr += hr;
- bi->handle = (u64)handle + rx_ring->xsk_umem->headroom;
+ bi->handle = xsk_umem_adjust_offset(rx_ring->xsk_umem, (u64)handle,
+ rx_ring->xsk_umem->headroom);
}
static bool ixgbe_alloc_buffer_zc(struct ixgbe_ring *rx_ring,
@@ -276,7 +275,7 @@ static bool ixgbe_alloc_buffer_zc(struct ixgbe_ring *rx_ring,
bi->addr = xdp_umem_get_data(umem, handle);
bi->addr += hr;
- bi->handle = handle + umem->headroom;
+ bi->handle = xsk_umem_adjust_offset(umem, handle, umem->headroom);
xsk_umem_discard_addr(umem);
return true;
@@ -303,7 +302,7 @@ static bool ixgbe_alloc_buffer_slow_zc(struct ixgbe_ring *rx_ring,
bi->addr = xdp_umem_get_data(umem, handle);
bi->addr += hr;
- bi->handle = handle + umem->headroom;
+ bi->handle = xsk_umem_adjust_offset(umem, handle, umem->headroom);
xsk_umem_discard_addr_rq(umem);
return true;
@@ -547,6 +546,14 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
q_vector->rx.total_packets += total_rx_packets;
q_vector->rx.total_bytes += total_rx_bytes;
+ if (xsk_umem_uses_need_wakeup(rx_ring->xsk_umem)) {
+ if (failure || rx_ring->next_to_clean == rx_ring->next_to_use)
+ xsk_set_rx_need_wakeup(rx_ring->xsk_umem);
+ else
+ xsk_clear_rx_need_wakeup(rx_ring->xsk_umem);
+
+ return (int)total_rx_packets;
+ }
return failure ? budget : (int)total_rx_packets;
}
@@ -615,6 +622,8 @@ static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget)
if (tx_desc) {
ixgbe_xdp_ring_update_tail(xdp_ring);
xsk_umem_consume_tx_done(xdp_ring->xsk_umem);
+ if (xsk_umem_uses_need_wakeup(xdp_ring->xsk_umem))
+ xsk_clear_tx_need_wakeup(xdp_ring->xsk_umem);
}
return !!budget && work_done;
@@ -688,11 +697,19 @@ bool ixgbe_clean_xdp_tx_irq(struct ixgbe_q_vector *q_vector,
if (xsk_frames)
xsk_umem_complete_tx(umem, xsk_frames);
+ if (xsk_umem_uses_need_wakeup(tx_ring->xsk_umem)) {
+ if (tx_ring->next_to_clean == tx_ring->next_to_use)
+ xsk_set_tx_need_wakeup(tx_ring->xsk_umem);
+ else
+ xsk_clear_tx_need_wakeup(tx_ring->xsk_umem);
+ }
+
xmit_done = ixgbe_xmit_zc(tx_ring, q_vector->tx.work_limit);
+
return budget > 0 && xmit_done;
}
-int ixgbe_xsk_async_xmit(struct net_device *dev, u32 qid)
+int ixgbe_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags)
{
struct ixgbe_adapter *adapter = netdev_priv(dev);
struct ixgbe_ring *ring;
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 8c011d4..75e849a6 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -2517,6 +2517,7 @@ void ixgbevf_reinit_locked(struct ixgbevf_adapter *adapter)
msleep(1);
ixgbevf_down(adapter);
+ pci_set_master(adapter->pdev);
ixgbevf_up(adapter);
clear_bit(__IXGBEVF_RESETTING, &adapter->state);
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
index ee3bab5..543a310 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
@@ -683,6 +683,7 @@ enum mvpp2_prs_l3_cast {
#define MVPP2_BM_SHORT_BUF_NUM 2048
#define MVPP2_BM_POOL_SIZE_MAX (16*1024 - MVPP2_BM_POOL_PTR_ALIGN/4)
#define MVPP2_BM_POOL_PTR_ALIGN 128
+#define MVPP2_BM_MAX_POOLS 8
/* BM cookie (32 bits) definition */
#define MVPP2_BM_COOKIE_POOL_OFFS 8
@@ -787,6 +788,9 @@ struct mvpp2 {
/* Aggregated TXQs */
struct mvpp2_tx_queue *aggr_txqs;
+ /* Are we using page_pool with per-cpu pools? */
+ int percpu_pools;
+
/* BM pools */
struct mvpp2_bm_pool *bm_pools;
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
index 12e799e..111b3b8 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
@@ -292,6 +292,26 @@ static void mvpp2_txq_inc_put(struct mvpp2_port *port,
txq_pcpu->txq_put_index = 0;
}
+/* Get number of maximum RXQ */
+static int mvpp2_get_nrxqs(struct mvpp2 *priv)
+{
+ unsigned int nrxqs;
+
+ if (priv->hw_version == MVPP22 && queue_mode == MVPP2_QDIST_SINGLE_MODE)
+ return 1;
+
+ /* According to the PPv2.2 datasheet and our experiments on
+ * PPv2.1, RX queues have an allocation granularity of 4 (when
+ * more than a single one on PPv2.2).
+ * Round up to nearest multiple of 4.
+ */
+ nrxqs = (num_possible_cpus() + 3) & ~0x3;
+ if (nrxqs > MVPP2_PORT_MAX_RXQ)
+ nrxqs = MVPP2_PORT_MAX_RXQ;
+
+ return nrxqs;
+}
+
/* Get number of physical egress port */
static inline int mvpp2_egress_port(struct mvpp2_port *port)
{
@@ -323,8 +343,7 @@ static void mvpp2_frag_free(const struct mvpp2_bm_pool *pool, void *data)
/* Buffer Manager configuration routines */
/* Create pool */
-static int mvpp2_bm_pool_create(struct platform_device *pdev,
- struct mvpp2 *priv,
+static int mvpp2_bm_pool_create(struct device *dev, struct mvpp2 *priv,
struct mvpp2_bm_pool *bm_pool, int size)
{
u32 val;
@@ -343,7 +362,7 @@ static int mvpp2_bm_pool_create(struct platform_device *pdev,
else
bm_pool->size_bytes = 2 * sizeof(u64) * size;
- bm_pool->virt_addr = dma_alloc_coherent(&pdev->dev, bm_pool->size_bytes,
+ bm_pool->virt_addr = dma_alloc_coherent(dev, bm_pool->size_bytes,
&bm_pool->dma_addr,
GFP_KERNEL);
if (!bm_pool->virt_addr)
@@ -351,9 +370,9 @@ static int mvpp2_bm_pool_create(struct platform_device *pdev,
if (!IS_ALIGNED((unsigned long)bm_pool->virt_addr,
MVPP2_BM_POOL_PTR_ALIGN)) {
- dma_free_coherent(&pdev->dev, bm_pool->size_bytes,
+ dma_free_coherent(dev, bm_pool->size_bytes,
bm_pool->virt_addr, bm_pool->dma_addr);
- dev_err(&pdev->dev, "BM pool %d is not %d bytes aligned\n",
+ dev_err(dev, "BM pool %d is not %d bytes aligned\n",
bm_pool->id, MVPP2_BM_POOL_PTR_ALIGN);
return -ENOMEM;
}
@@ -468,15 +487,14 @@ static int mvpp2_check_hw_buf_num(struct mvpp2 *priv, struct mvpp2_bm_pool *bm_p
}
/* Cleanup pool */
-static int mvpp2_bm_pool_destroy(struct platform_device *pdev,
- struct mvpp2 *priv,
+static int mvpp2_bm_pool_destroy(struct device *dev, struct mvpp2 *priv,
struct mvpp2_bm_pool *bm_pool)
{
int buf_num;
u32 val;
buf_num = mvpp2_check_hw_buf_num(priv, bm_pool);
- mvpp2_bm_bufs_free(&pdev->dev, priv, bm_pool, buf_num);
+ mvpp2_bm_bufs_free(dev, priv, bm_pool, buf_num);
/* Check buffer counters after free */
buf_num = mvpp2_check_hw_buf_num(priv, bm_pool);
@@ -490,24 +508,26 @@ static int mvpp2_bm_pool_destroy(struct platform_device *pdev,
val |= MVPP2_BM_STOP_MASK;
mvpp2_write(priv, MVPP2_BM_POOL_CTRL_REG(bm_pool->id), val);
- dma_free_coherent(&pdev->dev, bm_pool->size_bytes,
+ dma_free_coherent(dev, bm_pool->size_bytes,
bm_pool->virt_addr,
bm_pool->dma_addr);
return 0;
}
-static int mvpp2_bm_pools_init(struct platform_device *pdev,
- struct mvpp2 *priv)
+static int mvpp2_bm_pools_init(struct device *dev, struct mvpp2 *priv)
{
- int i, err, size;
+ int i, err, size, poolnum = MVPP2_BM_POOLS_NUM;
struct mvpp2_bm_pool *bm_pool;
+ if (priv->percpu_pools)
+ poolnum = mvpp2_get_nrxqs(priv) * 2;
+
/* Create all pools with maximum size */
size = MVPP2_BM_POOL_SIZE_MAX;
- for (i = 0; i < MVPP2_BM_POOLS_NUM; i++) {
+ for (i = 0; i < poolnum; i++) {
bm_pool = &priv->bm_pools[i];
bm_pool->id = i;
- err = mvpp2_bm_pool_create(pdev, priv, bm_pool, size);
+ err = mvpp2_bm_pool_create(dev, priv, bm_pool, size);
if (err)
goto err_unroll_pools;
mvpp2_bm_pool_bufsize_set(priv, bm_pool, 0);
@@ -515,17 +535,23 @@ static int mvpp2_bm_pools_init(struct platform_device *pdev,
return 0;
err_unroll_pools:
- dev_err(&pdev->dev, "failed to create BM pool %d, size %d\n", i, size);
+ dev_err(dev, "failed to create BM pool %d, size %d\n", i, size);
for (i = i - 1; i >= 0; i--)
- mvpp2_bm_pool_destroy(pdev, priv, &priv->bm_pools[i]);
+ mvpp2_bm_pool_destroy(dev, priv, &priv->bm_pools[i]);
return err;
}
-static int mvpp2_bm_init(struct platform_device *pdev, struct mvpp2 *priv)
+static int mvpp2_bm_init(struct device *dev, struct mvpp2 *priv)
{
- int i, err;
+ int i, err, poolnum = MVPP2_BM_POOLS_NUM;
- for (i = 0; i < MVPP2_BM_POOLS_NUM; i++) {
+ if (priv->percpu_pools)
+ poolnum = mvpp2_get_nrxqs(priv) * 2;
+
+ dev_info(dev, "using %d %s buffers\n", poolnum,
+ priv->percpu_pools ? "per-cpu" : "shared");
+
+ for (i = 0; i < poolnum; i++) {
/* Mask BM all interrupts */
mvpp2_write(priv, MVPP2_BM_INTR_MASK_REG(i), 0);
/* Clear BM cause register */
@@ -533,12 +559,12 @@ static int mvpp2_bm_init(struct platform_device *pdev, struct mvpp2 *priv)
}
/* Allocate and initialize BM pools */
- priv->bm_pools = devm_kcalloc(&pdev->dev, MVPP2_BM_POOLS_NUM,
+ priv->bm_pools = devm_kcalloc(dev, poolnum,
sizeof(*priv->bm_pools), GFP_KERNEL);
if (!priv->bm_pools)
return -ENOMEM;
- err = mvpp2_bm_pools_init(pdev, priv);
+ err = mvpp2_bm_pools_init(dev, priv);
if (err < 0)
return err;
return 0;
@@ -679,6 +705,13 @@ static int mvpp2_bm_bufs_add(struct mvpp2_port *port,
phys_addr_t phys_addr;
void *buf;
+ if (port->priv->percpu_pools &&
+ bm_pool->pkt_size > MVPP2_BM_LONG_PKT_SIZE) {
+ netdev_err(port->dev,
+ "attempted to use jumbo frames with per-cpu pools");
+ return 0;
+ }
+
buf_size = MVPP2_RX_BUF_SIZE(bm_pool->pkt_size);
total_size = MVPP2_RX_TOTAL_SIZE(buf_size);
@@ -722,7 +755,64 @@ mvpp2_bm_pool_use(struct mvpp2_port *port, unsigned pool, int pkt_size)
struct mvpp2_bm_pool *new_pool = &port->priv->bm_pools[pool];
int num;
- if (pool >= MVPP2_BM_POOLS_NUM) {
+ if ((port->priv->percpu_pools && pool > mvpp2_get_nrxqs(port->priv) * 2) ||
+ (!port->priv->percpu_pools && pool >= MVPP2_BM_POOLS_NUM)) {
+ netdev_err(port->dev, "Invalid pool %d\n", pool);
+ return NULL;
+ }
+
+ /* Allocate buffers in case BM pool is used as long pool, but packet
+ * size doesn't match MTU or BM pool hasn't being used yet
+ */
+ if (new_pool->pkt_size == 0) {
+ int pkts_num;
+
+ /* Set default buffer number or free all the buffers in case
+ * the pool is not empty
+ */
+ pkts_num = new_pool->buf_num;
+ if (pkts_num == 0) {
+ if (port->priv->percpu_pools) {
+ if (pool < port->nrxqs)
+ pkts_num = mvpp2_pools[MVPP2_BM_SHORT].buf_num;
+ else
+ pkts_num = mvpp2_pools[MVPP2_BM_LONG].buf_num;
+ } else {
+ pkts_num = mvpp2_pools[pool].buf_num;
+ }
+ } else {
+ mvpp2_bm_bufs_free(port->dev->dev.parent,
+ port->priv, new_pool, pkts_num);
+ }
+
+ new_pool->pkt_size = pkt_size;
+ new_pool->frag_size =
+ SKB_DATA_ALIGN(MVPP2_RX_BUF_SIZE(pkt_size)) +
+ MVPP2_SKB_SHINFO_SIZE;
+
+ /* Allocate buffers for this pool */
+ num = mvpp2_bm_bufs_add(port, new_pool, pkts_num);
+ if (num != pkts_num) {
+ WARN(1, "pool %d: %d of %d allocated\n",
+ new_pool->id, num, pkts_num);
+ return NULL;
+ }
+ }
+
+ mvpp2_bm_pool_bufsize_set(port->priv, new_pool,
+ MVPP2_RX_BUF_SIZE(new_pool->pkt_size));
+
+ return new_pool;
+}
+
+static struct mvpp2_bm_pool *
+mvpp2_bm_pool_use_percpu(struct mvpp2_port *port, int type,
+ unsigned int pool, int pkt_size)
+{
+ struct mvpp2_bm_pool *new_pool = &port->priv->bm_pools[pool];
+ int num;
+
+ if (pool > port->nrxqs * 2) {
netdev_err(port->dev, "Invalid pool %d\n", pool);
return NULL;
}
@@ -738,7 +828,7 @@ mvpp2_bm_pool_use(struct mvpp2_port *port, unsigned pool, int pkt_size)
*/
pkts_num = new_pool->buf_num;
if (pkts_num == 0)
- pkts_num = mvpp2_pools[pool].buf_num;
+ pkts_num = mvpp2_pools[type].buf_num;
else
mvpp2_bm_bufs_free(port->dev->dev.parent,
port->priv, new_pool, pkts_num);
@@ -763,11 +853,11 @@ mvpp2_bm_pool_use(struct mvpp2_port *port, unsigned pool, int pkt_size)
return new_pool;
}
-/* Initialize pools for swf */
-static int mvpp2_swf_bm_pool_init(struct mvpp2_port *port)
+/* Initialize pools for swf, shared buffers variant */
+static int mvpp2_swf_bm_pool_init_shared(struct mvpp2_port *port)
{
- int rxq;
enum mvpp2_bm_pool_log_num long_log_pool, short_log_pool;
+ int rxq;
/* If port pkt_size is higher than 1518B:
* HW Long pool - SW Jumbo pool, HW Short pool - SW Long pool
@@ -811,6 +901,47 @@ static int mvpp2_swf_bm_pool_init(struct mvpp2_port *port)
return 0;
}
+/* Initialize pools for swf, percpu buffers variant */
+static int mvpp2_swf_bm_pool_init_percpu(struct mvpp2_port *port)
+{
+ struct mvpp2_bm_pool *p;
+ int i;
+
+ for (i = 0; i < port->nrxqs; i++) {
+ p = mvpp2_bm_pool_use_percpu(port, MVPP2_BM_SHORT, i,
+ mvpp2_pools[MVPP2_BM_SHORT].pkt_size);
+ if (!p)
+ return -ENOMEM;
+
+ port->priv->bm_pools[i].port_map |= BIT(port->id);
+ mvpp2_rxq_short_pool_set(port, i, port->priv->bm_pools[i].id);
+ }
+
+ for (i = 0; i < port->nrxqs; i++) {
+ p = mvpp2_bm_pool_use_percpu(port, MVPP2_BM_LONG, i + port->nrxqs,
+ mvpp2_pools[MVPP2_BM_LONG].pkt_size);
+ if (!p)
+ return -ENOMEM;
+
+ port->priv->bm_pools[i + port->nrxqs].port_map |= BIT(port->id);
+ mvpp2_rxq_long_pool_set(port, i,
+ port->priv->bm_pools[i + port->nrxqs].id);
+ }
+
+ port->pool_long = NULL;
+ port->pool_short = NULL;
+
+ return 0;
+}
+
+static int mvpp2_swf_bm_pool_init(struct mvpp2_port *port)
+{
+ if (port->priv->percpu_pools)
+ return mvpp2_swf_bm_pool_init_percpu(port);
+ else
+ return mvpp2_swf_bm_pool_init_shared(port);
+}
+
static void mvpp2_set_hw_csum(struct mvpp2_port *port,
enum mvpp2_bm_pool_log_num new_long_pool)
{
@@ -837,6 +968,9 @@ static int mvpp2_bm_update_mtu(struct net_device *dev, int mtu)
enum mvpp2_bm_pool_log_num new_long_pool;
int pkt_size = MVPP2_RX_PKT_SIZE(mtu);
+ if (port->priv->percpu_pools)
+ goto out_set;
+
/* If port MTU is higher than 1518B:
* HW Long pool - SW Jumbo pool, HW Short pool - SW Long pool
* else: HW Long pool - SW Long pool, HW Short pool - SW Short pool
@@ -866,6 +1000,7 @@ static int mvpp2_bm_update_mtu(struct net_device *dev, int mtu)
mvpp2_set_hw_csum(port, new_long_pool);
}
+out_set:
dev->mtu = mtu;
dev->wanted_features = dev->features;
@@ -3699,10 +3834,48 @@ static int mvpp2_set_mac_address(struct net_device *dev, void *p)
return err;
}
+/* Shut down all the ports, reconfigure the pools as percpu or shared,
+ * then bring up again all ports.
+ */
+static int mvpp2_bm_switch_buffers(struct mvpp2 *priv, bool percpu)
+{
+ int numbufs = MVPP2_BM_POOLS_NUM, i;
+ struct mvpp2_port *port = NULL;
+ bool status[MVPP2_MAX_PORTS];
+
+ for (i = 0; i < priv->port_count; i++) {
+ port = priv->port_list[i];
+ status[i] = netif_running(port->dev);
+ if (status[i])
+ mvpp2_stop(port->dev);
+ }
+
+ /* nrxqs is the same for all ports */
+ if (priv->percpu_pools)
+ numbufs = port->nrxqs * 2;
+
+ for (i = 0; i < numbufs; i++)
+ mvpp2_bm_pool_destroy(port->dev->dev.parent, priv, &priv->bm_pools[i]);
+
+ devm_kfree(port->dev->dev.parent, priv->bm_pools);
+ priv->percpu_pools = percpu;
+ mvpp2_bm_init(port->dev->dev.parent, priv);
+
+ for (i = 0; i < priv->port_count; i++) {
+ port = priv->port_list[i];
+ mvpp2_swf_bm_pool_init(port);
+ if (status[i])
+ mvpp2_open(port->dev);
+ }
+
+ return 0;
+}
+
static int mvpp2_change_mtu(struct net_device *dev, int mtu)
{
struct mvpp2_port *port = netdev_priv(dev);
bool running = netif_running(dev);
+ struct mvpp2 *priv = port->priv;
int err;
if (!IS_ALIGNED(MVPP2_RX_PKT_SIZE(mtu), 8)) {
@@ -3711,6 +3884,31 @@ static int mvpp2_change_mtu(struct net_device *dev, int mtu)
mtu = ALIGN(MVPP2_RX_PKT_SIZE(mtu), 8);
}
+ if (MVPP2_RX_PKT_SIZE(mtu) > MVPP2_BM_LONG_PKT_SIZE) {
+ if (priv->percpu_pools) {
+ netdev_warn(dev, "mtu %d too high, switching to shared buffers", mtu);
+ mvpp2_bm_switch_buffers(priv, false);
+ }
+ } else {
+ bool jumbo = false;
+ int i;
+
+ for (i = 0; i < priv->port_count; i++)
+ if (priv->port_list[i] != port &&
+ MVPP2_RX_PKT_SIZE(priv->port_list[i]->dev->mtu) >
+ MVPP2_BM_LONG_PKT_SIZE) {
+ jumbo = true;
+ break;
+ }
+
+ /* No port is using jumbo frames */
+ if (!jumbo) {
+ dev_info(port->dev->dev.parent,
+ "all ports have a low MTU, switching to per-cpu buffers");
+ mvpp2_bm_switch_buffers(priv, true);
+ }
+ }
+
if (running)
mvpp2_stop_dev(port);
@@ -5017,18 +5215,7 @@ static int mvpp2_port_probe(struct platform_device *pdev,
}
ntxqs = MVPP2_MAX_TXQ;
- if (priv->hw_version == MVPP22 && queue_mode == MVPP2_QDIST_SINGLE_MODE) {
- nrxqs = 1;
- } else {
- /* According to the PPv2.2 datasheet and our experiments on
- * PPv2.1, RX queues have an allocation granularity of 4 (when
- * more than a single one on PPv2.2).
- * Round up to nearest multiple of 4.
- */
- nrxqs = (num_possible_cpus() + 3) & ~0x3;
- if (nrxqs > MVPP2_PORT_MAX_RXQ)
- nrxqs = MVPP2_PORT_MAX_RXQ;
- }
+ nrxqs = mvpp2_get_nrxqs(priv);
dev = alloc_etherdev_mqs(sizeof(*port), ntxqs, nrxqs);
if (!dev)
@@ -5190,7 +5377,8 @@ static int mvpp2_port_probe(struct platform_device *pdev,
dev->features |= NETIF_F_NTUPLE;
}
- mvpp2_set_hw_csum(port, port->pool_long->id);
+ if (!port->priv->percpu_pools)
+ mvpp2_set_hw_csum(port, port->pool_long->id);
dev->vlan_features |= features;
dev->gso_max_segs = MVPP2_MAX_TSO_SEGS;
@@ -5482,7 +5670,7 @@ static int mvpp2_init(struct platform_device *pdev, struct mvpp2 *priv)
mvpp2_write(priv, MVPP2_TX_SNOOP_REG, 0x1);
/* Buffer Manager initialization */
- err = mvpp2_bm_init(pdev, priv);
+ err = mvpp2_bm_init(&pdev->dev, priv);
if (err < 0)
return err;
@@ -5568,6 +5756,10 @@ static int mvpp2_probe(struct platform_device *pdev)
priv->sysctrl_base = NULL;
}
+ if (priv->hw_version == MVPP22 &&
+ mvpp2_get_nrxqs(priv) * 2 <= MVPP2_BM_MAX_POOLS)
+ priv->percpu_pools = 1;
+
mvpp2_setup_bm_pool();
@@ -5749,7 +5941,7 @@ static int mvpp2_remove(struct platform_device *pdev)
for (i = 0; i < MVPP2_BM_POOLS_NUM; i++) {
struct mvpp2_bm_pool *bm_pool = &priv->bm_pools[i];
- mvpp2_bm_pool_destroy(pdev, priv, bm_pool);
+ mvpp2_bm_pool_destroy(&pdev->dev, priv, bm_pool);
}
for (i = 0; i < MVPP2_MAX_THREADS; i++) {
diff --git a/drivers/net/ethernet/marvell/sky2.c b/drivers/net/ethernet/marvell/sky2.c
index c2e00bb5..5f56ee8 100644
--- a/drivers/net/ethernet/marvell/sky2.c
+++ b/drivers/net/ethernet/marvell/sky2.c
@@ -4931,6 +4931,13 @@ static const struct dmi_system_id msi_blacklist[] = {
DMI_MATCH(DMI_BOARD_NAME, "P6T"),
},
},
+ {
+ .ident = "ASUS P6X",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK Computer INC."),
+ DMI_MATCH(DMI_BOARD_NAME, "P6X"),
+ },
+ },
{}
};
diff --git a/drivers/net/ethernet/mediatek/Kconfig b/drivers/net/ethernet/mediatek/Kconfig
index b76cf2e..4968352 100644
--- a/drivers/net/ethernet/mediatek/Kconfig
+++ b/drivers/net/ethernet/mediatek/Kconfig
@@ -9,7 +9,7 @@
config NET_MEDIATEK_SOC
tristate "MediaTek SoC Gigabit Ethernet support"
- select PHYLIB
+ select PHYLINK
---help---
This driver supports the gigabit ethernet MACs in the
MediaTek SoC family.
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_path.c b/drivers/net/ethernet/mediatek/mtk_eth_path.c
index 28960e4..ef11cf3 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_path.c
+++ b/drivers/net/ethernet/mediatek/mtk_eth_path.c
@@ -239,10 +239,9 @@ static int mtk_eth_mux_setup(struct mtk_eth *eth, int path)
return err;
}
-static int mtk_gmac_sgmii_path_setup(struct mtk_eth *eth, int mac_id)
+int mtk_gmac_sgmii_path_setup(struct mtk_eth *eth, int mac_id)
{
- unsigned int val = 0;
- int sid, err, path;
+ int err, path;
path = (mac_id == 0) ? MTK_ETH_PATH_GMAC1_SGMII :
MTK_ETH_PATH_GMAC2_SGMII;
@@ -252,33 +251,10 @@ static int mtk_gmac_sgmii_path_setup(struct mtk_eth *eth, int mac_id)
if (err)
return err;
- /* The path GMAC to SGMII will be enabled once the SGMIISYS is being
- * setup done.
- */
- regmap_read(eth->ethsys, ETHSYS_SYSCFG0, &val);
-
- regmap_update_bits(eth->ethsys, ETHSYS_SYSCFG0,
- SYSCFG0_SGMII_MASK, ~(u32)SYSCFG0_SGMII_MASK);
-
- /* Decide how GMAC and SGMIISYS be mapped */
- sid = (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_SGMII)) ? 0 : mac_id;
-
- /* Setup SGMIISYS with the determined property */
- if (MTK_HAS_FLAGS(eth->sgmii->flags[sid], MTK_SGMII_PHYSPEED_AN))
- err = mtk_sgmii_setup_mode_an(eth->sgmii, sid);
- else
- err = mtk_sgmii_setup_mode_force(eth->sgmii, sid);
-
- if (err)
- return err;
-
- regmap_update_bits(eth->ethsys, ETHSYS_SYSCFG0,
- SYSCFG0_SGMII_MASK, val);
-
return 0;
}
-static int mtk_gmac_gephy_path_setup(struct mtk_eth *eth, int mac_id)
+int mtk_gmac_gephy_path_setup(struct mtk_eth *eth, int mac_id)
{
int err, path = 0;
@@ -296,7 +272,7 @@ static int mtk_gmac_gephy_path_setup(struct mtk_eth *eth, int mac_id)
return 0;
}
-static int mtk_gmac_rgmii_path_setup(struct mtk_eth *eth, int mac_id)
+int mtk_gmac_rgmii_path_setup(struct mtk_eth *eth, int mac_id)
{
int err, path;
@@ -311,46 +287,3 @@ static int mtk_gmac_rgmii_path_setup(struct mtk_eth *eth, int mac_id)
return 0;
}
-int mtk_setup_hw_path(struct mtk_eth *eth, int mac_id, int phymode)
-{
- int err;
-
- /* No mux'ing for MT7628/88 */
- if (MTK_HAS_CAPS(eth->soc->caps, MTK_SOC_MT7628))
- return 0;
-
- switch (phymode) {
- case PHY_INTERFACE_MODE_TRGMII:
- case PHY_INTERFACE_MODE_RGMII_TXID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_MII:
- case PHY_INTERFACE_MODE_REVMII:
- case PHY_INTERFACE_MODE_RMII:
- if (MTK_HAS_CAPS(eth->soc->caps, MTK_RGMII)) {
- err = mtk_gmac_rgmii_path_setup(eth, mac_id);
- if (err)
- return err;
- }
- break;
- case PHY_INTERFACE_MODE_SGMII:
- if (MTK_HAS_CAPS(eth->soc->caps, MTK_SGMII)) {
- err = mtk_gmac_sgmii_path_setup(eth, mac_id);
- if (err)
- return err;
- }
- break;
- case PHY_INTERFACE_MODE_GMII:
- if (MTK_HAS_CAPS(eth->soc->caps, MTK_GEPHY)) {
- err = mtk_gmac_gephy_path_setup(eth, mac_id);
- if (err)
- return err;
- }
- break;
- default:
- break;
- }
-
- return 0;
-}
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
index 8ddbb8d..c610693 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
@@ -18,6 +18,7 @@
#include <linux/tcp.h>
#include <linux/interrupt.h>
#include <linux/pinctrl/devinfo.h>
+#include <linux/phylink.h>
#include "mtk_eth_soc.h"
@@ -186,168 +187,339 @@ static void mtk_gmac0_rgmii_adjust(struct mtk_eth *eth, int speed)
mtk_w32(eth, val, TRGMII_TCK_CTRL);
}
-static void mtk_phy_link_adjust(struct net_device *dev)
+static void mtk_mac_config(struct phylink_config *config, unsigned int mode,
+ const struct phylink_link_state *state)
{
- struct mtk_mac *mac = netdev_priv(dev);
- u16 lcl_adv = 0, rmt_adv = 0;
- u8 flowctrl;
- u32 mcr = MAC_MCR_MAX_RX_1536 | MAC_MCR_IPG_CFG |
- MAC_MCR_FORCE_MODE | MAC_MCR_TX_EN |
- MAC_MCR_RX_EN | MAC_MCR_BACKOFF_EN |
- MAC_MCR_BACKPR_EN;
+ struct mtk_mac *mac = container_of(config, struct mtk_mac,
+ phylink_config);
+ struct mtk_eth *eth = mac->hw;
+ u32 mcr_cur, mcr_new, sid;
+ int val, ge_mode, err;
- if (unlikely(test_bit(MTK_RESETTING, &mac->hw->state)))
- return;
-
- switch (dev->phydev->speed) {
- case SPEED_1000:
- mcr |= MAC_MCR_SPEED_1000;
- break;
- case SPEED_100:
- mcr |= MAC_MCR_SPEED_100;
- break;
- }
-
- if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_GMAC1_TRGMII) && !mac->id) {
- if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_TRGMII_MT7621_CLK)) {
- if (mt7621_gmac0_rgmii_adjust(mac->hw,
- dev->phydev->interface))
- return;
- } else {
- if (!mac->trgmii)
- mtk_gmac0_rgmii_adjust(mac->hw,
- dev->phydev->speed);
- }
- }
-
- if (dev->phydev->link)
- mcr |= MAC_MCR_FORCE_LINK;
-
- if (dev->phydev->duplex) {
- mcr |= MAC_MCR_FORCE_DPX;
-
- if (dev->phydev->pause)
- rmt_adv = LPA_PAUSE_CAP;
- if (dev->phydev->asym_pause)
- rmt_adv |= LPA_PAUSE_ASYM;
-
- lcl_adv = linkmode_adv_to_lcl_adv_t(dev->phydev->advertising);
- flowctrl = mii_resolve_flowctrl_fdx(lcl_adv, rmt_adv);
-
- if (flowctrl & FLOW_CTRL_TX)
- mcr |= MAC_MCR_FORCE_TX_FC;
- if (flowctrl & FLOW_CTRL_RX)
- mcr |= MAC_MCR_FORCE_RX_FC;
-
- netif_dbg(mac->hw, link, dev, "rx pause %s, tx pause %s\n",
- flowctrl & FLOW_CTRL_RX ? "enabled" : "disabled",
- flowctrl & FLOW_CTRL_TX ? "enabled" : "disabled");
- }
-
- mtk_w32(mac->hw, mcr, MTK_MAC_MCR(mac->id));
-
- if (!of_phy_is_fixed_link(mac->of_node))
- phy_print_status(dev->phydev);
-}
-
-static int mtk_phy_connect_node(struct mtk_eth *eth, struct mtk_mac *mac,
- struct device_node *phy_node)
-{
- struct phy_device *phydev;
- int phy_mode;
-
- phy_mode = of_get_phy_mode(phy_node);
- if (phy_mode < 0) {
- dev_err(eth->dev, "incorrect phy-mode %d\n", phy_mode);
- return -EINVAL;
- }
-
- phydev = of_phy_connect(eth->netdev[mac->id], phy_node,
- mtk_phy_link_adjust, 0, phy_mode);
- if (!phydev) {
- dev_err(eth->dev, "could not connect to PHY\n");
- return -ENODEV;
- }
-
- dev_info(eth->dev,
- "connected mac %d to PHY at %s [uid=%08x, driver=%s]\n",
- mac->id, phydev_name(phydev), phydev->phy_id,
- phydev->drv->name);
-
- return 0;
-}
-
-static int mtk_phy_connect(struct net_device *dev)
-{
- struct mtk_mac *mac = netdev_priv(dev);
- struct mtk_eth *eth;
- struct device_node *np;
- u32 val;
- int err;
-
- eth = mac->hw;
- np = of_parse_phandle(mac->of_node, "phy-handle", 0);
- if (!np && of_phy_is_fixed_link(mac->of_node))
- if (!of_phy_register_fixed_link(mac->of_node))
- np = of_node_get(mac->of_node);
- if (!np)
- return -ENODEV;
-
- err = mtk_setup_hw_path(eth, mac->id, of_get_phy_mode(np));
- if (err)
- goto err_phy;
-
- mac->ge_mode = 0;
- switch (of_get_phy_mode(np)) {
- case PHY_INTERFACE_MODE_TRGMII:
- mac->trgmii = true;
- case PHY_INTERFACE_MODE_RGMII_TXID:
- case PHY_INTERFACE_MODE_RGMII_RXID:
- case PHY_INTERFACE_MODE_RGMII_ID:
- case PHY_INTERFACE_MODE_RGMII:
- case PHY_INTERFACE_MODE_SGMII:
- break;
- case PHY_INTERFACE_MODE_MII:
- case PHY_INTERFACE_MODE_GMII:
- mac->ge_mode = 1;
- break;
- case PHY_INTERFACE_MODE_REVMII:
- mac->ge_mode = 2;
- break;
- case PHY_INTERFACE_MODE_RMII:
- if (!mac->id)
+ /* MT76x8 has no hardware settings between for the MAC */
+ if (!MTK_HAS_CAPS(eth->soc->caps, MTK_SOC_MT7628) &&
+ mac->interface != state->interface) {
+ /* Setup soc pin functions */
+ switch (state->interface) {
+ case PHY_INTERFACE_MODE_TRGMII:
+ if (mac->id)
+ goto err_phy;
+ if (!MTK_HAS_CAPS(mac->hw->soc->caps,
+ MTK_GMAC1_TRGMII))
+ goto err_phy;
+ /* fall through */
+ case PHY_INTERFACE_MODE_RGMII_TXID:
+ case PHY_INTERFACE_MODE_RGMII_RXID:
+ case PHY_INTERFACE_MODE_RGMII_ID:
+ case PHY_INTERFACE_MODE_RGMII:
+ case PHY_INTERFACE_MODE_MII:
+ case PHY_INTERFACE_MODE_REVMII:
+ case PHY_INTERFACE_MODE_RMII:
+ if (MTK_HAS_CAPS(eth->soc->caps, MTK_RGMII)) {
+ err = mtk_gmac_rgmii_path_setup(eth, mac->id);
+ if (err)
+ goto init_err;
+ }
+ break;
+ case PHY_INTERFACE_MODE_1000BASEX:
+ case PHY_INTERFACE_MODE_2500BASEX:
+ case PHY_INTERFACE_MODE_SGMII:
+ if (MTK_HAS_CAPS(eth->soc->caps, MTK_SGMII)) {
+ err = mtk_gmac_sgmii_path_setup(eth, mac->id);
+ if (err)
+ goto init_err;
+ }
+ break;
+ case PHY_INTERFACE_MODE_GMII:
+ if (MTK_HAS_CAPS(eth->soc->caps, MTK_GEPHY)) {
+ err = mtk_gmac_gephy_path_setup(eth, mac->id);
+ if (err)
+ goto init_err;
+ }
+ break;
+ default:
goto err_phy;
- mac->ge_mode = 3;
- break;
- default:
- goto err_phy;
- }
+ }
- /* No MT7628/88 support for now */
- if (!MTK_HAS_CAPS(eth->soc->caps, MTK_SOC_MT7628)) {
+ /* Setup clock for 1st gmac */
+ if (!mac->id && state->interface != PHY_INTERFACE_MODE_SGMII &&
+ !phy_interface_mode_is_8023z(state->interface) &&
+ MTK_HAS_CAPS(mac->hw->soc->caps, MTK_GMAC1_TRGMII)) {
+ if (MTK_HAS_CAPS(mac->hw->soc->caps,
+ MTK_TRGMII_MT7621_CLK)) {
+ if (mt7621_gmac0_rgmii_adjust(mac->hw,
+ state->interface))
+ goto err_phy;
+ } else {
+ if (state->interface !=
+ PHY_INTERFACE_MODE_TRGMII)
+ mtk_gmac0_rgmii_adjust(mac->hw,
+ state->speed);
+ }
+ }
+
+ ge_mode = 0;
+ switch (state->interface) {
+ case PHY_INTERFACE_MODE_MII:
+ ge_mode = 1;
+ break;
+ case PHY_INTERFACE_MODE_REVMII:
+ ge_mode = 2;
+ break;
+ case PHY_INTERFACE_MODE_RMII:
+ if (mac->id)
+ goto err_phy;
+ ge_mode = 3;
+ break;
+ default:
+ break;
+ }
+
/* put the gmac into the right mode */
regmap_read(eth->ethsys, ETHSYS_SYSCFG0, &val);
val &= ~SYSCFG0_GE_MODE(SYSCFG0_GE_MASK, mac->id);
- val |= SYSCFG0_GE_MODE(mac->ge_mode, mac->id);
+ val |= SYSCFG0_GE_MODE(ge_mode, mac->id);
regmap_write(eth->ethsys, ETHSYS_SYSCFG0, val);
+
+ mac->interface = state->interface;
}
- /* couple phydev to net_device */
- if (mtk_phy_connect_node(eth, mac, np))
- goto err_phy;
+ /* SGMII */
+ if (state->interface == PHY_INTERFACE_MODE_SGMII ||
+ phy_interface_mode_is_8023z(state->interface)) {
+ /* The path GMAC to SGMII will be enabled once the SGMIISYS is
+ * being setup done.
+ */
+ regmap_read(eth->ethsys, ETHSYS_SYSCFG0, &val);
- of_node_put(np);
+ regmap_update_bits(eth->ethsys, ETHSYS_SYSCFG0,
+ SYSCFG0_SGMII_MASK,
+ ~(u32)SYSCFG0_SGMII_MASK);
- return 0;
+ /* Decide how GMAC and SGMIISYS be mapped */
+ sid = (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_SGMII)) ?
+ 0 : mac->id;
+
+ /* Setup SGMIISYS with the determined property */
+ if (state->interface != PHY_INTERFACE_MODE_SGMII)
+ err = mtk_sgmii_setup_mode_force(eth->sgmii, sid,
+ state);
+ else if (phylink_autoneg_inband(mode))
+ err = mtk_sgmii_setup_mode_an(eth->sgmii, sid);
+
+ if (err)
+ goto init_err;
+
+ regmap_update_bits(eth->ethsys, ETHSYS_SYSCFG0,
+ SYSCFG0_SGMII_MASK, val);
+ } else if (phylink_autoneg_inband(mode)) {
+ dev_err(eth->dev,
+ "In-band mode not supported in non SGMII mode!\n");
+ return;
+ }
+
+ /* Setup gmac */
+ mcr_cur = mtk_r32(mac->hw, MTK_MAC_MCR(mac->id));
+ mcr_new = mcr_cur;
+ mcr_new &= ~(MAC_MCR_SPEED_100 | MAC_MCR_SPEED_1000 |
+ MAC_MCR_FORCE_DPX | MAC_MCR_FORCE_TX_FC |
+ MAC_MCR_FORCE_RX_FC);
+ mcr_new |= MAC_MCR_MAX_RX_1536 | MAC_MCR_IPG_CFG | MAC_MCR_FORCE_MODE |
+ MAC_MCR_BACKOFF_EN | MAC_MCR_BACKPR_EN | MAC_MCR_FORCE_LINK;
+
+ switch (state->speed) {
+ case SPEED_2500:
+ case SPEED_1000:
+ mcr_new |= MAC_MCR_SPEED_1000;
+ break;
+ case SPEED_100:
+ mcr_new |= MAC_MCR_SPEED_100;
+ break;
+ }
+ if (state->duplex == DUPLEX_FULL) {
+ mcr_new |= MAC_MCR_FORCE_DPX;
+ if (state->pause & MLO_PAUSE_TX)
+ mcr_new |= MAC_MCR_FORCE_TX_FC;
+ if (state->pause & MLO_PAUSE_RX)
+ mcr_new |= MAC_MCR_FORCE_RX_FC;
+ }
+
+ /* Only update control register when needed! */
+ if (mcr_new != mcr_cur)
+ mtk_w32(mac->hw, mcr_new, MTK_MAC_MCR(mac->id));
+
+ return;
err_phy:
- if (of_phy_is_fixed_link(mac->of_node))
- of_phy_deregister_fixed_link(mac->of_node);
- of_node_put(np);
- dev_err(eth->dev, "%s: invalid phy\n", __func__);
- return -EINVAL;
+ dev_err(eth->dev, "%s: GMAC%d mode %s not supported!\n", __func__,
+ mac->id, phy_modes(state->interface));
+ return;
+
+init_err:
+ dev_err(eth->dev, "%s: GMAC%d mode %s err: %d!\n", __func__,
+ mac->id, phy_modes(state->interface), err);
}
+static int mtk_mac_link_state(struct phylink_config *config,
+ struct phylink_link_state *state)
+{
+ struct mtk_mac *mac = container_of(config, struct mtk_mac,
+ phylink_config);
+ u32 pmsr = mtk_r32(mac->hw, MTK_MAC_MSR(mac->id));
+
+ state->link = (pmsr & MAC_MSR_LINK);
+ state->duplex = (pmsr & MAC_MSR_DPX) >> 1;
+
+ switch (pmsr & (MAC_MSR_SPEED_1000 | MAC_MSR_SPEED_100)) {
+ case 0:
+ state->speed = SPEED_10;
+ break;
+ case MAC_MSR_SPEED_100:
+ state->speed = SPEED_100;
+ break;
+ case MAC_MSR_SPEED_1000:
+ state->speed = SPEED_1000;
+ break;
+ default:
+ state->speed = SPEED_UNKNOWN;
+ break;
+ }
+
+ state->pause &= (MLO_PAUSE_RX | MLO_PAUSE_TX);
+ if (pmsr & MAC_MSR_RX_FC)
+ state->pause |= MLO_PAUSE_RX;
+ if (pmsr & MAC_MSR_TX_FC)
+ state->pause |= MLO_PAUSE_TX;
+
+ return 1;
+}
+
+static void mtk_mac_an_restart(struct phylink_config *config)
+{
+ struct mtk_mac *mac = container_of(config, struct mtk_mac,
+ phylink_config);
+
+ mtk_sgmii_restart_an(mac->hw, mac->id);
+}
+
+static void mtk_mac_link_down(struct phylink_config *config, unsigned int mode,
+ phy_interface_t interface)
+{
+ struct mtk_mac *mac = container_of(config, struct mtk_mac,
+ phylink_config);
+ u32 mcr = mtk_r32(mac->hw, MTK_MAC_MCR(mac->id));
+
+ mcr &= ~(MAC_MCR_TX_EN | MAC_MCR_RX_EN);
+ mtk_w32(mac->hw, mcr, MTK_MAC_MCR(mac->id));
+}
+
+static void mtk_mac_link_up(struct phylink_config *config, unsigned int mode,
+ phy_interface_t interface,
+ struct phy_device *phy)
+{
+ struct mtk_mac *mac = container_of(config, struct mtk_mac,
+ phylink_config);
+ u32 mcr = mtk_r32(mac->hw, MTK_MAC_MCR(mac->id));
+
+ mcr |= MAC_MCR_TX_EN | MAC_MCR_RX_EN;
+ mtk_w32(mac->hw, mcr, MTK_MAC_MCR(mac->id));
+}
+
+static void mtk_validate(struct phylink_config *config,
+ unsigned long *supported,
+ struct phylink_link_state *state)
+{
+ struct mtk_mac *mac = container_of(config, struct mtk_mac,
+ phylink_config);
+ __ETHTOOL_DECLARE_LINK_MODE_MASK(mask) = { 0, };
+
+ if (state->interface != PHY_INTERFACE_MODE_NA &&
+ state->interface != PHY_INTERFACE_MODE_MII &&
+ state->interface != PHY_INTERFACE_MODE_GMII &&
+ !(MTK_HAS_CAPS(mac->hw->soc->caps, MTK_RGMII) &&
+ phy_interface_mode_is_rgmii(state->interface)) &&
+ !(MTK_HAS_CAPS(mac->hw->soc->caps, MTK_TRGMII) &&
+ !mac->id && state->interface == PHY_INTERFACE_MODE_TRGMII) &&
+ !(MTK_HAS_CAPS(mac->hw->soc->caps, MTK_SGMII) &&
+ (state->interface == PHY_INTERFACE_MODE_SGMII ||
+ phy_interface_mode_is_8023z(state->interface)))) {
+ linkmode_zero(supported);
+ return;
+ }
+
+ phylink_set_port_modes(mask);
+ phylink_set(mask, Autoneg);
+
+ switch (state->interface) {
+ case PHY_INTERFACE_MODE_TRGMII:
+ phylink_set(mask, 1000baseT_Full);
+ break;
+ case PHY_INTERFACE_MODE_1000BASEX:
+ case PHY_INTERFACE_MODE_2500BASEX:
+ phylink_set(mask, 1000baseX_Full);
+ phylink_set(mask, 2500baseX_Full);
+ break;
+ case PHY_INTERFACE_MODE_GMII:
+ case PHY_INTERFACE_MODE_RGMII:
+ case PHY_INTERFACE_MODE_RGMII_ID:
+ case PHY_INTERFACE_MODE_RGMII_RXID:
+ case PHY_INTERFACE_MODE_RGMII_TXID:
+ phylink_set(mask, 1000baseT_Half);
+ /* fall through */
+ case PHY_INTERFACE_MODE_SGMII:
+ phylink_set(mask, 1000baseT_Full);
+ phylink_set(mask, 1000baseX_Full);
+ /* fall through */
+ case PHY_INTERFACE_MODE_MII:
+ case PHY_INTERFACE_MODE_RMII:
+ case PHY_INTERFACE_MODE_REVMII:
+ case PHY_INTERFACE_MODE_NA:
+ default:
+ phylink_set(mask, 10baseT_Half);
+ phylink_set(mask, 10baseT_Full);
+ phylink_set(mask, 100baseT_Half);
+ phylink_set(mask, 100baseT_Full);
+ break;
+ }
+
+ if (state->interface == PHY_INTERFACE_MODE_NA) {
+ if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_SGMII)) {
+ phylink_set(mask, 1000baseT_Full);
+ phylink_set(mask, 1000baseX_Full);
+ phylink_set(mask, 2500baseX_Full);
+ }
+ if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_RGMII)) {
+ phylink_set(mask, 1000baseT_Full);
+ phylink_set(mask, 1000baseT_Half);
+ phylink_set(mask, 1000baseX_Full);
+ }
+ if (MTK_HAS_CAPS(mac->hw->soc->caps, MTK_GEPHY)) {
+ phylink_set(mask, 1000baseT_Full);
+ phylink_set(mask, 1000baseT_Half);
+ }
+ }
+
+ phylink_set(mask, Pause);
+ phylink_set(mask, Asym_Pause);
+
+ linkmode_and(supported, supported, mask);
+ linkmode_and(state->advertising, state->advertising, mask);
+
+ /* We can only operate at 2500BaseX or 1000BaseX. If requested
+ * to advertise both, only report advertising at 2500BaseX.
+ */
+ phylink_helper_basex_speed(state);
+}
+
+static const struct phylink_mac_ops mtk_phylink_ops = {
+ .validate = mtk_validate,
+ .mac_link_state = mtk_mac_link_state,
+ .mac_an_restart = mtk_mac_an_restart,
+ .mac_config = mtk_mac_config,
+ .mac_link_down = mtk_mac_link_down,
+ .mac_link_up = mtk_mac_link_up,
+};
+
static int mtk_mdio_init(struct mtk_eth *eth)
{
struct device_node *mii_np;
@@ -1948,9 +2120,7 @@ static irqreturn_t mtk_handle_irq_tx(int irq, void *_eth)
static irqreturn_t mtk_handle_irq(int irq, void *_eth)
{
struct mtk_eth *eth = _eth;
- u32 status;
- status = mtk_r32(eth, MTK_PDMA_INT_STATUS);
if (mtk_r32(eth, MTK_PDMA_INT_MASK) & MTK_RX_DONE_INT) {
if (mtk_r32(eth, MTK_PDMA_INT_STATUS) & MTK_RX_DONE_INT)
mtk_handle_irq_rx(irq, _eth);
@@ -2013,6 +2183,14 @@ static int mtk_open(struct net_device *dev)
{
struct mtk_mac *mac = netdev_priv(dev);
struct mtk_eth *eth = mac->hw;
+ int err;
+
+ err = phylink_of_phy_connect(mac->phylink, mac->of_node, 0);
+ if (err) {
+ netdev_err(dev, "%s: could not attach PHY: %d\n", __func__,
+ err);
+ return err;
+ }
/* we run 2 netdevs on the same dma ring so we only bring it up once */
if (!refcount_read(ð->dma_refcnt)) {
@@ -2030,7 +2208,7 @@ static int mtk_open(struct net_device *dev)
else
refcount_inc(ð->dma_refcnt);
- phy_start(dev->phydev);
+ phylink_start(mac->phylink);
netif_start_queue(dev);
return 0;
}
@@ -2063,8 +2241,11 @@ static int mtk_stop(struct net_device *dev)
struct mtk_mac *mac = netdev_priv(dev);
struct mtk_eth *eth = mac->hw;
+ phylink_stop(mac->phylink);
+
netif_tx_disable(dev);
- phy_stop(dev->phydev);
+
+ phylink_disconnect_phy(mac->phylink);
/* only shutdown DMA if this is the last user */
if (!refcount_dec_and_test(ð->dma_refcnt))
@@ -2159,15 +2340,6 @@ static int mtk_hw_init(struct mtk_eth *eth)
ethsys_reset(eth, RSTCTRL_FE);
ethsys_reset(eth, RSTCTRL_PPE);
- regmap_read(eth->ethsys, ETHSYS_SYSCFG0, &val);
- for (i = 0; i < MTK_MAC_COUNT; i++) {
- if (!eth->mac[i])
- continue;
- val &= ~SYSCFG0_GE_MODE(SYSCFG0_GE_MASK, eth->mac[i]->id);
- val |= SYSCFG0_GE_MODE(eth->mac[i]->ge_mode, eth->mac[i]->id);
- }
- regmap_write(eth->ethsys, ETHSYS_SYSCFG0, val);
-
if (eth->pctl) {
/* Set GE2 driving and slew rate */
regmap_write(eth->pctl, GPIO_DRV_SEL10, 0xa00);
@@ -2180,11 +2352,11 @@ static int mtk_hw_init(struct mtk_eth *eth)
}
/* Set linkdown as the default for each GMAC. Its own MCR would be set
- * up with the more appropriate value when mtk_phy_link_adjust call is
- * being invoked.
+ * up with the more appropriate value when mtk_mac_config call is being
+ * invoked.
*/
for (i = 0; i < MTK_MAC_COUNT; i++)
- mtk_w32(eth, 0, MTK_MAC_MCR(i));
+ mtk_w32(eth, MAC_MCR_FORCE_LINK_DOWN, MTK_MAC_MCR(i));
/* Indicates CDM to parse the MTK special tag from CPU
* which also is working out for untag packets.
@@ -2212,7 +2384,7 @@ static int mtk_hw_init(struct mtk_eth *eth)
mtk_w32(eth, MTK_RX_DONE_INT, MTK_QDMA_INT_GRP2);
mtk_w32(eth, 0x21021000, MTK_FE_INT_GRP);
- for (i = 0; i < 2; i++) {
+ for (i = 0; i < MTK_MAC_COUNT; i++) {
u32 val = mtk_r32(eth, MTK_GDMA_FWD_CFG(i));
/* setup the forward port to send frame to PDMA */
@@ -2264,7 +2436,7 @@ static int __init mtk_init(struct net_device *dev)
dev->dev_addr);
}
- return mtk_phy_connect(dev);
+ return 0;
}
static void mtk_uninit(struct net_device *dev)
@@ -2272,20 +2444,20 @@ static void mtk_uninit(struct net_device *dev)
struct mtk_mac *mac = netdev_priv(dev);
struct mtk_eth *eth = mac->hw;
- phy_disconnect(dev->phydev);
- if (of_phy_is_fixed_link(mac->of_node))
- of_phy_deregister_fixed_link(mac->of_node);
+ phylink_disconnect_phy(mac->phylink);
mtk_tx_irq_disable(eth, ~0);
mtk_rx_irq_disable(eth, ~0);
}
static int mtk_do_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
{
+ struct mtk_mac *mac = netdev_priv(dev);
+
switch (cmd) {
case SIOCGMIIPHY:
case SIOCGMIIREG:
case SIOCSMIIREG:
- return phy_mii_ioctl(dev->phydev, ifr, cmd);
+ return phylink_mii_ioctl(mac->phylink, ifr, cmd);
default:
break;
}
@@ -2326,16 +2498,6 @@ static void mtk_pending_work(struct work_struct *work)
eth->dev->pins->default_state);
mtk_hw_init(eth);
- for (i = 0; i < MTK_MAC_COUNT; i++) {
- if (!eth->mac[i] ||
- of_phy_is_fixed_link(eth->mac[i]->of_node))
- continue;
- err = phy_init_hw(eth->netdev[i]->phydev);
- if (err)
- dev_err(eth->dev, "%s: PHY init failed.\n",
- eth->netdev[i]->name);
- }
-
/* restart DMA and enable IRQs */
for (i = 0; i < MTK_MAC_COUNT; i++) {
if (!test_bit(i, &restart))
@@ -2398,9 +2560,7 @@ static int mtk_get_link_ksettings(struct net_device *ndev,
if (unlikely(test_bit(MTK_RESETTING, &mac->hw->state)))
return -EBUSY;
- phy_ethtool_ksettings_get(ndev->phydev, cmd);
-
- return 0;
+ return phylink_ethtool_ksettings_get(mac->phylink, cmd);
}
static int mtk_set_link_ksettings(struct net_device *ndev,
@@ -2411,7 +2571,7 @@ static int mtk_set_link_ksettings(struct net_device *ndev,
if (unlikely(test_bit(MTK_RESETTING, &mac->hw->state)))
return -EBUSY;
- return phy_ethtool_ksettings_set(ndev->phydev, cmd);
+ return phylink_ethtool_ksettings_set(mac->phylink, cmd);
}
static void mtk_get_drvinfo(struct net_device *dev,
@@ -2445,22 +2605,10 @@ static int mtk_nway_reset(struct net_device *dev)
if (unlikely(test_bit(MTK_RESETTING, &mac->hw->state)))
return -EBUSY;
- return genphy_restart_aneg(dev->phydev);
-}
+ if (!mac->phylink)
+ return -ENOTSUPP;
-static u32 mtk_get_link(struct net_device *dev)
-{
- struct mtk_mac *mac = netdev_priv(dev);
- int err;
-
- if (unlikely(test_bit(MTK_RESETTING, &mac->hw->state)))
- return -EBUSY;
-
- err = genphy_update_link(dev->phydev);
- if (err)
- return ethtool_op_get_link(dev);
-
- return dev->phydev->link;
+ return phylink_ethtool_nway_reset(mac->phylink);
}
static void mtk_get_strings(struct net_device *dev, u32 stringset, u8 *data)
@@ -2580,7 +2728,7 @@ static const struct ethtool_ops mtk_ethtool_ops = {
.get_msglevel = mtk_get_msglevel,
.set_msglevel = mtk_set_msglevel,
.nway_reset = mtk_nway_reset,
- .get_link = mtk_get_link,
+ .get_link = ethtool_op_get_link,
.get_strings = mtk_get_strings,
.get_sset_count = mtk_get_sset_count,
.get_ethtool_stats = mtk_get_ethtool_stats,
@@ -2608,9 +2756,10 @@ static const struct net_device_ops mtk_netdev_ops = {
static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)
{
- struct mtk_mac *mac;
const __be32 *_id = of_get_property(np, "reg", NULL);
- int id, err;
+ struct phylink *phylink;
+ int phy_mode, id, err;
+ struct mtk_mac *mac;
if (!_id) {
dev_err(eth->dev, "missing mac id\n");
@@ -2654,6 +2803,32 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)
u64_stats_init(&mac->hw_stats->syncp);
mac->hw_stats->reg_offset = id * MTK_STAT_OFFSET;
+ /* phylink create */
+ phy_mode = of_get_phy_mode(np);
+ if (phy_mode < 0) {
+ dev_err(eth->dev, "incorrect phy-mode\n");
+ err = -EINVAL;
+ goto free_netdev;
+ }
+
+ /* mac config is not set */
+ mac->interface = PHY_INTERFACE_MODE_NA;
+ mac->mode = MLO_AN_PHY;
+ mac->speed = SPEED_UNKNOWN;
+
+ mac->phylink_config.dev = ð->netdev[id]->dev;
+ mac->phylink_config.type = PHYLINK_NETDEV;
+
+ phylink = phylink_create(&mac->phylink_config,
+ of_fwnode_handle(mac->of_node),
+ phy_mode, &mtk_phylink_ops);
+ if (IS_ERR(phylink)) {
+ err = PTR_ERR(phylink);
+ goto free_netdev;
+ }
+
+ mac->phylink = phylink;
+
SET_NETDEV_DEV(eth->netdev[id], eth->dev);
eth->netdev[id]->watchdog_timeo = 5 * HZ;
eth->netdev[id]->netdev_ops = &mtk_netdev_ops;
@@ -2682,8 +2857,7 @@ static int mtk_probe(struct platform_device *pdev)
{
struct device_node *mac_np;
struct mtk_eth *eth;
- int err;
- int i;
+ int err, i;
eth = devm_kzalloc(&pdev->dev, sizeof(*eth), GFP_KERNEL);
if (!eth)
@@ -2869,6 +3043,7 @@ static int mtk_probe(struct platform_device *pdev)
static int mtk_remove(struct platform_device *pdev)
{
struct mtk_eth *eth = platform_get_drvdata(pdev);
+ struct mtk_mac *mac;
int i;
/* stop all devices to make sure that dma is properly shut down */
@@ -2876,6 +3051,8 @@ static int mtk_remove(struct platform_device *pdev)
if (!eth->netdev[i])
continue;
mtk_stop(eth->netdev[i]);
+ mac = netdev_priv(eth->netdev[i]);
+ phylink_disconnect_phy(mac->phylink);
}
mtk_hw_deinit(eth);
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h
index cc1466a..76bd12c 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h
@@ -14,6 +14,7 @@
#include <linux/of_net.h>
#include <linux/u64_stats_sync.h>
#include <linux/refcount.h>
+#include <linux/phylink.h>
#define MTK_QDMA_PAGE_SIZE 2048
#define MTK_MAX_RX_LENGTH 1536
@@ -330,12 +331,19 @@
#define MAC_MCR_SPEED_100 BIT(2)
#define MAC_MCR_FORCE_DPX BIT(1)
#define MAC_MCR_FORCE_LINK BIT(0)
-#define MAC_MCR_FIXED_LINK (MAC_MCR_MAX_RX_1536 | MAC_MCR_IPG_CFG | \
- MAC_MCR_FORCE_MODE | MAC_MCR_TX_EN | \
- MAC_MCR_RX_EN | MAC_MCR_BACKOFF_EN | \
- MAC_MCR_BACKPR_EN | MAC_MCR_FORCE_RX_FC | \
- MAC_MCR_FORCE_TX_FC | MAC_MCR_SPEED_1000 | \
- MAC_MCR_FORCE_DPX | MAC_MCR_FORCE_LINK)
+#define MAC_MCR_FORCE_LINK_DOWN (MAC_MCR_FORCE_MODE)
+
+/* Mac status registers */
+#define MTK_MAC_MSR(x) (0x10108 + (x * 0x100))
+#define MAC_MSR_EEE1G BIT(7)
+#define MAC_MSR_EEE100M BIT(6)
+#define MAC_MSR_RX_FC BIT(5)
+#define MAC_MSR_TX_FC BIT(4)
+#define MAC_MSR_SPEED_1000 BIT(3)
+#define MAC_MSR_SPEED_100 BIT(2)
+#define MAC_MSR_SPEED_MASK (MAC_MSR_SPEED_1000 | MAC_MSR_SPEED_100)
+#define MAC_MSR_DPX BIT(1)
+#define MAC_MSR_LINK BIT(0)
/* TRGMII RXC control register */
#define TRGMII_RCK_CTRL 0x10300
@@ -404,14 +412,38 @@
/* Register to auto-negotiation restart */
#define SGMSYS_PCS_CONTROL_1 0x0
#define SGMII_AN_RESTART BIT(9)
+#define SGMII_ISOLATE BIT(10)
+#define SGMII_AN_ENABLE BIT(12)
+#define SGMII_LINK_STATYS BIT(18)
+#define SGMII_AN_ABILITY BIT(19)
+#define SGMII_AN_COMPLETE BIT(21)
+#define SGMII_PCS_FAULT BIT(23)
+#define SGMII_AN_EXPANSION_CLR BIT(30)
/* Register to programmable link timer, the unit in 2 * 8ns */
#define SGMSYS_PCS_LINK_TIMER 0x18
#define SGMII_LINK_TIMER_DEFAULT (0x186a0 & GENMASK(19, 0))
/* Register to control remote fault */
-#define SGMSYS_SGMII_MODE 0x20
-#define SGMII_REMOTE_FAULT_DIS BIT(8)
+#define SGMSYS_SGMII_MODE 0x20
+#define SGMII_IF_MODE_BIT0 BIT(0)
+#define SGMII_SPEED_DUPLEX_AN BIT(1)
+#define SGMII_SPEED_10 0x0
+#define SGMII_SPEED_100 BIT(2)
+#define SGMII_SPEED_1000 BIT(3)
+#define SGMII_DUPLEX_FULL BIT(4)
+#define SGMII_IF_MODE_BIT5 BIT(5)
+#define SGMII_REMOTE_FAULT_DIS BIT(8)
+#define SGMII_CODE_SYNC_SET_VAL BIT(9)
+#define SGMII_CODE_SYNC_SET_EN BIT(10)
+#define SGMII_SEND_AN_ERROR_EN BIT(11)
+#define SGMII_IF_MODE_MASK GENMASK(5, 1)
+
+/* Register to set SGMII speed, ANA RG_ Control Signals III*/
+#define SGMSYS_ANA_RG_CS3 0x2028
+#define RG_PHY_SPEED_MASK (BIT(2) | BIT(3))
+#define RG_PHY_SPEED_1_25G 0x0
+#define RG_PHY_SPEED_3_125G BIT(2)
/* Register to power up QPHY */
#define SGMSYS_QPHY_PWR_STATE_CTRL 0xe8
@@ -858,22 +890,23 @@ struct mtk_eth {
/* struct mtk_mac - the structure that holds the info about the MACs of the
* SoC
* @id: The number of the MAC
- * @ge_mode: Interface mode kept for setup restoring
+ * @interface: Interface mode kept for detecting change in hw settings
* @of_node: Our devicetree node
* @hw: Backpointer to our main datastruture
* @hw_stats: Packet statistics counter
- * @trgmii Indicate if the MAC uses TRGMII connected to internal
- switch
*/
struct mtk_mac {
int id;
- int ge_mode;
+ phy_interface_t interface;
+ unsigned int mode;
+ int speed;
struct device_node *of_node;
+ struct phylink *phylink;
+ struct phylink_config phylink_config;
struct mtk_eth *hw;
struct mtk_hw_stats *hw_stats;
__be32 hwlro_ip[MTK_MAX_LRO_IP_CNT];
int hwlro_ip_cnt;
- bool trgmii;
};
/* the struct describing the SoC. these are declared in the soc_xyz.c files */
@@ -888,7 +921,12 @@ u32 mtk_r32(struct mtk_eth *eth, unsigned reg);
int mtk_sgmii_init(struct mtk_sgmii *ss, struct device_node *np,
u32 ana_rgc3);
int mtk_sgmii_setup_mode_an(struct mtk_sgmii *ss, int id);
-int mtk_sgmii_setup_mode_force(struct mtk_sgmii *ss, int id);
-int mtk_setup_hw_path(struct mtk_eth *eth, int mac_id, int phymode);
+int mtk_sgmii_setup_mode_force(struct mtk_sgmii *ss, int id,
+ const struct phylink_link_state *state);
+void mtk_sgmii_restart_an(struct mtk_eth *eth, int mac_id);
+
+int mtk_gmac_sgmii_path_setup(struct mtk_eth *eth, int mac_id);
+int mtk_gmac_gephy_path_setup(struct mtk_eth *eth, int mac_id);
+int mtk_gmac_rgmii_path_setup(struct mtk_eth *eth, int mac_id);
#endif /* MTK_ETH_H */
diff --git a/drivers/net/ethernet/mediatek/mtk_sgmii.c b/drivers/net/ethernet/mediatek/mtk_sgmii.c
index ff509d4..4db27df 100644
--- a/drivers/net/ethernet/mediatek/mtk_sgmii.c
+++ b/drivers/net/ethernet/mediatek/mtk_sgmii.c
@@ -16,8 +16,7 @@
int mtk_sgmii_init(struct mtk_sgmii *ss, struct device_node *r, u32 ana_rgc3)
{
struct device_node *np;
- const char *str;
- int i, err;
+ int i;
ss->ana_rgc3 = ana_rgc3;
@@ -29,19 +28,6 @@ int mtk_sgmii_init(struct mtk_sgmii *ss, struct device_node *r, u32 ana_rgc3)
ss->regmap[i] = syscon_node_to_regmap(np);
if (IS_ERR(ss->regmap[i]))
return PTR_ERR(ss->regmap[i]);
-
- err = of_property_read_string(np, "mediatek,physpeed", &str);
- if (err)
- return err;
-
- if (!strcmp(str, "2500"))
- ss->flags[i] |= MTK_SGMII_PHYSPEED_2500;
- else if (!strcmp(str, "1000"))
- ss->flags[i] |= MTK_SGMII_PHYSPEED_1000;
- else if (!strcmp(str, "auto"))
- ss->flags[i] |= MTK_SGMII_PHYSPEED_AN;
- else
- return -EINVAL;
}
return 0;
@@ -73,27 +59,45 @@ int mtk_sgmii_setup_mode_an(struct mtk_sgmii *ss, int id)
return 0;
}
-int mtk_sgmii_setup_mode_force(struct mtk_sgmii *ss, int id)
+int mtk_sgmii_setup_mode_force(struct mtk_sgmii *ss, int id,
+ const struct phylink_link_state *state)
{
unsigned int val;
- int mode;
if (!ss->regmap[id])
return -EINVAL;
regmap_read(ss->regmap[id], ss->ana_rgc3, &val);
- val &= ~GENMASK(3, 2);
- mode = ss->flags[id] & MTK_SGMII_PHYSPEED_MASK;
- val |= (mode == MTK_SGMII_PHYSPEED_1000) ? 0 : BIT(2);
+ val &= ~RG_PHY_SPEED_MASK;
+ if (state->interface == PHY_INTERFACE_MODE_2500BASEX)
+ val |= RG_PHY_SPEED_3_125G;
regmap_write(ss->regmap[id], ss->ana_rgc3, val);
/* Disable SGMII AN */
regmap_read(ss->regmap[id], SGMSYS_PCS_CONTROL_1, &val);
- val &= ~BIT(12);
+ val &= ~SGMII_AN_ENABLE;
regmap_write(ss->regmap[id], SGMSYS_PCS_CONTROL_1, val);
/* SGMII force mode setting */
- val = 0x31120019;
+ regmap_read(ss->regmap[id], SGMSYS_SGMII_MODE, &val);
+ val &= ~SGMII_IF_MODE_MASK;
+
+ switch (state->speed) {
+ case SPEED_10:
+ val |= SGMII_SPEED_10;
+ break;
+ case SPEED_100:
+ val |= SGMII_SPEED_100;
+ break;
+ case SPEED_2500:
+ case SPEED_1000:
+ val |= SGMII_SPEED_1000;
+ break;
+ };
+
+ if (state->duplex == DUPLEX_FULL)
+ val |= SGMII_DUPLEX_FULL;
+
regmap_write(ss->regmap[id], SGMSYS_SGMII_MODE, val);
/* Release PHYA power down state */
@@ -103,3 +107,20 @@ int mtk_sgmii_setup_mode_force(struct mtk_sgmii *ss, int id)
return 0;
}
+
+void mtk_sgmii_restart_an(struct mtk_eth *eth, int mac_id)
+{
+ struct mtk_sgmii *ss = eth->sgmii;
+ unsigned int val, sid;
+
+ /* Decide how GMAC and SGMIISYS be mapped */
+ sid = (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_SGMII)) ?
+ 0 : mac_id;
+
+ if (!ss->regmap[sid])
+ return;
+
+ regmap_read(ss->regmap[sid], SGMSYS_PCS_CONTROL_1, &val);
+ val |= SGMII_AN_RESTART;
+ regmap_write(ss->regmap[sid], SGMSYS_PCS_CONTROL_1, val);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
index 94c59939..d8313e2 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
@@ -639,7 +639,7 @@ static unsigned long *ptys2ethtool_link_mode(struct ptys2ethtool_config *cfg,
#define MLX4_BUILD_PTYS2ETHTOOL_CONFIG(reg_, speed_, ...) \
({ \
struct ptys2ethtool_config *cfg; \
- const unsigned int modes[] = { __VA_ARGS__ }; \
+ static const unsigned int modes[] = { __VA_ARGS__ }; \
unsigned int i; \
cfg = &ptys2ethtool_map[reg_]; \
cfg->speed = speed_; \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
index 37fef8c..0dba272 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
@@ -10,6 +10,7 @@
imply PTP_1588_CLOCK
imply VXLAN
imply MLXFW
+ imply PCI_HYPERV_INTERFACE
default n
---help---
Core driver for low level functionality of the ConnectX-4 and
@@ -32,7 +33,6 @@
config MLX5_CORE_EN
bool "Mellanox 5th generation network adapters (ConnectX series) Ethernet support"
depends on NETDEVICES && ETHERNET && INET && PCI && MLX5_CORE
- depends on IPV6=y || IPV6=n || MLX5_CORE=m
select PAGE_POOL
select DIMLIB
default n
@@ -154,3 +154,10 @@
Build support for TLS cryptography-offload accelaration in the NIC.
Note: Support for hardware with this capability needs to be selected
for this option to become available.
+
+config MLX5_SW_STEERING
+ bool "Mellanox Technologies software-managed steering"
+ depends on MLX5_CORE_EN && MLX5_ESWITCH
+ default y
+ help
+ Build support for software-managed steering in the NIC.
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index f4de9cc..5708fcc 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -15,7 +15,7 @@
health.o mcg.o cq.o alloc.o qp.o port.o mr.o pd.o \
transobj.o vport.o sriov.o fs_cmd.o fs_core.o pci_irq.o \
fs_counters.o rl.o lag.o dev.o events.o wq.o lib/gid.o \
- lib/devcom.o lib/pci_vsc.o diag/fs_tracepoint.o \
+ lib/devcom.o lib/pci_vsc.o lib/dm.o diag/fs_tracepoint.o \
diag/fw_tracer.o diag/crdump.o devlink.o
#
@@ -67,3 +67,10 @@
mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/tls_stats.o \
en_accel/ktls.o en_accel/ktls_tx.o
+
+mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o \
+ steering/dr_matcher.o steering/dr_rule.o \
+ steering/dr_icm_pool.o steering/dr_crc32.o \
+ steering/dr_ste.o steering/dr_send.o \
+ steering/dr_cmd.o steering/dr_fw.o \
+ steering/dr_action.o steering/fs_dr.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
index 973f908..ea934cd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -446,6 +446,8 @@ static int mlx5_internal_err_ret_value(struct mlx5_core_dev *dev, u16 op,
case MLX5_CMD_OP_CREATE_UMEM:
case MLX5_CMD_OP_DESTROY_UMEM:
case MLX5_CMD_OP_ALLOC_MEMIC:
+ case MLX5_CMD_OP_MODIFY_XRQ:
+ case MLX5_CMD_OP_RELEASE_XRQ_ERROR:
*status = MLX5_DRIVER_STATUS_ABORTED;
*synd = MLX5_DRIVER_SYND;
return -EIO;
@@ -637,6 +639,8 @@ const char *mlx5_command_str(int command)
MLX5_COMMAND_STR_CASE(DESTROY_UCTX);
MLX5_COMMAND_STR_CASE(CREATE_UMEM);
MLX5_COMMAND_STR_CASE(DESTROY_UMEM);
+ MLX5_COMMAND_STR_CASE(RELEASE_XRQ_ERROR);
+ MLX5_COMMAND_STR_CASE(MODIFY_XRQ);
default: return "unknown command opcode";
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c
index a400f44..381925c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c
@@ -4,6 +4,7 @@
#include <devlink.h>
#include "mlx5_core.h"
+#include "fs_core.h"
#include "eswitch.h"
static int mlx5_devlink_flash_update(struct devlink *devlink,
@@ -107,12 +108,121 @@ void mlx5_devlink_free(struct devlink *devlink)
devlink_free(devlink);
}
+static int mlx5_devlink_fs_mode_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+ char *value = val.vstr;
+ int err = 0;
+
+ if (!strcmp(value, "dmfs")) {
+ return 0;
+ } else if (!strcmp(value, "smfs")) {
+ u8 eswitch_mode;
+ bool smfs_cap;
+
+ eswitch_mode = mlx5_eswitch_mode(dev->priv.eswitch);
+ smfs_cap = mlx5_fs_dr_is_supported(dev);
+
+ if (!smfs_cap) {
+ err = -EOPNOTSUPP;
+ NL_SET_ERR_MSG_MOD(extack,
+ "Software managed steering is not supported by current device");
+ }
+
+ else if (eswitch_mode == MLX5_ESWITCH_OFFLOADS) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Software managed steering is not supported when eswitch offloads enabled.");
+ err = -EOPNOTSUPP;
+ }
+ } else {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Bad parameter: supported values are [\"dmfs\", \"smfs\"]");
+ err = -EINVAL;
+ }
+
+ return err;
+}
+
+static int mlx5_devlink_fs_mode_set(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+ enum mlx5_flow_steering_mode mode;
+
+ if (!strcmp(ctx->val.vstr, "smfs"))
+ mode = MLX5_FLOW_STEERING_MODE_SMFS;
+ else
+ mode = MLX5_FLOW_STEERING_MODE_DMFS;
+ dev->priv.steering->mode = mode;
+
+ return 0;
+}
+
+static int mlx5_devlink_fs_mode_get(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+
+ if (dev->priv.steering->mode == MLX5_FLOW_STEERING_MODE_SMFS)
+ strcpy(ctx->val.vstr, "smfs");
+ else
+ strcpy(ctx->val.vstr, "dmfs");
+ return 0;
+}
+
+enum mlx5_devlink_param_id {
+ MLX5_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX,
+ MLX5_DEVLINK_PARAM_FLOW_STEERING_MODE,
+};
+
+static const struct devlink_param mlx5_devlink_params[] = {
+ DEVLINK_PARAM_DRIVER(MLX5_DEVLINK_PARAM_FLOW_STEERING_MODE,
+ "flow_steering_mode", DEVLINK_PARAM_TYPE_STRING,
+ BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+ mlx5_devlink_fs_mode_get, mlx5_devlink_fs_mode_set,
+ mlx5_devlink_fs_mode_validate),
+};
+
+static void mlx5_devlink_set_params_init_values(struct devlink *devlink)
+{
+ struct mlx5_core_dev *dev = devlink_priv(devlink);
+ union devlink_param_value value;
+
+ if (dev->priv.steering->mode == MLX5_FLOW_STEERING_MODE_DMFS)
+ strcpy(value.vstr, "dmfs");
+ else
+ strcpy(value.vstr, "smfs");
+ devlink_param_driverinit_value_set(devlink,
+ MLX5_DEVLINK_PARAM_FLOW_STEERING_MODE,
+ value);
+}
+
int mlx5_devlink_register(struct devlink *devlink, struct device *dev)
{
- return devlink_register(devlink, dev);
+ int err;
+
+ err = devlink_register(devlink, dev);
+ if (err)
+ return err;
+
+ err = devlink_params_register(devlink, mlx5_devlink_params,
+ ARRAY_SIZE(mlx5_devlink_params));
+ if (err)
+ goto params_reg_err;
+ mlx5_devlink_set_params_init_values(devlink);
+ devlink_params_publish(devlink);
+ return 0;
+
+params_reg_err:
+ devlink_unregister(devlink);
+ return err;
}
void mlx5_devlink_unregister(struct devlink *devlink)
{
+ devlink_params_unregister(devlink, mlx5_devlink_params,
+ ARRAY_SIZE(mlx5_devlink_params));
devlink_unregister(devlink);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c
index 8a4930c..94d7b69 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c
@@ -546,16 +546,17 @@ static void mlx5_fw_tracer_save_trace(struct mlx5_fw_tracer *tracer,
trace_data->timestamp = timestamp;
trace_data->lost = lost;
trace_data->event_id = event_id;
- strncpy(trace_data->msg, msg, TRACE_STR_MSG);
+ strscpy_pad(trace_data->msg, msg, TRACE_STR_MSG);
tracer->st_arr.saved_traces_index =
(tracer->st_arr.saved_traces_index + 1) & (SAVED_TRACES_NUM - 1);
mutex_unlock(&tracer->st_arr.lock);
}
-static void mlx5_tracer_print_trace(struct tracer_string_format *str_frmt,
- struct mlx5_core_dev *dev,
- u64 trace_timestamp)
+static noinline
+void mlx5_tracer_print_trace(struct tracer_string_format *str_frmt,
+ struct mlx5_core_dev *dev,
+ u64 trace_timestamp)
{
char tmp[512];
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 4467927..8d76452 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -163,6 +163,14 @@ enum mlx5e_rq_group {
#define MLX5E_NUM_RQ_GROUPS(g) (1 + MLX5E_RQ_GROUP_##g)
};
+static inline u8 mlx5e_get_num_lag_ports(struct mlx5_core_dev *mdev)
+{
+ if (mlx5_lag_is_lacp_owner(mdev))
+ return 1;
+
+ return clamp_t(u8, MLX5_CAP_GEN(mdev, num_lag_ports), 1, MLX5_MAX_PORTS);
+}
+
static inline u16 mlx5_min_rx_wqes(int wq_type, u32 wq_size)
{
switch (wq_type) {
@@ -705,6 +713,7 @@ struct mlx5e_channel {
struct net_device *netdev;
__be32 mkey_be;
u8 num_tc;
+ u8 lag_port;
/* XDP_REDIRECT */
struct mlx5e_xdpsq xdpsq;
@@ -818,7 +827,7 @@ struct mlx5e_priv {
struct mlx5e_rq drop_rq;
struct mlx5e_channels channels;
- u32 tisn[MLX5E_MAX_NUM_TC];
+ u32 tisn[MLX5_MAX_PORTS][MLX5E_MAX_NUM_TC];
struct mlx5e_rqt indir_rqt;
struct mlx5e_tir indir_tir[MLX5E_NUM_INDIR_TIRS];
struct mlx5e_tir inner_indir_tir[MLX5E_NUM_INDIR_TIRS];
@@ -1056,12 +1065,6 @@ int mlx5e_modify_sq(struct mlx5_core_dev *mdev, u32 sqn,
void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq);
void mlx5e_tx_disable_queue(struct netdev_queue *txq);
-static inline bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
-{
- return (MLX5_CAP_ETH(mdev, tunnel_stateless_gre) &&
- MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ft_field_support.inner_ip_version));
-}
-
static inline bool mlx5_tx_swp_supported(struct mlx5_core_dev *mdev)
{
return MLX5_CAP_ETH(mdev, swp) &&
@@ -1107,6 +1110,7 @@ int mlx5e_create_tis(struct mlx5_core_dev *mdev, void *in, u32 *tisn);
void mlx5e_destroy_tis(struct mlx5_core_dev *mdev, u32 tisn);
int mlx5e_create_tises(struct mlx5e_priv *priv);
+void mlx5e_destroy_tises(struct mlx5e_priv *priv);
int mlx5e_update_nic_rx(struct mlx5e_priv *priv);
void mlx5e_update_carrier(struct mlx5e_priv *priv);
int mlx5e_close(struct net_device *netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index ca2161b..68d5930 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -95,9 +95,15 @@ struct mlx5e_tirc_config {
enum mlx5e_tunnel_types {
MLX5E_TT_IPV4_GRE,
MLX5E_TT_IPV6_GRE,
+ MLX5E_TT_IPV4_IPIP,
+ MLX5E_TT_IPV6_IPIP,
+ MLX5E_TT_IPV4_IPV6,
+ MLX5E_TT_IPV6_IPV6,
MLX5E_NUM_TUNNEL_TT,
};
+bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev);
+
/* L3/L4 traffic type classifier */
struct mlx5e_ttc_table {
struct mlx5e_flow_table ft;
@@ -232,5 +238,8 @@ void mlx5e_disable_cvlan_filter(struct mlx5e_priv *priv);
int mlx5e_create_flow_steering(struct mlx5e_priv *priv);
void mlx5e_destroy_flow_steering(struct mlx5e_priv *priv);
+bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_type);
+bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev);
+
#endif /* __MLX5E_FLOW_STEER_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
index c37b4ac..b3a249b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
@@ -30,22 +30,21 @@ mlx5e_hv_vhca_fill_ring_stats(struct mlx5e_priv *priv, int ch,
}
}
-static void mlx5e_hv_vhca_fill_stats(struct mlx5e_priv *priv, u64 *data,
+static void mlx5e_hv_vhca_fill_stats(struct mlx5e_priv *priv, void *data,
int buf_len)
{
int ch, i = 0;
for (ch = 0; ch < priv->max_nch; ch++) {
- u64 *buf = data + i;
+ void *buf = data + i;
if (WARN_ON_ONCE(buf +
sizeof(struct mlx5e_hv_vhca_per_ring_stats) >
data + buf_len))
return;
- mlx5e_hv_vhca_fill_ring_stats(priv, ch,
- (struct mlx5e_hv_vhca_per_ring_stats *)buf);
- i += sizeof(struct mlx5e_hv_vhca_per_ring_stats) / sizeof(u64);
+ mlx5e_hv_vhca_fill_ring_stats(priv, ch, buf);
+ i += sizeof(struct mlx5e_hv_vhca_per_ring_stats);
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
index 79301d1..eb2e1f2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
@@ -25,18 +25,33 @@ u16 mlx5e_get_linear_rq_headroom(struct mlx5e_params *params,
return headroom;
}
-u32 mlx5e_rx_get_linear_frag_sz(struct mlx5e_params *params,
- struct mlx5e_xsk_param *xsk)
+u32 mlx5e_rx_get_min_frag_sz(struct mlx5e_params *params,
+ struct mlx5e_xsk_param *xsk)
{
u32 hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu);
u16 linear_rq_headroom = mlx5e_get_linear_rq_headroom(params, xsk);
- u32 frag_sz = linear_rq_headroom + hw_mtu;
+
+ return linear_rq_headroom + hw_mtu;
+}
+
+u32 mlx5e_rx_get_linear_frag_sz(struct mlx5e_params *params,
+ struct mlx5e_xsk_param *xsk)
+{
+ u32 frag_sz = mlx5e_rx_get_min_frag_sz(params, xsk);
/* AF_XDP doesn't build SKBs in place. */
if (!xsk)
frag_sz = MLX5_SKB_FRAG_SZ(frag_sz);
- /* XDP in mlx5e doesn't support multiple packets per page. */
+ /* XDP in mlx5e doesn't support multiple packets per page. AF_XDP is a
+ * special case. It can run with frames smaller than a page, as it
+ * doesn't allocate pages dynamically. However, here we pretend that
+ * fragments are page-sized: it allows to treat XSK frames like pages
+ * by redirecting alloc and free operations to XSK rings and by using
+ * the fact there are no multiple packets per "page" (which is a frame).
+ * The latter is important, because frames may come in a random order,
+ * and we will have trouble assemblying a real page of multiple frames.
+ */
if (mlx5e_rx_is_xdp(params, xsk))
frag_sz = max_t(u32, frag_sz, PAGE_SIZE);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
index 3a615d6..989d8f4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h
@@ -76,6 +76,8 @@ static inline bool mlx5e_qid_validate(const struct mlx5e_profile *profile,
u16 mlx5e_get_linear_rq_headroom(struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk);
+u32 mlx5e_rx_get_min_frag_sz(struct mlx5e_params *params,
+ struct mlx5e_xsk_param *xsk);
u32 mlx5e_rx_get_linear_frag_sz(struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk);
u8 mlx5e_mpwqe_log_pkts_per_wqe(struct mlx5e_params *params,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c
index 4c4620d..f8ee18b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c
@@ -291,14 +291,14 @@ int mlx5e_tc_tun_create_header_ipv4(struct mlx5e_priv *priv,
*/
goto out;
}
-
- err = mlx5_packet_reformat_alloc(priv->mdev,
- e->reformat_type,
- ipv4_encap_size, encap_header,
- MLX5_FLOW_NAMESPACE_FDB,
- &e->encap_id);
- if (err)
+ e->pkt_reformat = mlx5_packet_reformat_alloc(priv->mdev,
+ e->reformat_type,
+ ipv4_encap_size, encap_header,
+ MLX5_FLOW_NAMESPACE_FDB);
+ if (IS_ERR(e->pkt_reformat)) {
+ err = PTR_ERR(e->pkt_reformat);
goto destroy_neigh_entry;
+ }
e->flags |= MLX5_ENCAP_ENTRY_VALID;
mlx5e_rep_queue_neigh_stats_work(netdev_priv(out_dev));
@@ -407,13 +407,14 @@ int mlx5e_tc_tun_create_header_ipv6(struct mlx5e_priv *priv,
goto out;
}
- err = mlx5_packet_reformat_alloc(priv->mdev,
- e->reformat_type,
- ipv6_encap_size, encap_header,
- MLX5_FLOW_NAMESPACE_FDB,
- &e->encap_id);
- if (err)
+ e->pkt_reformat = mlx5_packet_reformat_alloc(priv->mdev,
+ e->reformat_type,
+ ipv6_encap_size, encap_header,
+ MLX5_FLOW_NAMESPACE_FDB);
+ if (IS_ERR(e->pkt_reformat)) {
+ err = PTR_ERR(e->pkt_reformat);
goto destroy_neigh_entry;
+ }
e->flags |= MLX5_ENCAP_ENTRY_VALID;
mlx5e_rep_queue_neigh_stats_work(netdev_priv(out_dev));
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
index 1ed5c33..f049e0a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
@@ -122,6 +122,7 @@ bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di,
void *va, u16 *rx_headroom, u32 *len, bool xsk)
{
struct bpf_prog *prog = READ_ONCE(rq->xdp_prog);
+ struct xdp_umem *umem = rq->umem;
struct xdp_buff xdp;
u32 act;
int err;
@@ -138,8 +139,11 @@ bool mlx5e_xdp_handle(struct mlx5e_rq *rq, struct mlx5e_dma_info *di,
xdp.rxq = &rq->xdp_rxq;
act = bpf_prog_run_xdp(prog, &xdp);
- if (xsk)
- xdp.handle += xdp.data - xdp.data_hard_start;
+ if (xsk) {
+ u64 off = xdp.data - xdp.data_hard_start;
+
+ xdp.handle = xsk_umem_adjust_offset(umem, xdp.handle, off);
+ }
switch (act) {
case XDP_PASS:
*rx_headroom = xdp.data - xdp.data_hard_start;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
index 6a55573..475b6bd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c
@@ -24,7 +24,8 @@ int mlx5e_xsk_page_alloc_umem(struct mlx5e_rq *rq,
if (!xsk_umem_peek_addr_rq(umem, &handle))
return -ENOMEM;
- dma_info->xsk.handle = handle + rq->buff.umem_headroom;
+ dma_info->xsk.handle = xsk_umem_adjust_offset(umem, handle,
+ rq->buff.umem_headroom);
dma_info->xsk.data = xdp_umem_get_data(umem, dma_info->xsk.handle);
/* No need to add headroom to the DMA address. In striding RQ case, we
@@ -104,7 +105,7 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_mpwrq_linear(struct mlx5e_rq *rq,
/* head_offset is not used in this function, because di->xsk.data and
* di->addr point directly to the necessary place. Furthermore, in the
- * current implementation, one page = one packet = one frame, so
+ * current implementation, UMR pages are mapped to XSK frames, so
* head_offset should always be 0.
*/
WARN_ON_ONCE(head_offset);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
index 307b923..cab0e93 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.h
@@ -5,6 +5,7 @@
#define __MLX5_EN_XSK_RX_H__
#include "en.h"
+#include <net/xdp_sock.h>
/* RX data path */
@@ -24,4 +25,17 @@ struct sk_buff *mlx5e_xsk_skb_from_cqe_linear(struct mlx5e_rq *rq,
struct mlx5e_wqe_frag_info *wi,
u32 cqe_bcnt);
+static inline bool mlx5e_xsk_update_rx_wakeup(struct mlx5e_rq *rq, bool alloc_err)
+{
+ if (!xsk_umem_uses_need_wakeup(rq->umem))
+ return alloc_err;
+
+ if (unlikely(alloc_err))
+ xsk_set_rx_need_wakeup(rq->umem);
+ else
+ xsk_clear_rx_need_wakeup(rq->umem);
+
+ return false;
+}
+
#endif /* __MLX5_EN_XSK_RX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c
index d360750..631af8d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c
@@ -4,18 +4,23 @@
#include "setup.h"
#include "en/params.h"
+/* It matches XDP_UMEM_MIN_CHUNK_SIZE, but as this constant is private and may
+ * change unexpectedly, and mlx5e has a minimum valid stride size for striding
+ * RQ, keep this check in the driver.
+ */
+#define MLX5E_MIN_XSK_CHUNK_SIZE 2048
+
bool mlx5e_validate_xsk_param(struct mlx5e_params *params,
struct mlx5e_xsk_param *xsk,
struct mlx5_core_dev *mdev)
{
- /* AF_XDP doesn't support frames larger than PAGE_SIZE, and the current
- * mlx5e XDP implementation doesn't support multiple packets per page.
- */
- if (xsk->chunk_size != PAGE_SIZE)
+ /* AF_XDP doesn't support frames larger than PAGE_SIZE. */
+ if (xsk->chunk_size > PAGE_SIZE ||
+ xsk->chunk_size < MLX5E_MIN_XSK_CHUNK_SIZE)
return false;
/* Current MTU and XSK headroom don't allow packets to fit the frames. */
- if (mlx5e_rx_get_linear_frag_sz(params, xsk) > xsk->chunk_size)
+ if (mlx5e_rx_get_min_frag_sz(params, xsk) > xsk->chunk_size)
return false;
/* frag_sz is different for regular and XSK RQs, so ensure that linear
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
index fd2c75b..8782747 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
@@ -7,7 +7,7 @@
#include "en/params.h"
#include <net/xdp_sock.h>
-int mlx5e_xsk_async_xmit(struct net_device *dev, u32 qid)
+int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags)
{
struct mlx5e_priv *priv = netdev_priv(dev);
struct mlx5e_params *params = &priv->channels.params;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h
index 7add18b..79b487d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.h
@@ -5,11 +5,23 @@
#define __MLX5_EN_XSK_TX_H__
#include "en.h"
+#include <net/xdp_sock.h>
/* TX data path */
-int mlx5e_xsk_async_xmit(struct net_device *dev, u32 qid);
+int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags);
bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget);
+static inline void mlx5e_xsk_update_tx_wakeup(struct mlx5e_xdpsq *sq)
+{
+ if (!xsk_umem_uses_need_wakeup(sq->umem))
+ return;
+
+ if (sq->pc != sq->cc)
+ xsk_clear_tx_need_wakeup(sq->umem);
+ else
+ xsk_set_tx_need_wakeup(sq->umem);
+}
+
#endif /* __MLX5_EN_XSK_TX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c
index 8b93101..d195366 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c
@@ -109,13 +109,15 @@ build_progress_params(struct mlx5e_tx_wqe *wqe, u16 pc, u32 sqn,
static void tx_fill_wi(struct mlx5e_txqsq *sq,
u16 pi, u8 num_wqebbs,
- skb_frag_t *resync_dump_frag)
+ skb_frag_t *resync_dump_frag,
+ u32 num_bytes)
{
struct mlx5e_tx_wqe_info *wi = &sq->db.wqe_info[pi];
wi->skb = NULL;
wi->num_wqebbs = num_wqebbs;
wi->resync_dump_frag = resync_dump_frag;
+ wi->num_bytes = num_bytes;
}
void mlx5e_ktls_tx_offload_set_pending(struct mlx5e_ktls_offload_context_tx *priv_tx)
@@ -143,7 +145,7 @@ post_static_params(struct mlx5e_txqsq *sq,
umr_wqe = mlx5e_sq_fetch_wqe(sq, MLX5E_KTLS_STATIC_UMR_WQE_SZ, &pi);
build_static_params(umr_wqe, sq->pc, sq->sqn, priv_tx, fence);
- tx_fill_wi(sq, pi, MLX5E_KTLS_STATIC_WQEBBS, NULL);
+ tx_fill_wi(sq, pi, MLX5E_KTLS_STATIC_WQEBBS, NULL, 0);
sq->pc += MLX5E_KTLS_STATIC_WQEBBS;
}
@@ -157,7 +159,7 @@ post_progress_params(struct mlx5e_txqsq *sq,
wqe = mlx5e_sq_fetch_wqe(sq, MLX5E_KTLS_PROGRESS_WQE_SZ, &pi);
build_progress_params(wqe, sq->pc, sq->sqn, priv_tx, fence);
- tx_fill_wi(sq, pi, MLX5E_KTLS_PROGRESS_WQEBBS, NULL);
+ tx_fill_wi(sq, pi, MLX5E_KTLS_PROGRESS_WQEBBS, NULL, 0);
sq->pc += MLX5E_KTLS_PROGRESS_WQEBBS;
}
@@ -248,43 +250,36 @@ tx_post_resync_params(struct mlx5e_txqsq *sq,
mlx5e_ktls_tx_post_param_wqes(sq, priv_tx, skip_static_post, true);
}
+struct mlx5e_dump_wqe {
+ struct mlx5_wqe_ctrl_seg ctrl;
+ struct mlx5_wqe_data_seg data;
+};
+
static int
-tx_post_resync_dump(struct mlx5e_txqsq *sq, struct sk_buff *skb,
- skb_frag_t *frag, u32 tisn, bool first)
+tx_post_resync_dump(struct mlx5e_txqsq *sq, skb_frag_t *frag, u32 tisn, bool first)
{
struct mlx5_wqe_ctrl_seg *cseg;
- struct mlx5_wqe_eth_seg *eseg;
struct mlx5_wqe_data_seg *dseg;
- struct mlx5e_tx_wqe *wqe;
+ struct mlx5e_dump_wqe *wqe;
dma_addr_t dma_addr = 0;
- u16 ds_cnt, ds_cnt_inl;
u8 num_wqebbs;
- u16 pi, ihs;
+ u16 ds_cnt;
int fsz;
-
- ds_cnt = sizeof(*wqe) / MLX5_SEND_WQE_DS;
- ihs = eth_get_headlen(skb->dev, skb->data, skb_headlen(skb));
- ds_cnt_inl = DIV_ROUND_UP(ihs - INL_HDR_START_SZ, MLX5_SEND_WQE_DS);
- ds_cnt += ds_cnt_inl;
- ds_cnt += 1; /* one frag */
+ u16 pi;
wqe = mlx5e_sq_fetch_wqe(sq, sizeof(*wqe), &pi);
+ ds_cnt = sizeof(*wqe) / MLX5_SEND_WQE_DS;
num_wqebbs = DIV_ROUND_UP(ds_cnt, MLX5_SEND_WQEBB_NUM_DS);
cseg = &wqe->ctrl;
- eseg = &wqe->eth;
- dseg = wqe->data;
+ dseg = &wqe->data;
cseg->opmod_idx_opcode = cpu_to_be32((sq->pc << 8) | MLX5_OPCODE_DUMP);
cseg->qpn_ds = cpu_to_be32((sq->sqn << 8) | ds_cnt);
cseg->tisn = cpu_to_be32(tisn << 8);
cseg->fm_ce_se = first ? MLX5_FENCE_MODE_INITIATOR_SMALL : 0;
- eseg->inline_hdr.sz = cpu_to_be16(ihs);
- memcpy(eseg->inline_hdr.start, skb->data, ihs);
- dseg += ds_cnt_inl;
-
fsz = skb_frag_size(frag);
dma_addr = skb_frag_dma_map(sq->pdev, frag, 0, fsz,
DMA_TO_DEVICE);
@@ -296,7 +291,7 @@ tx_post_resync_dump(struct mlx5e_txqsq *sq, struct sk_buff *skb,
dseg->byte_count = cpu_to_be32(fsz);
mlx5e_dma_push(sq, dma_addr, fsz, MLX5E_DMA_MAP_PAGE);
- tx_fill_wi(sq, pi, num_wqebbs, frag);
+ tx_fill_wi(sq, pi, num_wqebbs, frag, fsz);
sq->pc += num_wqebbs;
WARN(num_wqebbs > MLX5E_KTLS_MAX_DUMP_WQEBBS,
@@ -323,7 +318,7 @@ static void tx_post_fence_nop(struct mlx5e_txqsq *sq)
struct mlx5_wq_cyc *wq = &sq->wq;
u16 pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
- tx_fill_wi(sq, pi, 1, NULL);
+ tx_fill_wi(sq, pi, 1, NULL, 0);
mlx5e_post_nop_fence(wq, sq->sqn, &sq->pc);
}
@@ -375,8 +370,7 @@ mlx5e_ktls_tx_handle_ooo(struct mlx5e_ktls_offload_context_tx *priv_tx,
tx_post_resync_params(sq, priv_tx, info.rcd_sn);
for (i = 0; i < info.nr_frags; i++)
- if (tx_post_resync_dump(sq, skb, info.frags[i],
- priv_tx->tisn, !i))
+ if (tx_post_resync_dump(sq, info.frags[i], priv_tx->tisn, !i))
goto err_out;
/* If no dump WQE was sent, we need to have a fence NOP WQE before the
@@ -412,7 +406,7 @@ struct sk_buff *mlx5e_ktls_handle_tx_skb(struct net_device *netdev,
goto out;
tls_ctx = tls_get_ctx(skb->sk);
- if (unlikely(WARN_ON_ONCE(tls_ctx->netdev != netdev)))
+ if (WARN_ON_ONCE(tls_ctx->netdev != netdev))
goto err_out;
priv_tx = mlx5e_get_ktls_tx_priv_ctx(tls_ctx);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 7347d67..c5a9c20 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -1431,7 +1431,7 @@ static __u32 mlx5e_get_wol_supported(struct mlx5_core_dev *mdev)
return ret;
}
-static __u32 mlx5e_refomrat_wol_mode_mlx5_to_linux(u8 mode)
+static __u32 mlx5e_reformat_wol_mode_mlx5_to_linux(u8 mode)
{
__u32 ret = 0;
@@ -1459,7 +1459,7 @@ static __u32 mlx5e_refomrat_wol_mode_mlx5_to_linux(u8 mode)
return ret;
}
-static u8 mlx5e_refomrat_wol_mode_linux_to_mlx5(__u32 mode)
+static u8 mlx5e_reformat_wol_mode_linux_to_mlx5(__u32 mode)
{
u8 ret = 0;
@@ -1505,7 +1505,7 @@ static void mlx5e_get_wol(struct net_device *netdev,
if (err)
return;
- wol->wolopts = mlx5e_refomrat_wol_mode_mlx5_to_linux(mlx5_wol_mode);
+ wol->wolopts = mlx5e_reformat_wol_mode_mlx5_to_linux(mlx5_wol_mode);
}
static int mlx5e_set_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
@@ -1521,7 +1521,7 @@ static int mlx5e_set_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
if (wol->wolopts & ~wol_supported)
return -EINVAL;
- mlx5_wol_mode = mlx5e_refomrat_wol_mode_linux_to_mlx5(wol->wolopts);
+ mlx5_wol_mode = mlx5e_reformat_wol_mode_linux_to_mlx5(wol->wolopts);
return mlx5_set_port_wol(mdev, mlx5_wol_mode);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
index 76cc10e..15b7f0f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs.c
@@ -747,8 +747,55 @@ static struct mlx5e_etype_proto ttc_tunnel_rules[] = {
.etype = ETH_P_IPV6,
.proto = IPPROTO_GRE,
},
+ [MLX5E_TT_IPV4_IPIP] = {
+ .etype = ETH_P_IP,
+ .proto = IPPROTO_IPIP,
+ },
+ [MLX5E_TT_IPV6_IPIP] = {
+ .etype = ETH_P_IPV6,
+ .proto = IPPROTO_IPIP,
+ },
+ [MLX5E_TT_IPV4_IPV6] = {
+ .etype = ETH_P_IP,
+ .proto = IPPROTO_IPV6,
+ },
+ [MLX5E_TT_IPV6_IPV6] = {
+ .etype = ETH_P_IPV6,
+ .proto = IPPROTO_IPV6,
+ },
+
};
+bool mlx5e_tunnel_proto_supported(struct mlx5_core_dev *mdev, u8 proto_type)
+{
+ switch (proto_type) {
+ case IPPROTO_GRE:
+ return MLX5_CAP_ETH(mdev, tunnel_stateless_gre);
+ case IPPROTO_IPIP:
+ case IPPROTO_IPV6:
+ return MLX5_CAP_ETH(mdev, tunnel_stateless_ip_over_ip);
+ default:
+ return false;
+ }
+}
+
+bool mlx5e_any_tunnel_proto_supported(struct mlx5_core_dev *mdev)
+{
+ int tt;
+
+ for (tt = 0; tt < MLX5E_NUM_TUNNEL_TT; tt++) {
+ if (mlx5e_tunnel_proto_supported(mdev, ttc_tunnel_rules[tt].proto))
+ return true;
+ }
+ return false;
+}
+
+bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
+{
+ return (mlx5e_any_tunnel_proto_supported(mdev) &&
+ MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ft_field_support.inner_ip_version));
+}
+
static u8 mlx5e_etype_to_ipv(u16 ethertype)
{
if (ethertype == ETH_P_IP)
@@ -838,6 +885,9 @@ static int mlx5e_generate_ttc_table_rules(struct mlx5e_priv *priv,
dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE;
dest.ft = params->inner_ttc->ft.t;
for (tt = 0; tt < MLX5E_NUM_TUNNEL_TT; tt++) {
+ if (!mlx5e_tunnel_proto_supported(priv->mdev,
+ ttc_tunnel_rules[tt].proto))
+ continue;
rules[tt] = mlx5e_generate_ttc_rule(priv, ft, &dest,
ttc_tunnel_rules[tt].etype,
ttc_tunnel_rules[tt].proto);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index fa4bf2d..7569287 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -1315,7 +1315,6 @@ static int mlx5e_open_txqsq(struct mlx5e_channel *c,
return 0;
err_free_txqsq:
- clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
mlx5e_free_txqsq(sq);
return err;
@@ -1403,7 +1402,6 @@ int mlx5e_open_icosq(struct mlx5e_channel *c, struct mlx5e_params *params,
return 0;
err_free_icosq:
- clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
mlx5e_free_icosq(sq);
return err;
@@ -1442,7 +1440,7 @@ int mlx5e_open_xdpsq(struct mlx5e_channel *c, struct mlx5e_params *params,
return err;
csp.tis_lst_sz = 1;
- csp.tisn = c->priv->tisn[0]; /* tc = 0 */
+ csp.tisn = c->priv->tisn[c->lag_port][0]; /* tc = 0 */
csp.cqn = sq->cq.mcq.cqn;
csp.wq_ctrl = &sq->wq_ctrl;
csp.min_inline_mode = sq->min_inline_mode;
@@ -1692,7 +1690,7 @@ static int mlx5e_open_sqs(struct mlx5e_channel *c,
for (tc = 0; tc < params->num_tc; tc++) {
int txq_ix = c->ix + tc * priv->max_nch;
- err = mlx5e_open_txqsq(c, c->priv->tisn[tc], txq_ix,
+ err = mlx5e_open_txqsq(c, c->priv->tisn[c->lag_port][tc], txq_ix,
params, &cparam->sq, &c->sq[tc], tc);
if (err)
goto err_close_sqs;
@@ -1926,6 +1924,13 @@ static void mlx5e_close_queues(struct mlx5e_channel *c)
mlx5e_close_cq(&c->icosq.cq);
}
+static u8 mlx5e_enumerate_lag_port(struct mlx5_core_dev *mdev, int ix)
+{
+ u16 port_aff_bias = mlx5_core_is_pf(mdev) ? 0 : MLX5_CAP_GEN(mdev, vhca_id);
+
+ return (ix + port_aff_bias) % mlx5e_get_num_lag_ports(mdev);
+}
+
static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
struct mlx5e_params *params,
struct mlx5e_channel_param *cparam,
@@ -1960,6 +1965,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
c->xdp = !!params->xdp_prog;
c->stats = &priv->channel_stats[ix].ch;
c->irq_desc = irq_to_desc(irq);
+ c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix);
err = mlx5e_alloc_xps_cpumask(c, params);
if (err)
@@ -3179,39 +3185,58 @@ void mlx5e_destroy_tis(struct mlx5_core_dev *mdev, u32 tisn)
mlx5_core_destroy_tis(mdev, tisn);
}
+void mlx5e_destroy_tises(struct mlx5e_priv *priv)
+{
+ int tc, i;
+
+ for (i = 0; i < mlx5e_get_num_lag_ports(priv->mdev); i++)
+ for (tc = 0; tc < priv->profile->max_tc; tc++)
+ mlx5e_destroy_tis(priv->mdev, priv->tisn[i][tc]);
+}
+
+static bool mlx5e_lag_should_assign_affinity(struct mlx5_core_dev *mdev)
+{
+ return MLX5_CAP_GEN(mdev, lag_tx_port_affinity) && mlx5e_get_num_lag_ports(mdev) > 1;
+}
+
int mlx5e_create_tises(struct mlx5e_priv *priv)
{
+ int tc, i;
int err;
- int tc;
- for (tc = 0; tc < priv->profile->max_tc; tc++) {
- u32 in[MLX5_ST_SZ_DW(create_tis_in)] = {};
- void *tisc;
+ for (i = 0; i < mlx5e_get_num_lag_ports(priv->mdev); i++) {
+ for (tc = 0; tc < priv->profile->max_tc; tc++) {
+ u32 in[MLX5_ST_SZ_DW(create_tis_in)] = {};
+ void *tisc;
- tisc = MLX5_ADDR_OF(create_tis_in, in, ctx);
+ tisc = MLX5_ADDR_OF(create_tis_in, in, ctx);
- MLX5_SET(tisc, tisc, prio, tc << 1);
+ MLX5_SET(tisc, tisc, prio, tc << 1);
- err = mlx5e_create_tis(priv->mdev, in, &priv->tisn[tc]);
- if (err)
- goto err_close_tises;
+ if (mlx5e_lag_should_assign_affinity(priv->mdev))
+ MLX5_SET(tisc, tisc, lag_tx_port_affinity, i + 1);
+
+ err = mlx5e_create_tis(priv->mdev, in, &priv->tisn[i][tc]);
+ if (err)
+ goto err_close_tises;
+ }
}
return 0;
err_close_tises:
- for (tc--; tc >= 0; tc--)
- mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+ for (; i >= 0; i--) {
+ for (tc--; tc >= 0; tc--)
+ mlx5e_destroy_tis(priv->mdev, priv->tisn[i][tc]);
+ tc = priv->profile->max_tc;
+ }
return err;
}
static void mlx5e_cleanup_nic_tx(struct mlx5e_priv *priv)
{
- int tc;
-
- for (tc = 0; tc < priv->profile->max_tc; tc++)
- mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+ mlx5e_destroy_tises(priv);
}
static void mlx5e_build_indir_tir_ctx_common(struct mlx5e_priv *priv,
@@ -3473,11 +3498,15 @@ static int mlx5e_setup_tc(struct net_device *dev, enum tc_setup_type type,
switch (type) {
#ifdef CONFIG_MLX5_ESWITCH
- case TC_SETUP_BLOCK:
+ case TC_SETUP_BLOCK: {
+ struct flow_block_offload *f = type_data;
+
+ f->unlocked_driver_cb = true;
return flow_block_cb_setup_simple(type_data,
&mlx5e_block_cb_list,
mlx5e_setup_tc_block_cb,
priv, priv, true);
+ }
#endif
case TC_SETUP_QDISC_MQPRIO:
return mlx5e_setup_tc_mqprio(priv, type_data);
@@ -4214,6 +4243,8 @@ static netdev_features_t mlx5e_tunnel_features_check(struct mlx5e_priv *priv,
switch (proto) {
case IPPROTO_GRE:
+ case IPPROTO_IPIP:
+ case IPPROTO_IPV6:
return features;
case IPPROTO_UDP:
udph = udp_hdr(skb);
@@ -4547,7 +4578,7 @@ const struct net_device_ops mlx5e_netdev_ops = {
.ndo_tx_timeout = mlx5e_tx_timeout,
.ndo_bpf = mlx5e_xdp,
.ndo_xdp_xmit = mlx5e_xdp_xmit,
- .ndo_xsk_async_xmit = mlx5e_xsk_async_xmit,
+ .ndo_xsk_wakeup = mlx5e_xsk_wakeup,
#ifdef CONFIG_MLX5_EN_ARFS
.ndo_rx_flow_steer = mlx5e_rx_flow_steer,
#endif
@@ -4850,7 +4881,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
netdev->hw_features |= NETIF_F_HW_VLAN_STAG_TX;
if (mlx5_vxlan_allowed(mdev->vxlan) || mlx5_geneve_tx_allowed(mdev) ||
- MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
+ mlx5e_any_tunnel_proto_supported(mdev)) {
netdev->hw_enc_features |= NETIF_F_HW_CSUM;
netdev->hw_enc_features |= NETIF_F_TSO;
netdev->hw_enc_features |= NETIF_F_TSO6;
@@ -4865,7 +4896,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
netdev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM;
}
- if (MLX5_CAP_ETH(mdev, tunnel_stateless_gre)) {
+ if (mlx5e_tunnel_proto_supported(mdev, IPPROTO_GRE)) {
netdev->hw_features |= NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM;
netdev->hw_enc_features |= NETIF_F_GSO_GRE |
@@ -4874,6 +4905,15 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
NETIF_F_GSO_GRE_CSUM;
}
+ if (mlx5e_tunnel_proto_supported(mdev, IPPROTO_IPIP)) {
+ netdev->hw_features |= NETIF_F_GSO_IPXIP4 |
+ NETIF_F_GSO_IPXIP6;
+ netdev->hw_enc_features |= NETIF_F_GSO_IPXIP4 |
+ NETIF_F_GSO_IPXIP6;
+ netdev->gso_partial_features |= NETIF_F_GSO_IPXIP4 |
+ NETIF_F_GSO_IPXIP6;
+ }
+
netdev->hw_features |= NETIF_F_GSO_PARTIAL;
netdev->gso_partial_features |= NETIF_F_GSO_UDP_L4;
netdev->hw_features |= NETIF_F_GSO_UDP_L4;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index 3c0d36b..95892a3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -38,6 +38,7 @@
#include <net/netevent.h>
#include <net/arp.h>
#include <net/devlink.h>
+#include <net/ipv6_stubs.h>
#include "eswitch.h"
#include "en.h"
@@ -499,16 +500,18 @@ void mlx5e_remove_sqs_fwd_rules(struct mlx5e_priv *priv)
mlx5e_sqs2vport_stop(esw, rep);
}
+static unsigned long mlx5e_rep_ipv6_interval(void)
+{
+ if (IS_ENABLED(CONFIG_IPV6) && ipv6_stub->nd_tbl)
+ return NEIGH_VAR(&ipv6_stub->nd_tbl->parms, DELAY_PROBE_TIME);
+
+ return ~0UL;
+}
+
static void mlx5e_rep_neigh_update_init_interval(struct mlx5e_rep_priv *rpriv)
{
-#if IS_ENABLED(CONFIG_IPV6)
- unsigned long ipv6_interval = NEIGH_VAR(&nd_tbl.parms,
- DELAY_PROBE_TIME);
-#else
- unsigned long ipv6_interval = ~0UL;
-#endif
- unsigned long ipv4_interval = NEIGH_VAR(&arp_tbl.parms,
- DELAY_PROBE_TIME);
+ unsigned long ipv4_interval = NEIGH_VAR(&arp_tbl.parms, DELAY_PROBE_TIME);
+ unsigned long ipv6_interval = mlx5e_rep_ipv6_interval();
struct net_device *netdev = rpriv->netdev;
struct mlx5e_priv *priv = netdev_priv(netdev);
@@ -763,6 +766,7 @@ mlx5e_rep_indr_setup_tc_block(struct net_device *netdev,
if (f->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS)
return -EOPNOTSUPP;
+ f->unlocked_driver_cb = true;
f->driver_block_list = &mlx5e_block_cb_list;
switch (f->command) {
@@ -916,7 +920,7 @@ static int mlx5e_rep_netevent_event(struct notifier_block *nb,
case NETEVENT_NEIGH_UPDATE:
n = ptr;
#if IS_ENABLED(CONFIG_IPV6)
- if (n->tbl != &nd_tbl && n->tbl != &arp_tbl)
+ if (n->tbl != ipv6_stub->nd_tbl && n->tbl != &arp_tbl)
#else
if (n->tbl != &arp_tbl)
#endif
@@ -943,7 +947,7 @@ static int mlx5e_rep_netevent_event(struct notifier_block *nb,
* done per device delay prob time parameter.
*/
#if IS_ENABLED(CONFIG_IPV6)
- if (!p->dev || (p->tbl != &nd_tbl && p->tbl != &arp_tbl))
+ if (!p->dev || (p->tbl != ipv6_stub->nd_tbl && p->tbl != &arp_tbl))
#else
if (!p->dev || p->tbl != &arp_tbl)
#endif
@@ -1245,9 +1249,11 @@ static int mlx5e_rep_setup_tc(struct net_device *dev, enum tc_setup_type type,
void *type_data)
{
struct mlx5e_priv *priv = netdev_priv(dev);
+ struct flow_block_offload *f = type_data;
switch (type) {
case TC_SETUP_BLOCK:
+ f->unlocked_driver_cb = true;
return flow_block_cb_setup_simple(type_data,
&mlx5e_rep_block_cb_list,
mlx5e_rep_setup_tc_cb,
@@ -1618,7 +1624,7 @@ static int mlx5e_init_rep_tx(struct mlx5e_priv *priv)
{
struct mlx5e_rep_priv *rpriv = priv->ppriv;
struct mlx5_rep_uplink_priv *uplink_priv;
- int tc, err;
+ int err;
err = mlx5e_create_tises(priv);
if (err) {
@@ -1654,18 +1660,15 @@ static int mlx5e_init_rep_tx(struct mlx5e_priv *priv)
tc_esw_cleanup:
mlx5e_tc_esw_cleanup(&uplink_priv->tc_ht);
destroy_tises:
- for (tc = 0; tc < priv->profile->max_tc; tc++)
- mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+ mlx5e_destroy_tises(priv);
return err;
}
static void mlx5e_cleanup_rep_tx(struct mlx5e_priv *priv)
{
struct mlx5e_rep_priv *rpriv = priv->ppriv;
- int tc;
- for (tc = 0; tc < priv->profile->max_tc; tc++)
- mlx5e_destroy_tis(priv->mdev, priv->tisn[tc]);
+ mlx5e_destroy_tises(priv);
if (rpriv->rep->vport == MLX5_VPORT_UPLINK) {
/* clean indirect TC block notifications */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h
index a0ae506..31f83c8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h
@@ -161,7 +161,7 @@ struct mlx5e_encap_entry {
*/
struct hlist_node encap_hlist;
struct list_head flows;
- u32 encap_id;
+ struct mlx5_pkt_reformat *pkt_reformat;
const struct ip_tunnel_info *tun_info;
unsigned char h_dest[ETH_ALEN]; /* destination eth addr */
@@ -183,7 +183,6 @@ struct mlx5e_rep_sq {
struct list_head list;
};
-void *mlx5e_alloc_nic_rep_priv(struct mlx5_core_dev *mdev);
void mlx5e_rep_register_vport_reps(struct mlx5_core_dev *mdev);
void mlx5e_rep_unregister_vport_reps(struct mlx5_core_dev *mdev);
bool mlx5e_is_uplink_rep(struct mlx5e_priv *priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 2fd2760..d6a5472 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -695,8 +695,11 @@ bool mlx5e_post_rx_mpwqes(struct mlx5e_rq *rq)
rq->mpwqe.umr_in_progress += rq->mpwqe.umr_last_bulk;
rq->mpwqe.actual_wq_head = head;
- /* If XSK Fill Ring doesn't have enough frames, busy poll by
- * rescheduling the NAPI poll.
+ /* If XSK Fill Ring doesn't have enough frames, report the error, so
+ * that one of the actions can be performed:
+ * 1. If need_wakeup is used, signal that the application has to kick
+ * the driver when it refills the Fill Ring.
+ * 2. Otherwise, busy poll by rescheduling the NAPI poll.
*/
if (unlikely(alloc_err == -ENOMEM && rq->umem))
return true;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 18e4c16..ac6fdcd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -369,17 +369,27 @@ static void mlx5e_grp_q_update_stats(struct mlx5e_priv *priv)
}
#define VNIC_ENV_OFF(c) MLX5_BYTE_OFF(query_vnic_env_out, c)
-static const struct counter_desc vnic_env_stats_desc[] = {
+static const struct counter_desc vnic_env_stats_steer_desc[] = {
{ "rx_steer_missed_packets",
VNIC_ENV_OFF(vport_env.nic_receive_steering_discard) },
};
-#define NUM_VNIC_ENV_COUNTERS ARRAY_SIZE(vnic_env_stats_desc)
+static const struct counter_desc vnic_env_stats_dev_oob_desc[] = {
+ { "dev_internal_queue_oob",
+ VNIC_ENV_OFF(vport_env.internal_rq_out_of_buffer) },
+};
+
+#define NUM_VNIC_ENV_STEER_COUNTERS(dev) \
+ (MLX5_CAP_GEN(dev, nic_receive_steering_discard) ? \
+ ARRAY_SIZE(vnic_env_stats_steer_desc) : 0)
+#define NUM_VNIC_ENV_DEV_OOB_COUNTERS(dev) \
+ (MLX5_CAP_GEN(dev, vnic_env_int_rq_oob) ? \
+ ARRAY_SIZE(vnic_env_stats_dev_oob_desc) : 0)
static int mlx5e_grp_vnic_env_get_num_stats(struct mlx5e_priv *priv)
{
- return MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard) ?
- NUM_VNIC_ENV_COUNTERS : 0;
+ return NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev) +
+ NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev);
}
static int mlx5e_grp_vnic_env_fill_strings(struct mlx5e_priv *priv, u8 *data,
@@ -387,12 +397,13 @@ static int mlx5e_grp_vnic_env_fill_strings(struct mlx5e_priv *priv, u8 *data,
{
int i;
- if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
- return idx;
-
- for (i = 0; i < NUM_VNIC_ENV_COUNTERS; i++)
+ for (i = 0; i < NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev); i++)
strcpy(data + (idx++) * ETH_GSTRING_LEN,
- vnic_env_stats_desc[i].format);
+ vnic_env_stats_steer_desc[i].format);
+
+ for (i = 0; i < NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); i++)
+ strcpy(data + (idx++) * ETH_GSTRING_LEN,
+ vnic_env_stats_dev_oob_desc[i].format);
return idx;
}
@@ -401,12 +412,13 @@ static int mlx5e_grp_vnic_env_fill_stats(struct mlx5e_priv *priv, u64 *data,
{
int i;
- if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
- return idx;
-
- for (i = 0; i < NUM_VNIC_ENV_COUNTERS; i++)
+ for (i = 0; i < NUM_VNIC_ENV_STEER_COUNTERS(priv->mdev); i++)
data[idx++] = MLX5E_READ_CTR64_BE(priv->stats.vnic.query_vnic_env_out,
- vnic_env_stats_desc, i);
+ vnic_env_stats_steer_desc, i);
+
+ for (i = 0; i < NUM_VNIC_ENV_DEV_OOB_COUNTERS(priv->mdev); i++)
+ data[idx++] = MLX5E_READ_CTR32_BE(priv->stats.vnic.query_vnic_env_out,
+ vnic_env_stats_dev_oob_desc, i);
return idx;
}
@@ -969,6 +981,147 @@ static void mlx5e_grp_pcie_update_stats(struct mlx5e_priv *priv)
mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_MPCNT, 0, 0);
}
+#define PPORT_PER_TC_PRIO_OFF(c) \
+ MLX5_BYTE_OFF(ppcnt_reg, \
+ counter_set.eth_per_tc_prio_grp_data_layout.c##_high)
+
+static const struct counter_desc pport_per_tc_prio_stats_desc[] = {
+ { "rx_prio%d_buf_discard", PPORT_PER_TC_PRIO_OFF(no_buffer_discard_uc) },
+};
+
+#define NUM_PPORT_PER_TC_PRIO_COUNTERS ARRAY_SIZE(pport_per_tc_prio_stats_desc)
+
+#define PPORT_PER_TC_CONGEST_PRIO_OFF(c) \
+ MLX5_BYTE_OFF(ppcnt_reg, \
+ counter_set.eth_per_tc_congest_prio_grp_data_layout.c##_high)
+
+static const struct counter_desc pport_per_tc_congest_prio_stats_desc[] = {
+ { "rx_prio%d_cong_discard", PPORT_PER_TC_CONGEST_PRIO_OFF(wred_discard) },
+ { "rx_prio%d_marked", PPORT_PER_TC_CONGEST_PRIO_OFF(ecn_marked_tc) },
+};
+
+#define NUM_PPORT_PER_TC_CONGEST_PRIO_COUNTERS \
+ ARRAY_SIZE(pport_per_tc_congest_prio_stats_desc)
+
+static int mlx5e_grp_per_tc_prio_get_num_stats(struct mlx5e_priv *priv)
+{
+ struct mlx5_core_dev *mdev = priv->mdev;
+
+ if (!MLX5_CAP_GEN(mdev, sbcam_reg))
+ return 0;
+
+ return NUM_PPORT_PER_TC_PRIO_COUNTERS * NUM_PPORT_PRIO;
+}
+
+static int mlx5e_grp_per_port_buffer_congest_fill_strings(struct mlx5e_priv *priv,
+ u8 *data, int idx)
+{
+ struct mlx5_core_dev *mdev = priv->mdev;
+ int i, prio;
+
+ if (!MLX5_CAP_GEN(mdev, sbcam_reg))
+ return idx;
+
+ for (prio = 0; prio < NUM_PPORT_PRIO; prio++) {
+ for (i = 0; i < NUM_PPORT_PER_TC_PRIO_COUNTERS; i++)
+ sprintf(data + (idx++) * ETH_GSTRING_LEN,
+ pport_per_tc_prio_stats_desc[i].format, prio);
+ for (i = 0; i < NUM_PPORT_PER_TC_CONGEST_PRIO_COUNTERS; i++)
+ sprintf(data + (idx++) * ETH_GSTRING_LEN,
+ pport_per_tc_congest_prio_stats_desc[i].format, prio);
+ }
+
+ return idx;
+}
+
+static int mlx5e_grp_per_port_buffer_congest_fill_stats(struct mlx5e_priv *priv,
+ u64 *data, int idx)
+{
+ struct mlx5e_pport_stats *pport = &priv->stats.pport;
+ struct mlx5_core_dev *mdev = priv->mdev;
+ int i, prio;
+
+ if (!MLX5_CAP_GEN(mdev, sbcam_reg))
+ return idx;
+
+ for (prio = 0; prio < NUM_PPORT_PRIO; prio++) {
+ for (i = 0; i < NUM_PPORT_PER_TC_PRIO_COUNTERS; i++)
+ data[idx++] =
+ MLX5E_READ_CTR64_BE(&pport->per_tc_prio_counters[prio],
+ pport_per_tc_prio_stats_desc, i);
+ for (i = 0; i < NUM_PPORT_PER_TC_CONGEST_PRIO_COUNTERS ; i++)
+ data[idx++] =
+ MLX5E_READ_CTR64_BE(&pport->per_tc_congest_prio_counters[prio],
+ pport_per_tc_congest_prio_stats_desc, i);
+ }
+
+ return idx;
+}
+
+static void mlx5e_grp_per_tc_prio_update_stats(struct mlx5e_priv *priv)
+{
+ struct mlx5e_pport_stats *pstats = &priv->stats.pport;
+ struct mlx5_core_dev *mdev = priv->mdev;
+ u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
+ int sz = MLX5_ST_SZ_BYTES(ppcnt_reg);
+ void *out;
+ int prio;
+
+ if (!MLX5_CAP_GEN(mdev, sbcam_reg))
+ return;
+
+ MLX5_SET(ppcnt_reg, in, pnat, 2);
+ MLX5_SET(ppcnt_reg, in, grp, MLX5_PER_TRAFFIC_CLASS_COUNTERS_GROUP);
+ for (prio = 0; prio < NUM_PPORT_PRIO; prio++) {
+ out = pstats->per_tc_prio_counters[prio];
+ MLX5_SET(ppcnt_reg, in, prio_tc, prio);
+ mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPCNT, 0, 0);
+ }
+}
+
+static int mlx5e_grp_per_tc_congest_prio_get_num_stats(struct mlx5e_priv *priv)
+{
+ struct mlx5_core_dev *mdev = priv->mdev;
+
+ if (!MLX5_CAP_GEN(mdev, sbcam_reg))
+ return 0;
+
+ return NUM_PPORT_PER_TC_CONGEST_PRIO_COUNTERS * NUM_PPORT_PRIO;
+}
+
+static void mlx5e_grp_per_tc_congest_prio_update_stats(struct mlx5e_priv *priv)
+{
+ struct mlx5e_pport_stats *pstats = &priv->stats.pport;
+ struct mlx5_core_dev *mdev = priv->mdev;
+ u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
+ int sz = MLX5_ST_SZ_BYTES(ppcnt_reg);
+ void *out;
+ int prio;
+
+ if (!MLX5_CAP_GEN(mdev, sbcam_reg))
+ return;
+
+ MLX5_SET(ppcnt_reg, in, pnat, 2);
+ MLX5_SET(ppcnt_reg, in, grp, MLX5_PER_TRAFFIC_CLASS_CONGESTION_GROUP);
+ for (prio = 0; prio < NUM_PPORT_PRIO; prio++) {
+ out = pstats->per_tc_congest_prio_counters[prio];
+ MLX5_SET(ppcnt_reg, in, prio_tc, prio);
+ mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPCNT, 0, 0);
+ }
+}
+
+static int mlx5e_grp_per_port_buffer_congest_get_num_stats(struct mlx5e_priv *priv)
+{
+ return mlx5e_grp_per_tc_prio_get_num_stats(priv) +
+ mlx5e_grp_per_tc_congest_prio_get_num_stats(priv);
+}
+
+static void mlx5e_grp_per_port_buffer_congest_update_stats(struct mlx5e_priv *priv)
+{
+ mlx5e_grp_per_tc_prio_update_stats(priv);
+ mlx5e_grp_per_tc_congest_prio_update_stats(priv);
+}
+
#define PPORT_PER_PRIO_OFF(c) \
MLX5_BYTE_OFF(ppcnt_reg, \
counter_set.eth_per_prio_grp_data_layout.c##_high)
@@ -1598,7 +1751,13 @@ const struct mlx5e_stats_grp mlx5e_stats_grps[] = {
.get_num_stats = mlx5e_grp_channels_get_num_stats,
.fill_strings = mlx5e_grp_channels_fill_strings,
.fill_stats = mlx5e_grp_channels_fill_stats,
- }
+ },
+ {
+ .get_num_stats = mlx5e_grp_per_port_buffer_congest_get_num_stats,
+ .fill_strings = mlx5e_grp_per_port_buffer_congest_fill_strings,
+ .fill_stats = mlx5e_grp_per_port_buffer_congest_fill_stats,
+ .update_stats = mlx5e_grp_per_port_buffer_congest_update_stats,
+ },
};
const int mlx5e_num_stats_grps = ARRAY_SIZE(mlx5e_stats_grps);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
index c281e56..79f261b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
@@ -207,6 +207,8 @@ struct mlx5e_pport_stats {
__be64 phy_counters[MLX5_ST_SZ_QW(ppcnt_reg)];
__be64 phy_statistical_counters[MLX5_ST_SZ_QW(ppcnt_reg)];
__be64 eth_ext_counters[MLX5_ST_SZ_QW(ppcnt_reg)];
+ __be64 per_tc_prio_counters[NUM_PPORT_PRIO][MLX5_ST_SZ_QW(ppcnt_reg)];
+ __be64 per_tc_congest_prio_counters[NUM_PPORT_PRIO][MLX5_ST_SZ_QW(ppcnt_reg)];
};
#define PCIE_PERF_GET(pcie_stats, c) \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index 5581a80..da7555f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -61,7 +61,7 @@
struct mlx5_nic_flow_attr {
u32 action;
u32 flow_tag;
- u32 mod_hdr_id;
+ struct mlx5_modify_hdr *modify_hdr;
u32 hairpin_tirn;
u8 match_level;
struct mlx5_flow_table *hairpin_ft;
@@ -201,7 +201,7 @@ struct mlx5e_mod_hdr_entry {
struct mod_hdr_key key;
- u32 mod_hdr_id;
+ struct mlx5_modify_hdr *modify_hdr;
refcount_t refcnt;
struct completion res_ready;
@@ -334,7 +334,7 @@ static void mlx5e_mod_hdr_put(struct mlx5e_priv *priv,
WARN_ON(!list_empty(&mh->flows));
if (mh->compl_result > 0)
- mlx5_modify_header_dealloc(priv->mdev, mh->mod_hdr_id);
+ mlx5_modify_header_dealloc(priv->mdev, mh->modify_hdr);
kfree(mh);
}
@@ -395,11 +395,11 @@ static int mlx5e_attach_mod_hdr(struct mlx5e_priv *priv,
hash_add(tbl->hlist, &mh->mod_hdr_hlist, hash_key);
mutex_unlock(&tbl->lock);
- err = mlx5_modify_header_alloc(priv->mdev, namespace,
- mh->key.num_actions,
- mh->key.actions,
- &mh->mod_hdr_id);
- if (err) {
+ mh->modify_hdr = mlx5_modify_header_alloc(priv->mdev, namespace,
+ mh->key.num_actions,
+ mh->key.actions);
+ if (IS_ERR(mh->modify_hdr)) {
+ err = PTR_ERR(mh->modify_hdr);
mh->compl_result = err;
goto alloc_header_err;
}
@@ -412,9 +412,9 @@ static int mlx5e_attach_mod_hdr(struct mlx5e_priv *priv,
list_add(&flow->mod_hdr, &mh->flows);
spin_unlock(&mh->flows_lock);
if (mlx5e_is_eswitch_flow(flow))
- flow->esw_attr->mod_hdr_id = mh->mod_hdr_id;
+ flow->esw_attr->modify_hdr = mh->modify_hdr;
else
- flow->nic_attr->mod_hdr_id = mh->mod_hdr_id;
+ flow->nic_attr->modify_hdr = mh->modify_hdr;
return 0;
@@ -906,7 +906,6 @@ mlx5e_tc_add_nic_flow(struct mlx5e_priv *priv,
struct mlx5_flow_destination dest[2] = {};
struct mlx5_flow_act flow_act = {
.action = attr->action,
- .reformat_id = 0,
.flags = FLOW_ACT_NO_APPEND,
};
struct mlx5_fc *counter = NULL;
@@ -947,7 +946,7 @@ mlx5e_tc_add_nic_flow(struct mlx5e_priv *priv,
if (attr->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR) {
err = mlx5e_attach_mod_hdr(priv, flow, parse_attr);
- flow_act.modify_id = attr->mod_hdr_id;
+ flow_act.modify_hdr = attr->modify_hdr;
kfree(parse_attr->mod_hdr_actions);
if (err)
return err;
@@ -989,10 +988,7 @@ mlx5e_tc_add_nic_flow(struct mlx5e_priv *priv,
&flow_act, dest, dest_ix);
mutex_unlock(&priv->fs.tc.t_lock);
- if (IS_ERR(flow->rule[0]))
- return PTR_ERR(flow->rule[0]);
-
- return 0;
+ return PTR_ERR_OR_ZERO(flow->rule[0]);
}
static void mlx5e_tc_del_nic_flow(struct mlx5e_priv *priv,
@@ -1304,14 +1300,13 @@ void mlx5e_tc_encap_flows_add(struct mlx5e_priv *priv,
struct mlx5e_tc_flow *flow;
int err;
- err = mlx5_packet_reformat_alloc(priv->mdev,
- e->reformat_type,
- e->encap_size, e->encap_header,
- MLX5_FLOW_NAMESPACE_FDB,
- &e->encap_id);
- if (err) {
- mlx5_core_warn(priv->mdev, "Failed to offload cached encapsulation header, %d\n",
- err);
+ e->pkt_reformat = mlx5_packet_reformat_alloc(priv->mdev,
+ e->reformat_type,
+ e->encap_size, e->encap_header,
+ MLX5_FLOW_NAMESPACE_FDB);
+ if (IS_ERR(e->pkt_reformat)) {
+ mlx5_core_warn(priv->mdev, "Failed to offload cached encapsulation header, %lu\n",
+ PTR_ERR(e->pkt_reformat));
return;
}
e->flags |= MLX5_ENCAP_ENTRY_VALID;
@@ -1326,7 +1321,7 @@ void mlx5e_tc_encap_flows_add(struct mlx5e_priv *priv,
esw_attr = flow->esw_attr;
spec = &esw_attr->parse_attr->spec;
- esw_attr->dests[flow->tmp_efi_index].encap_id = e->encap_id;
+ esw_attr->dests[flow->tmp_efi_index].pkt_reformat = e->pkt_reformat;
esw_attr->dests[flow->tmp_efi_index].flags |= MLX5_ESW_DEST_ENCAP_VALID;
/* Flow can be associated with multiple encap entries.
* Before offloading the flow verify that all of them have
@@ -1395,7 +1390,7 @@ void mlx5e_tc_encap_flows_del(struct mlx5e_priv *priv,
/* we know that the encap is valid */
e->flags &= ~MLX5_ENCAP_ENTRY_VALID;
- mlx5_packet_reformat_dealloc(priv->mdev, e->encap_id);
+ mlx5_packet_reformat_dealloc(priv->mdev, e->pkt_reformat);
}
static struct mlx5_fc *mlx5e_tc_get_counter(struct mlx5e_tc_flow *flow)
@@ -1494,7 +1489,7 @@ void mlx5e_tc_update_neigh_used_value(struct mlx5e_neigh_hash_entry *nhe)
tbl = &arp_tbl;
#if IS_ENABLED(CONFIG_IPV6)
else if (m_neigh->family == AF_INET6)
- tbl = &nd_tbl;
+ tbl = ipv6_stub->nd_tbl;
#endif
else
return;
@@ -1561,7 +1556,7 @@ static void mlx5e_encap_dealloc(struct mlx5e_priv *priv, struct mlx5e_encap_entr
mlx5e_rep_encap_entry_detach(netdev_priv(e->out_dev), e);
if (e->flags & MLX5_ENCAP_ENTRY_VALID)
- mlx5_packet_reformat_dealloc(priv->mdev, e->encap_id);
+ mlx5_packet_reformat_dealloc(priv->mdev, e->pkt_reformat);
}
kfree(e->encap_header);
@@ -1896,7 +1891,10 @@ static int __parse_cls_flower(struct mlx5e_priv *priv,
*match_level = MLX5_MATCH_L2;
}
} else if (*match_level != MLX5_MATCH_NONE) {
- MLX5_SET(fte_match_set_lyr_2_4, headers_c, svlan_tag, 1);
+ /* cvlan_tag enabled in match criteria and
+ * disabled in match value means both S & C tags
+ * don't exist (untagged of both)
+ */
MLX5_SET(fte_match_set_lyr_2_4, headers_c, cvlan_tag, 1);
*match_level = MLX5_MATCH_L2;
}
@@ -3045,7 +3043,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
flow->encaps[out_index].index = out_index;
*encap_dev = e->out_dev;
if (e->flags & MLX5_ENCAP_ENTRY_VALID) {
- attr->dests[out_index].encap_id = e->encap_id;
+ attr->dests[out_index].pkt_reformat = e->pkt_reformat;
attr->dests[out_index].flags |= MLX5_ESW_DEST_ENCAP_VALID;
*encap_valid = true;
} else {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index 49b06b2..257a7c9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -33,6 +33,7 @@
#include <linux/irq.h>
#include "en.h"
#include "en/xdp.h"
+#include "en/xsk/rx.h"
#include "en/xsk/tx.h"
static inline bool mlx5e_channel_no_affinity_change(struct mlx5e_channel *c)
@@ -81,6 +82,29 @@ void mlx5e_trigger_irq(struct mlx5e_icosq *sq)
mlx5e_notify_hw(wq, sq->pc, sq->uar_map, &nopwqe->ctrl);
}
+static bool mlx5e_napi_xsk_post(struct mlx5e_xdpsq *xsksq, struct mlx5e_rq *xskrq)
+{
+ bool busy_xsk = false, xsk_rx_alloc_err;
+
+ /* Handle the race between the application querying need_wakeup and the
+ * driver setting it:
+ * 1. Update need_wakeup both before and after the TX. If it goes to
+ * "yes", it can only happen with the first update.
+ * 2. If the application queried need_wakeup before we set it, the
+ * packets will be transmitted anyway, even w/o a wakeup.
+ * 3. Give a chance to clear need_wakeup after new packets were queued
+ * for TX.
+ */
+ mlx5e_xsk_update_tx_wakeup(xsksq);
+ busy_xsk |= mlx5e_xsk_tx(xsksq, MLX5E_TX_XSK_POLL_BUDGET);
+ mlx5e_xsk_update_tx_wakeup(xsksq);
+
+ xsk_rx_alloc_err = xskrq->post_wqes(xskrq);
+ busy_xsk |= mlx5e_xsk_update_rx_wakeup(xskrq, xsk_rx_alloc_err);
+
+ return busy_xsk;
+}
+
int mlx5e_napi_poll(struct napi_struct *napi, int budget)
{
struct mlx5e_channel *c = container_of(napi, struct mlx5e_channel,
@@ -122,8 +146,7 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
if (xsk_open) {
mlx5e_poll_ico_cq(&c->xskicosq.cq);
busy |= mlx5e_poll_xdpsq_cq(&xsksq->cq);
- busy_xsk |= mlx5e_xsk_tx(xsksq, MLX5E_TX_XSK_POLL_BUDGET);
- busy_xsk |= xskrq->post_wqes(xskrq);
+ busy_xsk |= mlx5e_napi_xsk_post(xsksq, xskrq);
}
busy |= busy_xsk;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
index 09d4c64..580c71c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
@@ -324,10 +324,13 @@ create_map_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
/**
* mlx5_eq_enable - Enable EQ for receiving EQEs
- * @dev - Device which owns the eq
- * @eq - EQ to enable
- * @nb - notifier call block
- * mlx5_eq_enable - must be called after EQ is created in device.
+ * @dev : Device which owns the eq
+ * @eq : EQ to enable
+ * @nb : Notifier call block
+ *
+ * Must be called after EQ is created in device.
+ *
+ * @return: 0 if no error
*/
int mlx5_eq_enable(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
struct notifier_block *nb)
@@ -344,11 +347,12 @@ int mlx5_eq_enable(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
EXPORT_SYMBOL(mlx5_eq_enable);
/**
- * mlx5_eq_disable - Enable EQ for receiving EQEs
- * @dev - Device which owns the eq
- * @eq - EQ to disable
- * @nb - notifier call block
- * mlx5_eq_disable - must be called before EQ is destroyed.
+ * mlx5_eq_disable - Disable EQ for receiving EQEs
+ * @dev : Device which owns the eq
+ * @eq : EQ to disable
+ * @nb : Notifier call block
+ *
+ * Must be called before EQ is destroyed.
*/
void mlx5_eq_disable(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
struct notifier_block *nb)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index f069240..30aae76 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1413,7 +1413,7 @@ static int esw_vport_egress_config(struct mlx5_eswitch *esw,
static bool element_type_supported(struct mlx5_eswitch *esw, int type)
{
- struct mlx5_core_dev *dev = esw->dev = esw->dev;
+ const struct mlx5_core_dev *dev = esw->dev;
switch (type) {
case SCHEDULING_CONTEXT_ELEMENT_TYPE_TSAR:
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index aba9e7a..6bd6f58 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -69,7 +69,7 @@ struct vport_ingress {
struct mlx5_flow_group *allow_spoofchk_only_grp;
struct mlx5_flow_group *allow_untagged_only_grp;
struct mlx5_flow_group *drop_grp;
- int modify_metadata_id;
+ struct mlx5_modify_hdr *modify_metadata;
struct mlx5_flow_handle *modify_metadata_rule;
struct mlx5_flow_handle *allow_rule;
struct mlx5_flow_handle *drop_rule;
@@ -153,6 +153,7 @@ struct mlx5_eswitch_fdb {
} legacy;
struct offloads_fdb {
+ struct mlx5_flow_namespace *ns;
struct mlx5_flow_table *slow_fdb;
struct mlx5_flow_group *send_to_vport_grp;
struct mlx5_flow_group *peer_miss_grp;
@@ -385,11 +386,11 @@ struct mlx5_esw_flow_attr {
struct {
u32 flags;
struct mlx5_eswitch_rep *rep;
+ struct mlx5_pkt_reformat *pkt_reformat;
struct mlx5_core_dev *mdev;
- u32 encap_id;
struct mlx5_termtbl_handle *termtbl;
} dests[MLX5_MAX_FLOW_FWD_VPORTS];
- u32 mod_hdr_id;
+ struct mlx5_modify_hdr *modify_hdr;
u8 inner_match_level;
u8 outer_match_level;
struct mlx5_fc *counter;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 7d3582e..00d71db 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -190,10 +190,10 @@ mlx5_eswitch_add_offloaded_rule(struct mlx5_eswitch *esw,
MLX5_FLOW_DEST_VPORT_VHCA_ID;
if (attr->dests[j].flags & MLX5_ESW_DEST_ENCAP) {
flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT;
- flow_act.reformat_id = attr->dests[j].encap_id;
+ flow_act.pkt_reformat = attr->dests[j].pkt_reformat;
dest[i].vport.flags |= MLX5_FLOW_DEST_VPORT_REFORMAT_ID;
- dest[i].vport.reformat_id =
- attr->dests[j].encap_id;
+ dest[i].vport.pkt_reformat =
+ attr->dests[j].pkt_reformat;
}
i++;
}
@@ -213,7 +213,7 @@ mlx5_eswitch_add_offloaded_rule(struct mlx5_eswitch *esw,
spec->match_criteria_enable |= MLX5_MATCH_INNER_HEADERS;
if (flow_act.action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)
- flow_act.modify_id = attr->mod_hdr_id;
+ flow_act.modify_hdr = attr->modify_hdr;
fdb = esw_get_prio_table(esw, attr->chain, attr->prio, !!split);
if (IS_ERR(fdb)) {
@@ -276,7 +276,7 @@ mlx5_eswitch_add_fwd_rule(struct mlx5_eswitch *esw,
dest[i].vport.flags |= MLX5_FLOW_DEST_VPORT_VHCA_ID;
if (attr->dests[i].flags & MLX5_ESW_DEST_ENCAP) {
dest[i].vport.flags |= MLX5_FLOW_DEST_VPORT_REFORMAT_ID;
- dest[i].vport.reformat_id = attr->dests[i].encap_id;
+ dest[i].vport.pkt_reformat = attr->dests[i].pkt_reformat;
}
}
dest[i].type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE;
@@ -1068,6 +1068,13 @@ static int esw_create_offloads_fdb_tables(struct mlx5_eswitch *esw, int nvports)
err = -EOPNOTSUPP;
goto ns_err;
}
+ esw->fdb_table.offloads.ns = root_ns;
+ err = mlx5_flow_namespace_set_mode(root_ns,
+ esw->dev->priv.steering->mode);
+ if (err) {
+ esw_warn(dev, "Failed to set FDB namespace steering mode\n");
+ goto ns_err;
+ }
max_flow_counter = (MLX5_CAP_GEN(dev, max_flow_counter_31_16) << 16) |
MLX5_CAP_GEN(dev, max_flow_counter_15_0);
@@ -1207,6 +1214,8 @@ static int esw_create_offloads_fdb_tables(struct mlx5_eswitch *esw, int nvports)
esw_destroy_offloads_fast_fdb_tables(esw);
mlx5_destroy_flow_table(esw->fdb_table.offloads.slow_fdb);
slow_fdb_err:
+ /* Holds true only as long as DMFS is the default */
+ mlx5_flow_namespace_set_mode(root_ns, MLX5_FLOW_STEERING_MODE_DMFS);
ns_err:
kvfree(flow_group_in);
return err;
@@ -1226,6 +1235,9 @@ static void esw_destroy_offloads_fdb_tables(struct mlx5_eswitch *esw)
mlx5_destroy_flow_table(esw->fdb_table.offloads.slow_fdb);
esw_destroy_offloads_fast_fdb_tables(esw);
+ /* Holds true only as long as DMFS is the default */
+ mlx5_flow_namespace_set_mode(esw->fdb_table.offloads.ns,
+ MLX5_FLOW_STEERING_MODE_DMFS);
}
static int esw_create_offloads_table(struct mlx5_eswitch *esw, int nvports)
@@ -1623,13 +1635,42 @@ static void mlx5_esw_offloads_unpair(struct mlx5_eswitch *esw)
esw_del_fdb_peer_miss_rules(esw);
}
+static int mlx5_esw_offloads_set_ns_peer(struct mlx5_eswitch *esw,
+ struct mlx5_eswitch *peer_esw,
+ bool pair)
+{
+ struct mlx5_flow_root_namespace *peer_ns;
+ struct mlx5_flow_root_namespace *ns;
+ int err;
+
+ peer_ns = peer_esw->dev->priv.steering->fdb_root_ns;
+ ns = esw->dev->priv.steering->fdb_root_ns;
+
+ if (pair) {
+ err = mlx5_flow_namespace_set_peer(ns, peer_ns);
+ if (err)
+ return err;
+
+ err = mlx5_flow_namespace_set_peer(peer_ns, ns);
+ if (err) {
+ mlx5_flow_namespace_set_peer(ns, NULL);
+ return err;
+ }
+ } else {
+ mlx5_flow_namespace_set_peer(ns, NULL);
+ mlx5_flow_namespace_set_peer(peer_ns, NULL);
+ }
+
+ return 0;
+}
+
static int mlx5_esw_offloads_devcom_event(int event,
void *my_data,
void *event_data)
{
struct mlx5_eswitch *esw = my_data;
- struct mlx5_eswitch *peer_esw = event_data;
struct mlx5_devcom *devcom = esw->dev->priv.devcom;
+ struct mlx5_eswitch *peer_esw = event_data;
int err;
switch (event) {
@@ -1638,9 +1679,12 @@ static int mlx5_esw_offloads_devcom_event(int event,
mlx5_eswitch_vport_match_metadata_enabled(peer_esw))
break;
- err = mlx5_esw_offloads_pair(esw, peer_esw);
+ err = mlx5_esw_offloads_set_ns_peer(esw, peer_esw, true);
if (err)
goto err_out;
+ err = mlx5_esw_offloads_pair(esw, peer_esw);
+ if (err)
+ goto err_peer;
err = mlx5_esw_offloads_pair(peer_esw, esw);
if (err)
@@ -1656,6 +1700,7 @@ static int mlx5_esw_offloads_devcom_event(int event,
mlx5_devcom_set_paired(devcom, MLX5_DEVCOM_ESW_OFFLOADS, false);
mlx5_esw_offloads_unpair(peer_esw);
mlx5_esw_offloads_unpair(esw);
+ mlx5_esw_offloads_set_ns_peer(esw, peer_esw, false);
break;
}
@@ -1663,7 +1708,8 @@ static int mlx5_esw_offloads_devcom_event(int event,
err_pair:
mlx5_esw_offloads_unpair(esw);
-
+err_peer:
+ mlx5_esw_offloads_set_ns_peer(esw, peer_esw, false);
err_out:
mlx5_core_err(esw->dev, "esw offloads devcom event failure, event %u err %d",
event, err);
@@ -1734,7 +1780,7 @@ static int esw_vport_ingress_prio_tag_config(struct mlx5_eswitch *esw,
if (vport->ingress.modify_metadata_rule) {
flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
- flow_act.modify_id = vport->ingress.modify_metadata_id;
+ flow_act.modify_hdr = vport->ingress.modify_metadata;
}
vport->ingress.allow_rule =
@@ -1770,9 +1816,11 @@ static int esw_vport_add_ingress_acl_modify_metadata(struct mlx5_eswitch *esw,
MLX5_SET(set_action_in, action, data,
mlx5_eswitch_get_vport_metadata_for_match(esw, vport->vport));
- err = mlx5_modify_header_alloc(esw->dev, MLX5_FLOW_NAMESPACE_ESW_INGRESS,
- 1, action, &vport->ingress.modify_metadata_id);
- if (err) {
+ vport->ingress.modify_metadata =
+ mlx5_modify_header_alloc(esw->dev, MLX5_FLOW_NAMESPACE_ESW_INGRESS,
+ 1, action);
+ if (IS_ERR(vport->ingress.modify_metadata)) {
+ err = PTR_ERR(vport->ingress.modify_metadata);
esw_warn(esw->dev,
"failed to alloc modify header for vport %d ingress acl (%d)\n",
vport->vport, err);
@@ -1780,7 +1828,7 @@ static int esw_vport_add_ingress_acl_modify_metadata(struct mlx5_eswitch *esw,
}
flow_act.action = MLX5_FLOW_CONTEXT_ACTION_MOD_HDR | MLX5_FLOW_CONTEXT_ACTION_ALLOW;
- flow_act.modify_id = vport->ingress.modify_metadata_id;
+ flow_act.modify_hdr = vport->ingress.modify_metadata;
vport->ingress.modify_metadata_rule = mlx5_add_flow_rules(vport->ingress.acl,
&spec, &flow_act, NULL, 0);
if (IS_ERR(vport->ingress.modify_metadata_rule)) {
@@ -1794,7 +1842,7 @@ static int esw_vport_add_ingress_acl_modify_metadata(struct mlx5_eswitch *esw,
out:
if (err)
- mlx5_modify_header_dealloc(esw->dev, vport->ingress.modify_metadata_id);
+ mlx5_modify_header_dealloc(esw->dev, vport->ingress.modify_metadata);
return err;
}
@@ -1803,7 +1851,7 @@ void esw_vport_del_ingress_acl_modify_metadata(struct mlx5_eswitch *esw,
{
if (vport->ingress.modify_metadata_rule) {
mlx5_del_flow_rules(vport->ingress.modify_metadata_rule);
- mlx5_modify_header_dealloc(esw->dev, vport->ingress.modify_metadata_id);
+ mlx5_modify_header_dealloc(esw->dev, vport->ingress.modify_metadata);
vport->ingress.modify_metadata_rule = NULL;
}
@@ -2113,9 +2161,10 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
else
esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_NONE;
+ mlx5_rdma_enable_roce(esw->dev);
err = esw_offloads_steering_init(esw);
if (err)
- return err;
+ goto err_steering_init;
err = esw_set_passing_vport_metadata(esw, true);
if (err)
@@ -2130,8 +2179,6 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
esw_offloads_devcom_init(esw);
mutex_init(&esw->offloads.termtbl_mutex);
- mlx5_rdma_enable_roce(esw->dev);
-
return 0;
err_reps:
@@ -2139,6 +2186,8 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
esw_set_passing_vport_metadata(esw, false);
err_vport_metadata:
esw_offloads_steering_cleanup(esw);
+err_steering_init:
+ mlx5_rdma_disable_roce(esw->dev);
return err;
}
@@ -2163,12 +2212,12 @@ static int esw_offloads_stop(struct mlx5_eswitch *esw,
void esw_offloads_disable(struct mlx5_eswitch *esw)
{
- mlx5_rdma_disable_roce(esw->dev);
esw_offloads_devcom_cleanup(esw);
esw_offloads_unload_all_reps(esw);
mlx5_eswitch_disable_pf_vf_vports(esw);
esw_set_passing_vport_metadata(esw, false);
esw_offloads_steering_cleanup(esw);
+ mlx5_rdma_disable_roce(esw->dev);
esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_NONE;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
index b84a225..579c306 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
@@ -107,6 +107,50 @@ static int mlx5_cmd_stub_delete_fte(struct mlx5_flow_root_namespace *ns,
return 0;
}
+static int mlx5_cmd_stub_packet_reformat_alloc(struct mlx5_flow_root_namespace *ns,
+ int reformat_type,
+ size_t size,
+ void *reformat_data,
+ enum mlx5_flow_namespace_type namespace,
+ struct mlx5_pkt_reformat *pkt_reformat)
+{
+ return 0;
+}
+
+static void mlx5_cmd_stub_packet_reformat_dealloc(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_pkt_reformat *pkt_reformat)
+{
+}
+
+static int mlx5_cmd_stub_modify_header_alloc(struct mlx5_flow_root_namespace *ns,
+ u8 namespace, u8 num_actions,
+ void *modify_actions,
+ struct mlx5_modify_hdr *modify_hdr)
+{
+ return 0;
+}
+
+static void mlx5_cmd_stub_modify_header_dealloc(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_modify_hdr *modify_hdr)
+{
+}
+
+static int mlx5_cmd_stub_set_peer(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_root_namespace *peer_ns)
+{
+ return 0;
+}
+
+static int mlx5_cmd_stub_create_ns(struct mlx5_flow_root_namespace *ns)
+{
+ return 0;
+}
+
+static int mlx5_cmd_stub_destroy_ns(struct mlx5_flow_root_namespace *ns)
+{
+ return 0;
+}
+
static int mlx5_cmd_update_root_ft(struct mlx5_flow_root_namespace *ns,
struct mlx5_flow_table *ft, u32 underlay_qpn,
bool disconnect)
@@ -182,7 +226,7 @@ static int mlx5_cmd_create_flow_table(struct mlx5_flow_root_namespace *ns,
} else {
MLX5_SET(create_flow_table_in, in,
flow_table_context.table_miss_action,
- ns->def_miss_action);
+ ft->def_miss_action);
}
break;
@@ -262,7 +306,7 @@ static int mlx5_cmd_modify_flow_table(struct mlx5_flow_root_namespace *ns,
} else {
MLX5_SET(modify_flow_table_in, in,
flow_table_context.table_miss_action,
- ns->def_miss_action);
+ ft->def_miss_action);
}
}
@@ -412,11 +456,13 @@ static int mlx5_cmd_set_fte(struct mlx5_core_dev *dev,
} else {
MLX5_SET(flow_context, in_flow_context, action,
fte->action.action);
- MLX5_SET(flow_context, in_flow_context, packet_reformat_id,
- fte->action.reformat_id);
+ if (fte->action.pkt_reformat)
+ MLX5_SET(flow_context, in_flow_context, packet_reformat_id,
+ fte->action.pkt_reformat->id);
}
- MLX5_SET(flow_context, in_flow_context, modify_header_id,
- fte->action.modify_id);
+ if (fte->action.modify_hdr)
+ MLX5_SET(flow_context, in_flow_context, modify_header_id,
+ fte->action.modify_hdr->id);
vlan = MLX5_ADDR_OF(flow_context, in_flow_context, push_vlan);
@@ -468,7 +514,7 @@ static int mlx5_cmd_set_fte(struct mlx5_core_dev *dev,
MLX5_FLOW_DEST_VPORT_REFORMAT_ID));
MLX5_SET(extended_dest_format, in_dests,
packet_reformat_id,
- dst->dest_attr.vport.reformat_id);
+ dst->dest_attr.vport.pkt_reformat->id);
}
break;
default:
@@ -643,14 +689,15 @@ int mlx5_cmd_fc_bulk_query(struct mlx5_core_dev *dev, u32 base_id, int bulk_len,
return mlx5_cmd_exec(dev, in, sizeof(in), out, outlen);
}
-int mlx5_packet_reformat_alloc(struct mlx5_core_dev *dev,
- int reformat_type,
- size_t size,
- void *reformat_data,
- enum mlx5_flow_namespace_type namespace,
- u32 *packet_reformat_id)
+static int mlx5_cmd_packet_reformat_alloc(struct mlx5_flow_root_namespace *ns,
+ int reformat_type,
+ size_t size,
+ void *reformat_data,
+ enum mlx5_flow_namespace_type namespace,
+ struct mlx5_pkt_reformat *pkt_reformat)
{
u32 out[MLX5_ST_SZ_DW(alloc_packet_reformat_context_out)];
+ struct mlx5_core_dev *dev = ns->dev;
void *packet_reformat_context_in;
int max_encap_size;
void *reformat;
@@ -693,35 +740,36 @@ int mlx5_packet_reformat_alloc(struct mlx5_core_dev *dev,
memset(out, 0, sizeof(out));
err = mlx5_cmd_exec(dev, in, inlen, out, sizeof(out));
- *packet_reformat_id = MLX5_GET(alloc_packet_reformat_context_out,
- out, packet_reformat_id);
+ pkt_reformat->id = MLX5_GET(alloc_packet_reformat_context_out,
+ out, packet_reformat_id);
kfree(in);
return err;
}
-EXPORT_SYMBOL(mlx5_packet_reformat_alloc);
-void mlx5_packet_reformat_dealloc(struct mlx5_core_dev *dev,
- u32 packet_reformat_id)
+static void mlx5_cmd_packet_reformat_dealloc(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_pkt_reformat *pkt_reformat)
{
u32 in[MLX5_ST_SZ_DW(dealloc_packet_reformat_context_in)];
u32 out[MLX5_ST_SZ_DW(dealloc_packet_reformat_context_out)];
+ struct mlx5_core_dev *dev = ns->dev;
memset(in, 0, sizeof(in));
MLX5_SET(dealloc_packet_reformat_context_in, in, opcode,
MLX5_CMD_OP_DEALLOC_PACKET_REFORMAT_CONTEXT);
MLX5_SET(dealloc_packet_reformat_context_in, in, packet_reformat_id,
- packet_reformat_id);
+ pkt_reformat->id);
mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
}
-EXPORT_SYMBOL(mlx5_packet_reformat_dealloc);
-int mlx5_modify_header_alloc(struct mlx5_core_dev *dev,
- u8 namespace, u8 num_actions,
- void *modify_actions, u32 *modify_header_id)
+static int mlx5_cmd_modify_header_alloc(struct mlx5_flow_root_namespace *ns,
+ u8 namespace, u8 num_actions,
+ void *modify_actions,
+ struct mlx5_modify_hdr *modify_hdr)
{
u32 out[MLX5_ST_SZ_DW(alloc_modify_header_context_out)];
int max_actions, actions_size, inlen, err;
+ struct mlx5_core_dev *dev = ns->dev;
void *actions_in;
u8 table_type;
u32 *in;
@@ -772,26 +820,26 @@ int mlx5_modify_header_alloc(struct mlx5_core_dev *dev,
memset(out, 0, sizeof(out));
err = mlx5_cmd_exec(dev, in, inlen, out, sizeof(out));
- *modify_header_id = MLX5_GET(alloc_modify_header_context_out, out, modify_header_id);
+ modify_hdr->id = MLX5_GET(alloc_modify_header_context_out, out, modify_header_id);
kfree(in);
return err;
}
-EXPORT_SYMBOL(mlx5_modify_header_alloc);
-void mlx5_modify_header_dealloc(struct mlx5_core_dev *dev, u32 modify_header_id)
+static void mlx5_cmd_modify_header_dealloc(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_modify_hdr *modify_hdr)
{
u32 in[MLX5_ST_SZ_DW(dealloc_modify_header_context_in)];
u32 out[MLX5_ST_SZ_DW(dealloc_modify_header_context_out)];
+ struct mlx5_core_dev *dev = ns->dev;
memset(in, 0, sizeof(in));
MLX5_SET(dealloc_modify_header_context_in, in, opcode,
MLX5_CMD_OP_DEALLOC_MODIFY_HEADER_CONTEXT);
MLX5_SET(dealloc_modify_header_context_in, in, modify_header_id,
- modify_header_id);
+ modify_hdr->id);
mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
}
-EXPORT_SYMBOL(mlx5_modify_header_dealloc);
static const struct mlx5_flow_cmds mlx5_flow_cmds = {
.create_flow_table = mlx5_cmd_create_flow_table,
@@ -803,6 +851,13 @@ static const struct mlx5_flow_cmds mlx5_flow_cmds = {
.update_fte = mlx5_cmd_update_fte,
.delete_fte = mlx5_cmd_delete_fte,
.update_root_ft = mlx5_cmd_update_root_ft,
+ .packet_reformat_alloc = mlx5_cmd_packet_reformat_alloc,
+ .packet_reformat_dealloc = mlx5_cmd_packet_reformat_dealloc,
+ .modify_header_alloc = mlx5_cmd_modify_header_alloc,
+ .modify_header_dealloc = mlx5_cmd_modify_header_dealloc,
+ .set_peer = mlx5_cmd_stub_set_peer,
+ .create_ns = mlx5_cmd_stub_create_ns,
+ .destroy_ns = mlx5_cmd_stub_destroy_ns,
};
static const struct mlx5_flow_cmds mlx5_flow_cmd_stubs = {
@@ -815,9 +870,16 @@ static const struct mlx5_flow_cmds mlx5_flow_cmd_stubs = {
.update_fte = mlx5_cmd_stub_update_fte,
.delete_fte = mlx5_cmd_stub_delete_fte,
.update_root_ft = mlx5_cmd_stub_update_root_ft,
+ .packet_reformat_alloc = mlx5_cmd_stub_packet_reformat_alloc,
+ .packet_reformat_dealloc = mlx5_cmd_stub_packet_reformat_dealloc,
+ .modify_header_alloc = mlx5_cmd_stub_modify_header_alloc,
+ .modify_header_dealloc = mlx5_cmd_stub_modify_header_dealloc,
+ .set_peer = mlx5_cmd_stub_set_peer,
+ .create_ns = mlx5_cmd_stub_create_ns,
+ .destroy_ns = mlx5_cmd_stub_destroy_ns,
};
-static const struct mlx5_flow_cmds *mlx5_fs_cmd_get_fw_cmds(void)
+const struct mlx5_flow_cmds *mlx5_fs_cmd_get_fw_cmds(void)
{
return &mlx5_flow_cmds;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.h b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.h
index bc46063..d62de64 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.h
@@ -75,6 +75,30 @@ struct mlx5_flow_cmds {
struct mlx5_flow_table *ft,
u32 underlay_qpn,
bool disconnect);
+
+ int (*packet_reformat_alloc)(struct mlx5_flow_root_namespace *ns,
+ int reformat_type,
+ size_t size,
+ void *reformat_data,
+ enum mlx5_flow_namespace_type namespace,
+ struct mlx5_pkt_reformat *pkt_reformat);
+
+ void (*packet_reformat_dealloc)(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_pkt_reformat *pkt_reformat);
+
+ int (*modify_header_alloc)(struct mlx5_flow_root_namespace *ns,
+ u8 namespace, u8 num_actions,
+ void *modify_actions,
+ struct mlx5_modify_hdr *modify_hdr);
+
+ void (*modify_header_dealloc)(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_modify_hdr *modify_hdr);
+
+ int (*set_peer)(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_root_namespace *peer_ns);
+
+ int (*create_ns)(struct mlx5_flow_root_namespace *ns);
+ int (*destroy_ns)(struct mlx5_flow_root_namespace *ns);
};
int mlx5_cmd_fc_alloc(struct mlx5_core_dev *dev, u32 *id);
@@ -90,5 +114,6 @@ int mlx5_cmd_fc_bulk_query(struct mlx5_core_dev *dev, u32 base_id, int bulk_len,
u32 *out);
const struct mlx5_flow_cmds *mlx5_fs_cmd_get_default(enum fs_flow_table_type type);
+const struct mlx5_flow_cmds *mlx5_fs_cmd_get_fw_cmds(void);
#endif
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
index 3e99799..3bbb493 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
@@ -60,7 +60,8 @@
ADD_PRIO(num_prios_val, 0, num_levels_val, {},\
__VA_ARGS__)\
-#define ADD_NS(...) {.type = FS_TYPE_NAMESPACE,\
+#define ADD_NS(def_miss_act, ...) {.type = FS_TYPE_NAMESPACE, \
+ .def_miss_action = def_miss_act,\
.children = (struct init_tree_node[]) {__VA_ARGS__},\
.ar_size = INIT_TREE_NODE_ARRAY_SIZE(__VA_ARGS__) \
}
@@ -131,33 +132,41 @@ static struct init_tree_node {
int num_leaf_prios;
int prio;
int num_levels;
+ enum mlx5_flow_table_miss_action def_miss_action;
} root_fs = {
.type = FS_TYPE_NAMESPACE,
.ar_size = 7,
- .children = (struct init_tree_node[]) {
- ADD_PRIO(0, BY_PASS_MIN_LEVEL, 0,
- FS_CHAINING_CAPS,
- ADD_NS(ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS,
- BY_PASS_PRIO_NUM_LEVELS))),
- ADD_PRIO(0, LAG_MIN_LEVEL, 0,
- FS_CHAINING_CAPS,
- ADD_NS(ADD_MULTIPLE_PRIO(LAG_NUM_PRIOS,
- LAG_PRIO_NUM_LEVELS))),
- ADD_PRIO(0, OFFLOADS_MIN_LEVEL, 0, {},
- ADD_NS(ADD_MULTIPLE_PRIO(OFFLOADS_NUM_PRIOS, OFFLOADS_MAX_FT))),
- ADD_PRIO(0, ETHTOOL_MIN_LEVEL, 0,
- FS_CHAINING_CAPS,
- ADD_NS(ADD_MULTIPLE_PRIO(ETHTOOL_NUM_PRIOS,
- ETHTOOL_PRIO_NUM_LEVELS))),
- ADD_PRIO(0, KERNEL_MIN_LEVEL, 0, {},
- ADD_NS(ADD_MULTIPLE_PRIO(KERNEL_NIC_TC_NUM_PRIOS, KERNEL_NIC_TC_NUM_LEVELS),
- ADD_MULTIPLE_PRIO(KERNEL_NIC_NUM_PRIOS,
- KERNEL_NIC_PRIO_NUM_LEVELS))),
- ADD_PRIO(0, BY_PASS_MIN_LEVEL, 0,
- FS_CHAINING_CAPS,
- ADD_NS(ADD_MULTIPLE_PRIO(LEFTOVERS_NUM_PRIOS, LEFTOVERS_NUM_LEVELS))),
- ADD_PRIO(0, ANCHOR_MIN_LEVEL, 0, {},
- ADD_NS(ADD_MULTIPLE_PRIO(ANCHOR_NUM_PRIOS, ANCHOR_NUM_LEVELS))),
+ .children = (struct init_tree_node[]){
+ ADD_PRIO(0, BY_PASS_MIN_LEVEL, 0, FS_CHAINING_CAPS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS,
+ BY_PASS_PRIO_NUM_LEVELS))),
+ ADD_PRIO(0, LAG_MIN_LEVEL, 0, FS_CHAINING_CAPS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(LAG_NUM_PRIOS,
+ LAG_PRIO_NUM_LEVELS))),
+ ADD_PRIO(0, OFFLOADS_MIN_LEVEL, 0, {},
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(OFFLOADS_NUM_PRIOS,
+ OFFLOADS_MAX_FT))),
+ ADD_PRIO(0, ETHTOOL_MIN_LEVEL, 0, FS_CHAINING_CAPS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(ETHTOOL_NUM_PRIOS,
+ ETHTOOL_PRIO_NUM_LEVELS))),
+ ADD_PRIO(0, KERNEL_MIN_LEVEL, 0, {},
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(KERNEL_NIC_TC_NUM_PRIOS,
+ KERNEL_NIC_TC_NUM_LEVELS),
+ ADD_MULTIPLE_PRIO(KERNEL_NIC_NUM_PRIOS,
+ KERNEL_NIC_PRIO_NUM_LEVELS))),
+ ADD_PRIO(0, BY_PASS_MIN_LEVEL, 0, FS_CHAINING_CAPS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(LEFTOVERS_NUM_PRIOS,
+ LEFTOVERS_NUM_LEVELS))),
+ ADD_PRIO(0, ANCHOR_MIN_LEVEL, 0, {},
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(ANCHOR_NUM_PRIOS,
+ ANCHOR_NUM_LEVELS))),
}
};
@@ -167,11 +176,32 @@ static struct init_tree_node egress_root_fs = {
.children = (struct init_tree_node[]) {
ADD_PRIO(0, MLX5_BY_PASS_NUM_PRIOS, 0,
FS_CHAINING_CAPS_EGRESS,
- ADD_NS(ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS,
BY_PASS_PRIO_NUM_LEVELS))),
}
};
+#define RDMA_RX_BYPASS_PRIO 0
+#define RDMA_RX_KERNEL_PRIO 1
+static struct init_tree_node rdma_rx_root_fs = {
+ .type = FS_TYPE_NAMESPACE,
+ .ar_size = 2,
+ .children = (struct init_tree_node[]) {
+ [RDMA_RX_BYPASS_PRIO] =
+ ADD_PRIO(0, MLX5_BY_PASS_NUM_REGULAR_PRIOS, 0,
+ FS_CHAINING_CAPS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF,
+ ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_REGULAR_PRIOS,
+ BY_PASS_PRIO_NUM_LEVELS))),
+ [RDMA_RX_KERNEL_PRIO] =
+ ADD_PRIO(0, MLX5_BY_PASS_NUM_REGULAR_PRIOS + 1, 0,
+ FS_CHAINING_CAPS,
+ ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_SWITCH_DOMAIN,
+ ADD_MULTIPLE_PRIO(1, 1))),
+ }
+};
+
enum fs_i_lock_class {
FS_LOCK_GRANDPARENT,
FS_LOCK_PARENT,
@@ -1014,6 +1044,7 @@ static struct mlx5_flow_table *__mlx5_create_flow_table(struct mlx5_flow_namespa
tree_init_node(&ft->node, del_hw_flow_table, del_sw_flow_table);
log_table_sz = ft->max_fte ? ilog2(ft->max_fte) : 0;
next_ft = find_next_chained_ft(fs_prio);
+ ft->def_miss_action = ns->def_miss_action;
err = root->cmds->create_flow_table(root, ft, log_table_sz, next_ft);
if (err)
goto free_ft;
@@ -1384,7 +1415,8 @@ static bool mlx5_flow_dests_cmp(struct mlx5_flow_destination *d1,
((d1->vport.flags & MLX5_FLOW_DEST_VPORT_VHCA_ID) ?
(d1->vport.vhca_id == d2->vport.vhca_id) : true) &&
((d1->vport.flags & MLX5_FLOW_DEST_VPORT_REFORMAT_ID) ?
- (d1->vport.reformat_id == d2->vport.reformat_id) : true)) ||
+ (d1->vport.pkt_reformat->id ==
+ d2->vport.pkt_reformat->id) : true)) ||
(d1->type == MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE &&
d1->ft == d2->ft) ||
(d1->type == MLX5_FLOW_DESTINATION_TYPE_TIR &&
@@ -2056,16 +2088,18 @@ struct mlx5_flow_namespace *mlx5_get_flow_namespace(struct mlx5_core_dev *dev,
if (steering->sniffer_tx_root_ns)
return &steering->sniffer_tx_root_ns->ns;
return NULL;
- case MLX5_FLOW_NAMESPACE_RDMA_RX:
- if (steering->rdma_rx_root_ns)
- return &steering->rdma_rx_root_ns->ns;
- return NULL;
default:
break;
}
if (type == MLX5_FLOW_NAMESPACE_EGRESS) {
root_ns = steering->egress_root_ns;
+ } else if (type == MLX5_FLOW_NAMESPACE_RDMA_RX) {
+ root_ns = steering->rdma_rx_root_ns;
+ prio = RDMA_RX_BYPASS_PRIO;
+ } else if (type == MLX5_FLOW_NAMESPACE_RDMA_RX_KERNEL) {
+ root_ns = steering->rdma_rx_root_ns;
+ prio = RDMA_RX_KERNEL_PRIO;
} else { /* Must be NIC RX */
root_ns = steering->root_ns;
prio = type;
@@ -2155,7 +2189,8 @@ static struct mlx5_flow_namespace *fs_init_namespace(struct mlx5_flow_namespace
return ns;
}
-static struct mlx5_flow_namespace *fs_create_namespace(struct fs_prio *prio)
+static struct mlx5_flow_namespace *fs_create_namespace(struct fs_prio *prio,
+ int def_miss_act)
{
struct mlx5_flow_namespace *ns;
@@ -2164,6 +2199,7 @@ static struct mlx5_flow_namespace *fs_create_namespace(struct fs_prio *prio)
return ERR_PTR(-ENOMEM);
fs_init_namespace(ns);
+ ns->def_miss_action = def_miss_act;
tree_init_node(&ns->node, NULL, del_sw_ns);
tree_add_node(&ns->node, &prio->node);
list_add_tail(&ns->node.list, &prio->node.children);
@@ -2230,7 +2266,7 @@ static int init_root_tree_recursive(struct mlx5_flow_steering *steering,
base = &fs_prio->node;
} else if (init_node->type == FS_TYPE_NAMESPACE) {
fs_get_obj(fs_prio, fs_parent_node);
- fs_ns = fs_create_namespace(fs_prio);
+ fs_ns = fs_create_namespace(fs_prio, init_node->def_miss_action);
if (IS_ERR(fs_ns))
return PTR_ERR(fs_ns);
base = &fs_ns->node;
@@ -2494,18 +2530,25 @@ static int init_sniffer_rx_root_ns(struct mlx5_flow_steering *steering)
static int init_rdma_rx_root_ns(struct mlx5_flow_steering *steering)
{
- struct fs_prio *prio;
+ int err;
steering->rdma_rx_root_ns = create_root_ns(steering, FS_FT_RDMA_RX);
if (!steering->rdma_rx_root_ns)
return -ENOMEM;
- steering->rdma_rx_root_ns->def_miss_action =
- MLX5_FLOW_TABLE_MISS_ACTION_SWITCH_DOMAIN;
+ err = init_root_tree(steering, &rdma_rx_root_fs,
+ &steering->rdma_rx_root_ns->ns.node);
+ if (err)
+ goto out_err;
- /* Create single prio */
- prio = fs_create_prio(&steering->rdma_rx_root_ns->ns, 0, 1);
- return PTR_ERR_OR_ZERO(prio);
+ set_prio_attrs(steering->rdma_rx_root_ns);
+
+ return 0;
+
+out_err:
+ cleanup_root_ns(steering->rdma_rx_root_ns);
+ steering->rdma_rx_root_ns = NULL;
+ return err;
}
static int init_fdb_root_ns(struct mlx5_flow_steering *steering)
{
@@ -2543,7 +2586,7 @@ static int init_fdb_root_ns(struct mlx5_flow_steering *steering)
}
for (chain = 0; chain <= FDB_MAX_CHAIN; chain++) {
- ns = fs_create_namespace(maj_prio);
+ ns = fs_create_namespace(maj_prio, MLX5_FLOW_TABLE_MISS_ACTION_DEF);
if (IS_ERR(ns)) {
err = PTR_ERR(ns);
goto out_err;
@@ -2846,3 +2889,160 @@ int mlx5_fs_remove_rx_underlay_qpn(struct mlx5_core_dev *dev, u32 underlay_qpn)
return err;
}
EXPORT_SYMBOL(mlx5_fs_remove_rx_underlay_qpn);
+
+static struct mlx5_flow_root_namespace
+*get_root_namespace(struct mlx5_core_dev *dev, enum mlx5_flow_namespace_type ns_type)
+{
+ struct mlx5_flow_namespace *ns;
+
+ if (ns_type == MLX5_FLOW_NAMESPACE_ESW_EGRESS ||
+ ns_type == MLX5_FLOW_NAMESPACE_ESW_INGRESS)
+ ns = mlx5_get_flow_vport_acl_namespace(dev, ns_type, 0);
+ else
+ ns = mlx5_get_flow_namespace(dev, ns_type);
+ if (!ns)
+ return NULL;
+
+ return find_root(&ns->node);
+}
+
+struct mlx5_modify_hdr *mlx5_modify_header_alloc(struct mlx5_core_dev *dev,
+ u8 ns_type, u8 num_actions,
+ void *modify_actions)
+{
+ struct mlx5_flow_root_namespace *root;
+ struct mlx5_modify_hdr *modify_hdr;
+ int err;
+
+ root = get_root_namespace(dev, ns_type);
+ if (!root)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ modify_hdr = kzalloc(sizeof(*modify_hdr), GFP_KERNEL);
+ if (!modify_hdr)
+ return ERR_PTR(-ENOMEM);
+
+ modify_hdr->ns_type = ns_type;
+ err = root->cmds->modify_header_alloc(root, ns_type, num_actions,
+ modify_actions, modify_hdr);
+ if (err) {
+ kfree(modify_hdr);
+ return ERR_PTR(err);
+ }
+
+ return modify_hdr;
+}
+EXPORT_SYMBOL(mlx5_modify_header_alloc);
+
+void mlx5_modify_header_dealloc(struct mlx5_core_dev *dev,
+ struct mlx5_modify_hdr *modify_hdr)
+{
+ struct mlx5_flow_root_namespace *root;
+
+ root = get_root_namespace(dev, modify_hdr->ns_type);
+ if (WARN_ON(!root))
+ return;
+ root->cmds->modify_header_dealloc(root, modify_hdr);
+ kfree(modify_hdr);
+}
+EXPORT_SYMBOL(mlx5_modify_header_dealloc);
+
+struct mlx5_pkt_reformat *mlx5_packet_reformat_alloc(struct mlx5_core_dev *dev,
+ int reformat_type,
+ size_t size,
+ void *reformat_data,
+ enum mlx5_flow_namespace_type ns_type)
+{
+ struct mlx5_pkt_reformat *pkt_reformat;
+ struct mlx5_flow_root_namespace *root;
+ int err;
+
+ root = get_root_namespace(dev, ns_type);
+ if (!root)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ pkt_reformat = kzalloc(sizeof(*pkt_reformat), GFP_KERNEL);
+ if (!pkt_reformat)
+ return ERR_PTR(-ENOMEM);
+
+ pkt_reformat->ns_type = ns_type;
+ pkt_reformat->reformat_type = reformat_type;
+ err = root->cmds->packet_reformat_alloc(root, reformat_type, size,
+ reformat_data, ns_type,
+ pkt_reformat);
+ if (err) {
+ kfree(pkt_reformat);
+ return ERR_PTR(err);
+ }
+
+ return pkt_reformat;
+}
+EXPORT_SYMBOL(mlx5_packet_reformat_alloc);
+
+void mlx5_packet_reformat_dealloc(struct mlx5_core_dev *dev,
+ struct mlx5_pkt_reformat *pkt_reformat)
+{
+ struct mlx5_flow_root_namespace *root;
+
+ root = get_root_namespace(dev, pkt_reformat->ns_type);
+ if (WARN_ON(!root))
+ return;
+ root->cmds->packet_reformat_dealloc(root, pkt_reformat);
+ kfree(pkt_reformat);
+}
+EXPORT_SYMBOL(mlx5_packet_reformat_dealloc);
+
+int mlx5_flow_namespace_set_peer(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_root_namespace *peer_ns)
+{
+ if (peer_ns && ns->mode != peer_ns->mode) {
+ mlx5_core_err(ns->dev,
+ "Can't peer namespace of different steering mode\n");
+ return -EINVAL;
+ }
+
+ return ns->cmds->set_peer(ns, peer_ns);
+}
+
+/* This function should be called only at init stage of the namespace.
+ * It is not safe to call this function while steering operations
+ * are executed in the namespace.
+ */
+int mlx5_flow_namespace_set_mode(struct mlx5_flow_namespace *ns,
+ enum mlx5_flow_steering_mode mode)
+{
+ struct mlx5_flow_root_namespace *root;
+ const struct mlx5_flow_cmds *cmds;
+ int err;
+
+ root = find_root(&ns->node);
+ if (&root->ns != ns)
+ /* Can't set cmds to non root namespace */
+ return -EINVAL;
+
+ if (root->table_type != FS_FT_FDB)
+ return -EOPNOTSUPP;
+
+ if (root->mode == mode)
+ return 0;
+
+ if (mode == MLX5_FLOW_STEERING_MODE_SMFS)
+ cmds = mlx5_fs_cmd_get_dr_cmds();
+ else
+ cmds = mlx5_fs_cmd_get_fw_cmds();
+ if (!cmds)
+ return -EOPNOTSUPP;
+
+ err = cmds->create_ns(root);
+ if (err) {
+ mlx5_core_err(root->dev, "Failed to create flow namespace (%d)\n",
+ err);
+ return err;
+ }
+
+ root->cmds->destroy_ns(root);
+ root->cmds = cmds;
+ root->mode = mode;
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h
index c1252d6..00717eb 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.h
@@ -37,6 +37,24 @@
#include <linux/mlx5/fs.h>
#include <linux/rhashtable.h>
#include <linux/llist.h>
+#include <steering/fs_dr.h>
+
+struct mlx5_modify_hdr {
+ enum mlx5_flow_namespace_type ns_type;
+ union {
+ struct mlx5_fs_dr_action action;
+ u32 id;
+ };
+};
+
+struct mlx5_pkt_reformat {
+ enum mlx5_flow_namespace_type ns_type;
+ int reformat_type; /* from mlx5_ifc */
+ union {
+ struct mlx5_fs_dr_action action;
+ u32 id;
+ };
+};
/* FS_TYPE_PRIO_CHAINS is a PRIO that will have namespaces only,
* and those are in parallel to one another when going over them to connect
@@ -80,9 +98,15 @@ enum fs_fte_status {
FS_FTE_STATUS_EXISTING = 1UL << 0,
};
+enum mlx5_flow_steering_mode {
+ MLX5_FLOW_STEERING_MODE_DMFS,
+ MLX5_FLOW_STEERING_MODE_SMFS
+};
+
struct mlx5_flow_steering {
struct mlx5_core_dev *dev;
- struct kmem_cache *fgs_cache;
+ enum mlx5_flow_steering_mode mode;
+ struct kmem_cache *fgs_cache;
struct kmem_cache *ftes_cache;
struct mlx5_flow_root_namespace *root_ns;
struct mlx5_flow_root_namespace *fdb_root_ns;
@@ -128,6 +152,7 @@ struct mlx5_flow_handle {
/* Type of children is mlx5_flow_group */
struct mlx5_flow_table {
struct fs_node node;
+ struct mlx5_fs_dr_table fs_dr_table;
u32 id;
u16 vport;
unsigned int max_fte;
@@ -145,6 +170,7 @@ struct mlx5_flow_table {
struct list_head fwd_rules;
u32 flags;
struct rhltable fgs_hash;
+ enum mlx5_flow_table_miss_action def_miss_action;
};
struct mlx5_ft_underlay_qp {
@@ -167,6 +193,7 @@ struct mlx5_ft_underlay_qp {
/* Type of children is mlx5_flow_rule */
struct fs_fte {
struct fs_node node;
+ struct mlx5_fs_dr_rule fs_dr_rule;
u32 val[MLX5_ST_SZ_DW_MATCH_PARAM];
u32 dests_size;
u32 index;
@@ -191,6 +218,7 @@ struct fs_prio {
struct mlx5_flow_namespace {
/* parent == NULL => root ns */
struct fs_node node;
+ enum mlx5_flow_table_miss_action def_miss_action;
};
struct mlx5_flow_group_mask {
@@ -201,6 +229,7 @@ struct mlx5_flow_group_mask {
/* Type of children is fs_fte */
struct mlx5_flow_group {
struct fs_node node;
+ struct mlx5_fs_dr_matcher fs_dr_matcher;
struct mlx5_flow_group_mask mask;
u32 start_index;
u32 max_ftes;
@@ -212,6 +241,8 @@ struct mlx5_flow_group {
struct mlx5_flow_root_namespace {
struct mlx5_flow_namespace ns;
+ enum mlx5_flow_steering_mode mode;
+ struct mlx5_fs_dr_domain fs_dr_domain;
enum fs_flow_table_type table_type;
struct mlx5_core_dev *dev;
struct mlx5_flow_table *root_ft;
@@ -219,7 +250,6 @@ struct mlx5_flow_root_namespace {
struct mutex chain_lock;
struct list_head underlay_qpns;
const struct mlx5_flow_cmds *cmds;
- enum mlx5_flow_table_miss_action def_miss_action;
};
int mlx5_init_fc_stats(struct mlx5_core_dev *dev);
@@ -230,6 +260,14 @@ void mlx5_fc_queue_stats_work(struct mlx5_core_dev *dev,
void mlx5_fc_update_sampling_interval(struct mlx5_core_dev *dev,
unsigned long interval);
+const struct mlx5_flow_cmds *mlx5_fs_cmd_get_fw_cmds(void);
+
+int mlx5_flow_namespace_set_peer(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_root_namespace *peer_ns);
+
+int mlx5_flow_namespace_set_mode(struct mlx5_flow_namespace *ns,
+ enum mlx5_flow_steering_mode mode);
+
int mlx5_init_fs(struct mlx5_core_dev *dev);
void mlx5_cleanup_fs(struct mlx5_core_dev *dev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index 9314777..d685122 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -590,7 +590,8 @@ mlx5_fw_fatal_reporter_dump(struct devlink_health_reporter *reporter,
data_size = crdump_size - offset;
else
data_size = MLX5_CR_DUMP_CHUNK_SIZE;
- err = devlink_fmsg_binary_put(fmsg, cr_data, data_size);
+ err = devlink_fmsg_binary_put(fmsg, (char *)cr_data + offset,
+ data_size);
if (err)
goto free_data;
}
@@ -700,6 +701,16 @@ static void poll_health(struct timer_list *t)
if (dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
goto out;
+ fatal_error = check_fatal_sensors(dev);
+
+ if (fatal_error && !health->fatal_error) {
+ mlx5_core_err(dev, "Fatal error %u detected\n", fatal_error);
+ dev->priv.health.fatal_error = fatal_error;
+ print_health_info(dev);
+ mlx5_trigger_health_work(dev);
+ goto out;
+ }
+
count = ioread32be(health->health_counter);
if (count == health->prev)
++health->miss_counter;
@@ -718,15 +729,6 @@ static void poll_health(struct timer_list *t)
if (health->synd && health->synd != prev_synd)
queue_work(health->wq, &health->report_work);
- fatal_error = check_fatal_sensors(dev);
-
- if (fatal_error && !health->fatal_error) {
- mlx5_core_err(dev, "Fatal error %u detected\n", fatal_error);
- dev->priv.health.fatal_error = fatal_error;
- print_health_info(dev);
- mlx5_trigger_health_work(dev);
- }
-
out:
mod_timer(&health->timer, get_next_poll_jiffies());
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
index 1a2560e..3ed8ab2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
@@ -279,7 +279,7 @@ static int mlx5i_init_tx(struct mlx5e_priv *priv)
return err;
}
- err = mlx5i_create_tis(priv->mdev, ipriv->qp.qpn, &priv->tisn[0]);
+ err = mlx5i_create_tis(priv->mdev, ipriv->qp.qpn, &priv->tisn[0][0]);
if (err) {
mlx5_core_warn(priv->mdev, "create tis failed, %d\n", err);
goto err_destroy_underlay_qp;
@@ -296,7 +296,7 @@ static void mlx5i_cleanup_tx(struct mlx5e_priv *priv)
{
struct mlx5i_priv *ipriv = priv->ppriv;
- mlx5e_destroy_tis(priv->mdev, priv->tisn[0]);
+ mlx5e_destroy_tis(priv->mdev, priv->tisn[0][0]);
mlx5i_destroy_underlay_qp(priv->mdev, &ipriv->qp);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
index c5a491e..96e6418 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c
@@ -210,7 +210,7 @@ static int mlx5i_pkey_open(struct net_device *netdev)
goto err_unint_underlay_qp;
}
- err = mlx5i_create_tis(mdev, ipriv->qp.qpn, &epriv->tisn[0]);
+ err = mlx5i_create_tis(mdev, ipriv->qp.qpn, &epriv->tisn[0][0]);
if (err) {
mlx5_core_warn(mdev, "create child tis failed, %d\n", err);
goto err_remove_rx_uderlay_qp;
@@ -228,7 +228,7 @@ static int mlx5i_pkey_open(struct net_device *netdev)
return 0;
err_clear_state_opened_flag:
- mlx5e_destroy_tis(mdev, epriv->tisn[0]);
+ mlx5e_destroy_tis(mdev, epriv->tisn[0][0]);
err_remove_rx_uderlay_qp:
mlx5_fs_remove_rx_underlay_qpn(mdev, ipriv->qp.qpn);
err_unint_underlay_qp:
@@ -257,7 +257,7 @@ static int mlx5i_pkey_close(struct net_device *netdev)
mlx5i_uninit_underlay_qp(priv);
mlx5e_deactivate_priv_channels(priv);
mlx5e_close_channels(&priv->channels);
- mlx5e_destroy_tis(mdev, priv->tisn[0]);
+ mlx5e_destroy_tis(mdev, priv->tisn[0][0]);
unlock:
mutex_unlock(&priv->state_lock);
return 0;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c b/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c
index e697663..5d20d61 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c
@@ -248,6 +248,9 @@ static int mlx5_lag_fib_event(struct notifier_block *nb,
struct net_device *fib_dev;
struct fib_info *fi;
+ if (!net_eq(info->net, &init_net))
+ return NOTIFY_DONE;
+
if (info->family != AF_INET)
return NOTIFY_DONE;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c
index ea1d4d2..3fc575d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c
@@ -2,6 +2,7 @@
// Copyright (c) 2019 Mellanox Technologies.
#include "mlx5_core.h"
+#include "lib/mlx5.h"
int mlx5_create_encryption_key(struct mlx5_core_dev *mdev,
void *key, u32 sz_bytes,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/dm.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/dm.c
new file mode 100644
index 0000000..e065c2f
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/dm.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2019 Mellanox Technologies
+
+#include <linux/mlx5/driver.h>
+#include <linux/mlx5/device.h>
+
+#include "mlx5_core.h"
+#include "lib/mlx5.h"
+
+struct mlx5_dm {
+ /* protect access to icm bitmask */
+ spinlock_t lock;
+ unsigned long *steering_sw_icm_alloc_blocks;
+ unsigned long *header_modify_sw_icm_alloc_blocks;
+};
+
+struct mlx5_dm *mlx5_dm_create(struct mlx5_core_dev *dev)
+{
+ u64 header_modify_icm_blocks = 0;
+ u64 steering_icm_blocks = 0;
+ struct mlx5_dm *dm;
+
+ if (!(MLX5_CAP_GEN_64(dev, general_obj_types) & MLX5_GENERAL_OBJ_TYPES_CAP_SW_ICM))
+ return 0;
+
+ dm = kzalloc(sizeof(*dm), GFP_KERNEL);
+ if (!dm)
+ return ERR_PTR(-ENOMEM);
+
+ spin_lock_init(&dm->lock);
+
+ if (MLX5_CAP64_DEV_MEM(dev, steering_sw_icm_start_address)) {
+ steering_icm_blocks =
+ BIT(MLX5_CAP_DEV_MEM(dev, log_steering_sw_icm_size) -
+ MLX5_LOG_SW_ICM_BLOCK_SIZE(dev));
+
+ dm->steering_sw_icm_alloc_blocks =
+ kcalloc(BITS_TO_LONGS(steering_icm_blocks),
+ sizeof(unsigned long), GFP_KERNEL);
+ if (!dm->steering_sw_icm_alloc_blocks)
+ goto err_steering;
+ }
+
+ if (MLX5_CAP64_DEV_MEM(dev, header_modify_sw_icm_start_address)) {
+ header_modify_icm_blocks =
+ BIT(MLX5_CAP_DEV_MEM(dev, log_header_modify_sw_icm_size) -
+ MLX5_LOG_SW_ICM_BLOCK_SIZE(dev));
+
+ dm->header_modify_sw_icm_alloc_blocks =
+ kcalloc(BITS_TO_LONGS(header_modify_icm_blocks),
+ sizeof(unsigned long), GFP_KERNEL);
+ if (!dm->header_modify_sw_icm_alloc_blocks)
+ goto err_modify_hdr;
+ }
+
+ return dm;
+
+err_modify_hdr:
+ kfree(dm->steering_sw_icm_alloc_blocks);
+
+err_steering:
+ kfree(dm);
+
+ return ERR_PTR(-ENOMEM);
+}
+
+void mlx5_dm_cleanup(struct mlx5_core_dev *dev)
+{
+ struct mlx5_dm *dm = dev->dm;
+
+ if (!dev->dm)
+ return;
+
+ if (dm->steering_sw_icm_alloc_blocks) {
+ WARN_ON(!bitmap_empty(dm->steering_sw_icm_alloc_blocks,
+ BIT(MLX5_CAP_DEV_MEM(dev, log_steering_sw_icm_size) -
+ MLX5_LOG_SW_ICM_BLOCK_SIZE(dev))));
+ kfree(dm->steering_sw_icm_alloc_blocks);
+ }
+
+ if (dm->header_modify_sw_icm_alloc_blocks) {
+ WARN_ON(!bitmap_empty(dm->header_modify_sw_icm_alloc_blocks,
+ BIT(MLX5_CAP_DEV_MEM(dev,
+ log_header_modify_sw_icm_size) -
+ MLX5_LOG_SW_ICM_BLOCK_SIZE(dev))));
+ kfree(dm->header_modify_sw_icm_alloc_blocks);
+ }
+
+ kfree(dm);
+}
+
+int mlx5_dm_sw_icm_alloc(struct mlx5_core_dev *dev, enum mlx5_sw_icm_type type,
+ u64 length, u16 uid, phys_addr_t *addr, u32 *obj_id)
+{
+ u32 num_blocks = DIV_ROUND_UP_ULL(length, MLX5_SW_ICM_BLOCK_SIZE(dev));
+ u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {};
+ u32 in[MLX5_ST_SZ_DW(create_sw_icm_in)] = {};
+ struct mlx5_dm *dm = dev->dm;
+ unsigned long *block_map;
+ u64 icm_start_addr;
+ u32 log_icm_size;
+ u32 max_blocks;
+ u64 block_idx;
+ void *sw_icm;
+ int ret;
+
+ if (!dev->dm)
+ return -EOPNOTSUPP;
+
+ if (!length || (length & (length - 1)) ||
+ length & (MLX5_SW_ICM_BLOCK_SIZE(dev) - 1))
+ return -EINVAL;
+
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+ MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_OBJ_TYPE_SW_ICM);
+ MLX5_SET(general_obj_in_cmd_hdr, in, uid, uid);
+
+ switch (type) {
+ case MLX5_SW_ICM_TYPE_STEERING:
+ icm_start_addr = MLX5_CAP64_DEV_MEM(dev, steering_sw_icm_start_address);
+ log_icm_size = MLX5_CAP_DEV_MEM(dev, log_steering_sw_icm_size);
+ block_map = dm->steering_sw_icm_alloc_blocks;
+ break;
+ case MLX5_SW_ICM_TYPE_HEADER_MODIFY:
+ icm_start_addr = MLX5_CAP64_DEV_MEM(dev, header_modify_sw_icm_start_address);
+ log_icm_size = MLX5_CAP_DEV_MEM(dev,
+ log_header_modify_sw_icm_size);
+ block_map = dm->header_modify_sw_icm_alloc_blocks;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (!block_map)
+ return -EOPNOTSUPP;
+
+ max_blocks = BIT(log_icm_size - MLX5_LOG_SW_ICM_BLOCK_SIZE(dev));
+ spin_lock(&dm->lock);
+ block_idx = bitmap_find_next_zero_area(block_map,
+ max_blocks,
+ 0,
+ num_blocks, 0);
+
+ if (block_idx < max_blocks)
+ bitmap_set(block_map,
+ block_idx, num_blocks);
+
+ spin_unlock(&dm->lock);
+
+ if (block_idx >= max_blocks)
+ return -ENOMEM;
+
+ sw_icm = MLX5_ADDR_OF(create_sw_icm_in, in, sw_icm);
+ icm_start_addr += block_idx << MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
+ MLX5_SET64(sw_icm, sw_icm, sw_icm_start_addr,
+ icm_start_addr);
+ MLX5_SET(sw_icm, sw_icm, log_sw_icm_size, ilog2(length));
+
+ ret = mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
+ if (ret) {
+ spin_lock(&dm->lock);
+ bitmap_clear(block_map,
+ block_idx, num_blocks);
+ spin_unlock(&dm->lock);
+
+ return ret;
+ }
+
+ *addr = icm_start_addr;
+ *obj_id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mlx5_dm_sw_icm_alloc);
+
+int mlx5_dm_sw_icm_dealloc(struct mlx5_core_dev *dev, enum mlx5_sw_icm_type type,
+ u64 length, u16 uid, phys_addr_t addr, u32 obj_id)
+{
+ u32 num_blocks = DIV_ROUND_UP_ULL(length, MLX5_SW_ICM_BLOCK_SIZE(dev));
+ u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {};
+ u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {};
+ struct mlx5_dm *dm = dev->dm;
+ unsigned long *block_map;
+ u64 icm_start_addr;
+ u64 start_idx;
+ int err;
+
+ if (!dev->dm)
+ return -EOPNOTSUPP;
+
+ switch (type) {
+ case MLX5_SW_ICM_TYPE_STEERING:
+ icm_start_addr = MLX5_CAP64_DEV_MEM(dev, steering_sw_icm_start_address);
+ block_map = dm->steering_sw_icm_alloc_blocks;
+ break;
+ case MLX5_SW_ICM_TYPE_HEADER_MODIFY:
+ icm_start_addr = MLX5_CAP64_DEV_MEM(dev, header_modify_sw_icm_start_address);
+ block_map = dm->header_modify_sw_icm_alloc_blocks;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+ MLX5_CMD_OP_DESTROY_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_OBJ_TYPE_SW_ICM);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, obj_id);
+ MLX5_SET(general_obj_in_cmd_hdr, in, uid, uid);
+
+ err = mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
+ if (err)
+ return err;
+
+ start_idx = (addr - icm_start_addr) >> MLX5_LOG_SW_ICM_BLOCK_SIZE(dev);
+ spin_lock(&dm->lock);
+ bitmap_clear(block_map,
+ start_idx, num_blocks);
+ spin_unlock(&dm->lock);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mlx5_dm_sw_icm_dealloc);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/hv.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/hv.c
index cf08d02..583dc7e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/hv.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/hv.c
@@ -12,7 +12,7 @@ static int mlx5_hv_config_common(struct mlx5_core_dev *dev, void *buf, int len,
int bytes_returned;
int block_id;
- if (offset % HV_CONFIG_BLOCK_SIZE_MAX || len % HV_CONFIG_BLOCK_SIZE_MAX)
+ if (offset % HV_CONFIG_BLOCK_SIZE_MAX || len != HV_CONFIG_BLOCK_SIZE_MAX)
return -EINVAL;
block_id = offset / HV_CONFIG_BLOCK_SIZE_MAX;
@@ -25,8 +25,8 @@ static int mlx5_hv_config_common(struct mlx5_core_dev *dev, void *buf, int len,
HV_CONFIG_BLOCK_SIZE_MAX, block_id);
/* Make sure len bytes were read successfully */
- if (read)
- rc |= !(len == bytes_returned);
+ if (read && !rc && len != bytes_returned)
+ rc = -EIO;
if (rc) {
mlx5_core_err(dev, "Failed to %s hv config, err = %d, len = %d, offset = %d\n",
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 61388ca..9648c22 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -496,6 +496,12 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev)
ODP_CAP_SET_MAX(dev, xrc_odp_caps.write);
ODP_CAP_SET_MAX(dev, xrc_odp_caps.read);
ODP_CAP_SET_MAX(dev, xrc_odp_caps.atomic);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.srq_receive);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.send);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.receive);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.write);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.read);
+ ODP_CAP_SET_MAX(dev, dc_odp_caps.atomic);
if (do_set)
err = set_caps(dev, set_ctx, set_sz,
@@ -870,6 +876,10 @@ static int mlx5_init_once(struct mlx5_core_dev *dev)
goto err_eswitch_cleanup;
}
+ dev->dm = mlx5_dm_create(dev);
+ if (IS_ERR(dev->dm))
+ mlx5_core_warn(dev, "Failed to init device memory%d\n", err);
+
dev->tracer = mlx5_fw_tracer_create(dev);
dev->hv_vhca = mlx5_hv_vhca_create(dev);
@@ -904,6 +914,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev)
{
mlx5_hv_vhca_destroy(dev->hv_vhca);
mlx5_fw_tracer_destroy(dev->tracer);
+ mlx5_dm_cleanup(dev);
mlx5_fpga_cleanup(dev);
mlx5_eswitch_cleanup(dev->priv.eswitch);
mlx5_sriov_cleanup(dev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
index 87b75b2..b100489 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
@@ -198,6 +198,9 @@ int mlx5_set_mtpps(struct mlx5_core_dev *mdev, u32 *mtpps, u32 mtpps_size);
int mlx5_query_mtppse(struct mlx5_core_dev *mdev, u8 pin, u8 *arm, u8 *mode);
int mlx5_set_mtppse(struct mlx5_core_dev *mdev, u8 pin, u8 arm, u8 mode);
+struct mlx5_dm *mlx5_dm_create(struct mlx5_core_dev *dev);
+void mlx5_dm_cleanup(struct mlx5_core_dev *dev);
+
#define MLX5_PPS_CAP(mdev) (MLX5_CAP_GEN((mdev), pps) && \
MLX5_CAP_GEN((mdev), pps_modify) && \
MLX5_CAP_MCAM_FEATURE((mdev), mtpps_fs) && \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/qp.c b/drivers/net/ethernet/mellanox/mlx5/core/qp.c
index b8ba74d..c3aea4c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/qp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/qp.c
@@ -53,7 +53,7 @@ mlx5_get_rsc(struct mlx5_qp_table *table, u32 rsn)
common = radix_tree_lookup(&table->tree, rsn);
if (common)
- atomic_inc(&common->refcount);
+ refcount_inc(&common->refcount);
spin_unlock_irqrestore(&table->lock, flags);
@@ -62,7 +62,7 @@ mlx5_get_rsc(struct mlx5_qp_table *table, u32 rsn)
void mlx5_core_put_rsc(struct mlx5_core_rsc_common *common)
{
- if (atomic_dec_and_test(&common->refcount))
+ if (refcount_dec_and_test(&common->refcount))
complete(&common->free);
}
@@ -162,7 +162,7 @@ static int rsc_event_notifier(struct notifier_block *nb,
common = mlx5_get_rsc(table, rsn);
if (!common) {
- mlx5_core_warn(dev, "Async event for bogus resource 0x%x\n", rsn);
+ mlx5_core_dbg(dev, "Async event for unknown resource 0x%x\n", rsn);
return NOTIFY_OK;
}
@@ -209,7 +209,7 @@ static int create_resource_common(struct mlx5_core_dev *dev,
if (err)
return err;
- atomic_set(&qp->common.refcount, 1);
+ refcount_set(&qp->common.refcount, 1);
init_completion(&qp->common.free);
qp->pid = current->pid;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/rdma.c b/drivers/net/ethernet/mellanox/mlx5/core/rdma.c
index 17ce9dd..0fc7de4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/rdma.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/rdma.c
@@ -14,9 +14,6 @@ static void mlx5_rdma_disable_roce_steering(struct mlx5_core_dev *dev)
{
struct mlx5_core_roce *roce = &dev->priv.roce;
- if (!roce->ft)
- return;
-
mlx5_del_flow_rules(roce->allow_rule);
mlx5_destroy_flow_group(roce->fg);
mlx5_destroy_flow_table(roce->ft);
@@ -51,7 +48,7 @@ static int mlx5_rdma_enable_roce_steering(struct mlx5_core_dev *dev)
return -ENOMEM;
}
- ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_RDMA_RX);
+ ns = mlx5_get_flow_namespace(dev, MLX5_FLOW_NAMESPACE_RDMA_RX_KERNEL);
if (!ns) {
mlx5_core_err(dev, "Failed to get RDMA RX namespace");
err = -EOPNOTSUPP;
@@ -145,6 +142,11 @@ static int mlx5_rdma_add_roce_addr(struct mlx5_core_dev *dev)
void mlx5_rdma_disable_roce(struct mlx5_core_dev *dev)
{
+ struct mlx5_core_roce *roce = &dev->priv.roce;
+
+ if (!roce->ft)
+ return;
+
mlx5_rdma_disable_roce_steering(dev);
mlx5_rdma_del_roce_addr(dev);
mlx5_nic_vport_disable_roce(dev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/steering/Makefile
new file mode 100644
index 0000000..c78512e
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+subdir-ccflags-y += -I$(src)/..
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
new file mode 100644
index 0000000..7d81a77
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c
@@ -0,0 +1,1588 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+enum dr_action_domain {
+ DR_ACTION_DOMAIN_NIC_INGRESS,
+ DR_ACTION_DOMAIN_NIC_EGRESS,
+ DR_ACTION_DOMAIN_FDB_INGRESS,
+ DR_ACTION_DOMAIN_FDB_EGRESS,
+ DR_ACTION_DOMAIN_MAX,
+};
+
+enum dr_action_valid_state {
+ DR_ACTION_STATE_ERR,
+ DR_ACTION_STATE_NO_ACTION,
+ DR_ACTION_STATE_REFORMAT,
+ DR_ACTION_STATE_MODIFY_HDR,
+ DR_ACTION_STATE_MODIFY_VLAN,
+ DR_ACTION_STATE_NON_TERM,
+ DR_ACTION_STATE_TERM,
+ DR_ACTION_STATE_MAX,
+};
+
+static const enum dr_action_valid_state
+next_action_state[DR_ACTION_DOMAIN_MAX][DR_ACTION_STATE_MAX][DR_ACTION_TYP_MAX] = {
+ [DR_ACTION_DOMAIN_NIC_INGRESS] = {
+ [DR_ACTION_STATE_NO_ACTION] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_QP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_TAG] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_TNL_L2_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_TNL_L3_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ },
+ [DR_ACTION_STATE_REFORMAT] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_QP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_TAG] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ },
+ [DR_ACTION_STATE_MODIFY_HDR] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_QP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_TAG] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_HDR,
+ },
+ [DR_ACTION_STATE_MODIFY_VLAN] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_QP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_TAG] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ },
+ [DR_ACTION_STATE_NON_TERM] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_QP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_TAG] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_TNL_L2_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_TNL_L3_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ },
+ [DR_ACTION_STATE_TERM] = {
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_TERM,
+ },
+ },
+ [DR_ACTION_DOMAIN_NIC_EGRESS] = {
+ [DR_ACTION_STATE_NO_ACTION] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ },
+ [DR_ACTION_STATE_REFORMAT] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_REFORMAT,
+ },
+ [DR_ACTION_STATE_MODIFY_HDR] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ },
+ [DR_ACTION_STATE_MODIFY_VLAN] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ },
+ [DR_ACTION_STATE_NON_TERM] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ },
+ [DR_ACTION_STATE_TERM] = {
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_TERM,
+ },
+ },
+ [DR_ACTION_DOMAIN_FDB_INGRESS] = {
+ [DR_ACTION_STATE_NO_ACTION] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_TNL_L2_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_TNL_L3_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_REFORMAT] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_MODIFY_HDR] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_MODIFY_VLAN] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ },
+ [DR_ACTION_STATE_NON_TERM] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_TNL_L2_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_TNL_L3_TO_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_POP_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_TERM] = {
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_TERM,
+ },
+ },
+ [DR_ACTION_DOMAIN_FDB_EGRESS] = {
+ [DR_ACTION_STATE_NO_ACTION] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_REFORMAT] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_MODIFY_HDR] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_MODIFY_VLAN] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_NON_TERM] = {
+ [DR_ACTION_TYP_DROP] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_FT] = DR_ACTION_STATE_TERM,
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_NON_TERM,
+ [DR_ACTION_TYP_MODIFY_HDR] = DR_ACTION_STATE_MODIFY_HDR,
+ [DR_ACTION_TYP_L2_TO_TNL_L2] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_L2_TO_TNL_L3] = DR_ACTION_STATE_REFORMAT,
+ [DR_ACTION_TYP_PUSH_VLAN] = DR_ACTION_STATE_MODIFY_VLAN,
+ [DR_ACTION_TYP_VPORT] = DR_ACTION_STATE_TERM,
+ },
+ [DR_ACTION_STATE_TERM] = {
+ [DR_ACTION_TYP_CTR] = DR_ACTION_STATE_TERM,
+ },
+ },
+};
+
+struct dr_action_modify_field_conv {
+ u16 hw_field;
+ u8 start;
+ u8 end;
+ u8 l3_type;
+ u8 l4_type;
+};
+
+static const struct dr_action_modify_field_conv dr_action_conv_arr[] = {
+ [MLX5_ACTION_IN_FIELD_OUT_SMAC_47_16] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L2_1, .start = 16, .end = 47,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_SMAC_15_0] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L2_1, .start = 0, .end = 15,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_ETHERTYPE] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L2_2, .start = 32, .end = 47,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DMAC_47_16] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L2_0, .start = 16, .end = 47,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DMAC_15_0] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L2_0, .start = 0, .end = 15,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_IP_DSCP] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_1, .start = 0, .end = 5,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_TCP_FLAGS] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_0, .start = 48, .end = 56,
+ .l4_type = MLX5DR_ACTION_MDFY_HW_HDR_L4_TCP,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_TCP_SPORT] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_0, .start = 0, .end = 15,
+ .l4_type = MLX5DR_ACTION_MDFY_HW_HDR_L4_TCP,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_TCP_DPORT] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_0, .start = 16, .end = 31,
+ .l4_type = MLX5DR_ACTION_MDFY_HW_HDR_L4_TCP,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_IP_TTL] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_1, .start = 8, .end = 15,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV4,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_IPV6_HOPLIMIT] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_1, .start = 8, .end = 15,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_UDP_SPORT] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_0, .start = 0, .end = 15,
+ .l4_type = MLX5DR_ACTION_MDFY_HW_HDR_L4_UDP,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_UDP_DPORT] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_0, .start = 16, .end = 31,
+ .l4_type = MLX5DR_ACTION_MDFY_HW_HDR_L4_UDP,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_SIPV6_127_96] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_3, .start = 32, .end = 63,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_SIPV6_95_64] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_3, .start = 0, .end = 31,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_SIPV6_63_32] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_4, .start = 32, .end = 63,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_SIPV6_31_0] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_4, .start = 0, .end = 31,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DIPV6_127_96] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_0, .start = 32, .end = 63,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DIPV6_95_64] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_0, .start = 0, .end = 31,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DIPV6_63_32] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_2, .start = 32, .end = 63,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DIPV6_31_0] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_2, .start = 0, .end = 31,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_SIPV4] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_0, .start = 0, .end = 31,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV4,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_DIPV4] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L3_0, .start = 32, .end = 63,
+ .l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV4,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_A] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_METADATA, .start = 0, .end = 31,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_B] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_METADATA, .start = 32, .end = 63,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_C_0] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_REG_0, .start = 32, .end = 63,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_C_1] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_REG_0, .start = 0, .end = 31,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_C_2] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_REG_1, .start = 32, .end = 63,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_C_3] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_REG_1, .start = 0, .end = 31,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_C_4] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_REG_2, .start = 32, .end = 63,
+ },
+ [MLX5_ACTION_IN_FIELD_METADATA_REG_C_5] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_REG_2, .start = 0, .end = 31,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_TCP_SEQ_NUM] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_1, .start = 32, .end = 63,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_TCP_ACK_NUM] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L4_1, .start = 0, .end = 31,
+ },
+ [MLX5_ACTION_IN_FIELD_OUT_FIRST_VID] = {
+ .hw_field = MLX5DR_ACTION_MDFY_HW_FLD_L2_2, .start = 0, .end = 15,
+ },
+};
+
+#define MAX_VLANS 2
+struct dr_action_vlan_info {
+ int count;
+ u32 headers[MAX_VLANS];
+};
+
+struct dr_action_apply_attr {
+ u32 modify_index;
+ u16 modify_actions;
+ u32 decap_index;
+ u16 decap_actions;
+ u8 decap_with_vlan:1;
+ u64 final_icm_addr;
+ u32 flow_tag;
+ u32 ctr_id;
+ u16 gvmi;
+ u16 hit_gvmi;
+ u32 reformat_id;
+ u32 reformat_size;
+ struct dr_action_vlan_info vlans;
+};
+
+static int
+dr_action_reformat_to_action_type(enum mlx5dr_action_reformat_type reformat_type,
+ enum mlx5dr_action_type *action_type)
+{
+ switch (reformat_type) {
+ case DR_ACTION_REFORMAT_TYP_TNL_L2_TO_L2:
+ *action_type = DR_ACTION_TYP_TNL_L2_TO_L2;
+ break;
+ case DR_ACTION_REFORMAT_TYP_L2_TO_TNL_L2:
+ *action_type = DR_ACTION_TYP_L2_TO_TNL_L2;
+ break;
+ case DR_ACTION_REFORMAT_TYP_TNL_L3_TO_L2:
+ *action_type = DR_ACTION_TYP_TNL_L3_TO_L2;
+ break;
+ case DR_ACTION_REFORMAT_TYP_L2_TO_TNL_L3:
+ *action_type = DR_ACTION_TYP_L2_TO_TNL_L3;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void dr_actions_init_next_ste(u8 **last_ste,
+ u32 *added_stes,
+ enum mlx5dr_ste_entry_type entry_type,
+ u16 gvmi)
+{
+ (*added_stes)++;
+ *last_ste += DR_STE_SIZE;
+ mlx5dr_ste_init(*last_ste, MLX5DR_STE_LU_TYPE_DONT_CARE, entry_type, gvmi);
+}
+
+static void dr_actions_apply_tx(struct mlx5dr_domain *dmn,
+ u8 *action_type_set,
+ u8 *last_ste,
+ struct dr_action_apply_attr *attr,
+ u32 *added_stes)
+{
+ bool encap = action_type_set[DR_ACTION_TYP_L2_TO_TNL_L2] ||
+ action_type_set[DR_ACTION_TYP_L2_TO_TNL_L3];
+
+ /* We want to make sure the modify header comes before L2
+ * encapsulation. The reason for that is that we support
+ * modify headers for outer headers only
+ */
+ if (action_type_set[DR_ACTION_TYP_MODIFY_HDR]) {
+ mlx5dr_ste_set_entry_type(last_ste, MLX5DR_STE_TYPE_MODIFY_PKT);
+ mlx5dr_ste_set_rewrite_actions(last_ste,
+ attr->modify_actions,
+ attr->modify_index);
+ }
+
+ if (action_type_set[DR_ACTION_TYP_PUSH_VLAN]) {
+ int i;
+
+ for (i = 0; i < attr->vlans.count; i++) {
+ if (i || action_type_set[DR_ACTION_TYP_MODIFY_HDR])
+ dr_actions_init_next_ste(&last_ste,
+ added_stes,
+ MLX5DR_STE_TYPE_TX,
+ attr->gvmi);
+
+ mlx5dr_ste_set_tx_push_vlan(last_ste,
+ attr->vlans.headers[i],
+ encap);
+ }
+ }
+
+ if (encap) {
+ /* Modify header and encapsulation require a different STEs.
+ * Since modify header STE format doesn't support encapsulation
+ * tunneling_action.
+ */
+ if (action_type_set[DR_ACTION_TYP_MODIFY_HDR] ||
+ action_type_set[DR_ACTION_TYP_PUSH_VLAN])
+ dr_actions_init_next_ste(&last_ste,
+ added_stes,
+ MLX5DR_STE_TYPE_TX,
+ attr->gvmi);
+
+ mlx5dr_ste_set_tx_encap(last_ste,
+ attr->reformat_id,
+ attr->reformat_size,
+ action_type_set[DR_ACTION_TYP_L2_TO_TNL_L3]);
+ /* Whenever prio_tag_required enabled, we can be sure that the
+ * previous table (ACL) already push vlan to our packet,
+ * And due to HW limitation we need to set this bit, otherwise
+ * push vlan + reformat will not work.
+ */
+ if (MLX5_CAP_GEN(dmn->mdev, prio_tag_required))
+ mlx5dr_ste_set_go_back_bit(last_ste);
+ }
+
+ if (action_type_set[DR_ACTION_TYP_CTR])
+ mlx5dr_ste_set_counter_id(last_ste, attr->ctr_id);
+}
+
+static void dr_actions_apply_rx(u8 *action_type_set,
+ u8 *last_ste,
+ struct dr_action_apply_attr *attr,
+ u32 *added_stes)
+{
+ if (action_type_set[DR_ACTION_TYP_CTR])
+ mlx5dr_ste_set_counter_id(last_ste, attr->ctr_id);
+
+ if (action_type_set[DR_ACTION_TYP_TNL_L3_TO_L2]) {
+ mlx5dr_ste_set_entry_type(last_ste, MLX5DR_STE_TYPE_MODIFY_PKT);
+ mlx5dr_ste_set_rx_decap_l3(last_ste, attr->decap_with_vlan);
+ mlx5dr_ste_set_rewrite_actions(last_ste,
+ attr->decap_actions,
+ attr->decap_index);
+ }
+
+ if (action_type_set[DR_ACTION_TYP_TNL_L2_TO_L2])
+ mlx5dr_ste_set_rx_decap(last_ste);
+
+ if (action_type_set[DR_ACTION_TYP_POP_VLAN]) {
+ int i;
+
+ for (i = 0; i < attr->vlans.count; i++) {
+ if (i ||
+ action_type_set[DR_ACTION_TYP_TNL_L2_TO_L2] ||
+ action_type_set[DR_ACTION_TYP_TNL_L3_TO_L2])
+ dr_actions_init_next_ste(&last_ste,
+ added_stes,
+ MLX5DR_STE_TYPE_RX,
+ attr->gvmi);
+
+ mlx5dr_ste_set_rx_pop_vlan(last_ste);
+ }
+ }
+
+ if (action_type_set[DR_ACTION_TYP_MODIFY_HDR]) {
+ if (mlx5dr_ste_get_entry_type(last_ste) == MLX5DR_STE_TYPE_MODIFY_PKT)
+ dr_actions_init_next_ste(&last_ste,
+ added_stes,
+ MLX5DR_STE_TYPE_MODIFY_PKT,
+ attr->gvmi);
+ else
+ mlx5dr_ste_set_entry_type(last_ste, MLX5DR_STE_TYPE_MODIFY_PKT);
+
+ mlx5dr_ste_set_rewrite_actions(last_ste,
+ attr->modify_actions,
+ attr->modify_index);
+ }
+
+ if (action_type_set[DR_ACTION_TYP_TAG]) {
+ if (mlx5dr_ste_get_entry_type(last_ste) == MLX5DR_STE_TYPE_MODIFY_PKT)
+ dr_actions_init_next_ste(&last_ste,
+ added_stes,
+ MLX5DR_STE_TYPE_RX,
+ attr->gvmi);
+
+ mlx5dr_ste_rx_set_flow_tag(last_ste, attr->flow_tag);
+ }
+}
+
+/* Apply the actions on the rule STE array starting from the last_ste.
+ * Actions might require more than one STE, new_num_stes will return
+ * the new size of the STEs array, rule with actions.
+ */
+static void dr_actions_apply(struct mlx5dr_domain *dmn,
+ enum mlx5dr_ste_entry_type ste_type,
+ u8 *action_type_set,
+ u8 *last_ste,
+ struct dr_action_apply_attr *attr,
+ u32 *new_num_stes)
+{
+ u32 added_stes = 0;
+
+ if (ste_type == MLX5DR_STE_TYPE_RX)
+ dr_actions_apply_rx(action_type_set, last_ste, attr, &added_stes);
+ else
+ dr_actions_apply_tx(dmn, action_type_set, last_ste, attr, &added_stes);
+
+ last_ste += added_stes * DR_STE_SIZE;
+ *new_num_stes += added_stes;
+
+ mlx5dr_ste_set_hit_gvmi(last_ste, attr->hit_gvmi);
+ mlx5dr_ste_set_hit_addr(last_ste, attr->final_icm_addr, 1);
+}
+
+static enum dr_action_domain
+dr_action_get_action_domain(enum mlx5dr_domain_type domain,
+ enum mlx5dr_ste_entry_type ste_type)
+{
+ switch (domain) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ return DR_ACTION_DOMAIN_NIC_INGRESS;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ return DR_ACTION_DOMAIN_NIC_EGRESS;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ if (ste_type == MLX5DR_STE_TYPE_RX)
+ return DR_ACTION_DOMAIN_FDB_INGRESS;
+ return DR_ACTION_DOMAIN_FDB_EGRESS;
+ default:
+ WARN_ON(true);
+ return DR_ACTION_DOMAIN_MAX;
+ }
+}
+
+static
+int dr_action_validate_and_get_next_state(enum dr_action_domain action_domain,
+ u32 action_type,
+ u32 *state)
+{
+ u32 cur_state = *state;
+
+ /* Check action state machine is valid */
+ *state = next_action_state[action_domain][cur_state][action_type];
+
+ if (*state == DR_ACTION_STATE_ERR)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+static int dr_action_handle_cs_recalc(struct mlx5dr_domain *dmn,
+ struct mlx5dr_action *dest_action,
+ u64 *final_icm_addr)
+{
+ int ret;
+
+ switch (dest_action->action_type) {
+ case DR_ACTION_TYP_FT:
+ /* Allow destination flow table only if table is a terminating
+ * table, since there is an *assumption* that in such case FW
+ * will recalculate the CS.
+ */
+ if (dest_action->dest_tbl.is_fw_tbl) {
+ *final_icm_addr = dest_action->dest_tbl.fw_tbl.rx_icm_addr;
+ } else {
+ mlx5dr_dbg(dmn,
+ "Destination FT should be terminating when modify TTL is used\n");
+ return -EINVAL;
+ }
+ break;
+
+ case DR_ACTION_TYP_VPORT:
+ /* If destination is vport we will get the FW flow table
+ * that recalculates the CS and forwards to the vport.
+ */
+ ret = mlx5dr_domain_cache_get_recalc_cs_ft_addr(dest_action->vport.dmn,
+ dest_action->vport.num,
+ final_icm_addr);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed to get FW cs recalc flow table\n");
+ return ret;
+ }
+ break;
+
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+#define WITH_VLAN_NUM_HW_ACTIONS 6
+
+int mlx5dr_actions_build_ste_arr(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_action *actions[],
+ u32 num_actions,
+ u8 *ste_arr,
+ u32 *new_hw_ste_arr_sz)
+{
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+ bool rx_rule = nic_dmn->ste_type == MLX5DR_STE_TYPE_RX;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ u8 action_type_set[DR_ACTION_TYP_MAX] = {};
+ struct mlx5dr_action *dest_action = NULL;
+ u32 state = DR_ACTION_STATE_NO_ACTION;
+ struct dr_action_apply_attr attr = {};
+ enum dr_action_domain action_domain;
+ bool recalc_cs_required = false;
+ u8 *last_ste;
+ int i, ret;
+
+ attr.gvmi = dmn->info.caps.gvmi;
+ attr.hit_gvmi = dmn->info.caps.gvmi;
+ attr.final_icm_addr = nic_dmn->default_icm_addr;
+ action_domain = dr_action_get_action_domain(dmn->type, nic_dmn->ste_type);
+
+ for (i = 0; i < num_actions; i++) {
+ struct mlx5dr_action *action;
+ int max_actions_type = 1;
+ u32 action_type;
+
+ action = actions[i];
+ action_type = action->action_type;
+
+ switch (action_type) {
+ case DR_ACTION_TYP_DROP:
+ attr.final_icm_addr = nic_dmn->drop_icm_addr;
+ break;
+ case DR_ACTION_TYP_FT:
+ dest_action = action;
+ if (!action->dest_tbl.is_fw_tbl) {
+ if (action->dest_tbl.tbl->dmn != dmn) {
+ mlx5dr_dbg(dmn,
+ "Destination table belongs to a different domain\n");
+ goto out_invalid_arg;
+ }
+ if (action->dest_tbl.tbl->level <= matcher->tbl->level) {
+ mlx5dr_dbg(dmn,
+ "Destination table level should be higher than source table\n");
+ goto out_invalid_arg;
+ }
+ attr.final_icm_addr = rx_rule ?
+ action->dest_tbl.tbl->rx.s_anchor->chunk->icm_addr :
+ action->dest_tbl.tbl->tx.s_anchor->chunk->icm_addr;
+ } else {
+ struct mlx5dr_cmd_query_flow_table_details output;
+ int ret;
+
+ /* get the relevant addresses */
+ if (!action->dest_tbl.fw_tbl.rx_icm_addr) {
+ ret = mlx5dr_cmd_query_flow_table(action->dest_tbl.fw_tbl.mdev,
+ action->dest_tbl.fw_tbl.ft->type,
+ action->dest_tbl.fw_tbl.ft->id,
+ &output);
+ if (!ret) {
+ action->dest_tbl.fw_tbl.tx_icm_addr =
+ output.sw_owner_icm_root_1;
+ action->dest_tbl.fw_tbl.rx_icm_addr =
+ output.sw_owner_icm_root_0;
+ } else {
+ mlx5dr_dbg(dmn,
+ "Failed mlx5_cmd_query_flow_table ret: %d\n",
+ ret);
+ return ret;
+ }
+ }
+ attr.final_icm_addr = rx_rule ?
+ action->dest_tbl.fw_tbl.rx_icm_addr :
+ action->dest_tbl.fw_tbl.tx_icm_addr;
+ }
+ break;
+ case DR_ACTION_TYP_QP:
+ mlx5dr_info(dmn, "Domain doesn't support QP\n");
+ goto out_invalid_arg;
+ case DR_ACTION_TYP_CTR:
+ attr.ctr_id = action->ctr.ctr_id +
+ action->ctr.offeset;
+ break;
+ case DR_ACTION_TYP_TAG:
+ attr.flow_tag = action->flow_tag;
+ break;
+ case DR_ACTION_TYP_TNL_L2_TO_L2:
+ break;
+ case DR_ACTION_TYP_TNL_L3_TO_L2:
+ attr.decap_index = action->rewrite.index;
+ attr.decap_actions = action->rewrite.num_of_actions;
+ attr.decap_with_vlan =
+ attr.decap_actions == WITH_VLAN_NUM_HW_ACTIONS;
+ break;
+ case DR_ACTION_TYP_MODIFY_HDR:
+ attr.modify_index = action->rewrite.index;
+ attr.modify_actions = action->rewrite.num_of_actions;
+ recalc_cs_required = action->rewrite.modify_ttl;
+ break;
+ case DR_ACTION_TYP_L2_TO_TNL_L2:
+ case DR_ACTION_TYP_L2_TO_TNL_L3:
+ attr.reformat_size = action->reformat.reformat_size;
+ attr.reformat_id = action->reformat.reformat_id;
+ break;
+ case DR_ACTION_TYP_VPORT:
+ attr.hit_gvmi = action->vport.caps->vhca_gvmi;
+ dest_action = action;
+ if (rx_rule) {
+ /* Loopback on WIRE vport is not supported */
+ if (action->vport.num == WIRE_PORT)
+ goto out_invalid_arg;
+
+ attr.final_icm_addr = action->vport.caps->icm_address_rx;
+ } else {
+ attr.final_icm_addr = action->vport.caps->icm_address_tx;
+ }
+ break;
+ case DR_ACTION_TYP_POP_VLAN:
+ max_actions_type = MAX_VLANS;
+ attr.vlans.count++;
+ break;
+ case DR_ACTION_TYP_PUSH_VLAN:
+ max_actions_type = MAX_VLANS;
+ if (attr.vlans.count == MAX_VLANS)
+ return -EINVAL;
+
+ attr.vlans.headers[attr.vlans.count++] = action->push_vlan.vlan_hdr;
+ break;
+ default:
+ goto out_invalid_arg;
+ }
+
+ /* Check action duplication */
+ if (++action_type_set[action_type] > max_actions_type) {
+ mlx5dr_dbg(dmn, "Action type %d supports only max %d time(s)\n",
+ action_type, max_actions_type);
+ goto out_invalid_arg;
+ }
+
+ /* Check action state machine is valid */
+ if (dr_action_validate_and_get_next_state(action_domain,
+ action_type,
+ &state)) {
+ mlx5dr_dbg(dmn, "Invalid action sequence provided\n");
+ return -EOPNOTSUPP;
+ }
+ }
+
+ *new_hw_ste_arr_sz = nic_matcher->num_of_builders;
+ last_ste = ste_arr + DR_STE_SIZE * (nic_matcher->num_of_builders - 1);
+
+ /* Due to a HW bug, modifying TTL on RX flows will cause an incorrect
+ * checksum calculation. In this case we will use a FW table to
+ * recalculate.
+ */
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB &&
+ rx_rule && recalc_cs_required && dest_action) {
+ ret = dr_action_handle_cs_recalc(dmn, dest_action, &attr.final_icm_addr);
+ if (ret) {
+ mlx5dr_dbg(dmn,
+ "Failed to handle checksum recalculation err %d\n",
+ ret);
+ return ret;
+ }
+ }
+
+ dr_actions_apply(dmn,
+ nic_dmn->ste_type,
+ action_type_set,
+ last_ste,
+ &attr,
+ new_hw_ste_arr_sz);
+
+ return 0;
+
+out_invalid_arg:
+ return -EINVAL;
+}
+
+#define CVLAN_ETHERTYPE 0x8100
+#define SVLAN_ETHERTYPE 0x88a8
+#define HDR_LEN_L2_ONLY 14
+#define HDR_LEN_L2_VLAN 18
+#define REWRITE_HW_ACTION_NUM 6
+
+static int dr_actions_l2_rewrite(struct mlx5dr_domain *dmn,
+ struct mlx5dr_action *action,
+ void *data, size_t data_sz)
+{
+ struct mlx5_ifc_l2_hdr_bits *l2_hdr = data;
+ u64 ops[REWRITE_HW_ACTION_NUM] = {};
+ u32 hdr_fld_4b;
+ u16 hdr_fld_2b;
+ u16 vlan_type;
+ bool vlan;
+ int i = 0;
+ int ret;
+
+ vlan = (data_sz != HDR_LEN_L2_ONLY);
+
+ /* dmac_47_16 */
+ MLX5_SET(dr_action_hw_set, ops + i,
+ opcode, MLX5DR_ACTION_MDFY_HW_OP_SET);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_length, 0);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_field_code, MLX5DR_ACTION_MDFY_HW_FLD_L2_0);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_left_shifter, 16);
+ hdr_fld_4b = MLX5_GET(l2_hdr, l2_hdr, dmac_47_16);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ inline_data, hdr_fld_4b);
+ i++;
+
+ /* smac_47_16 */
+ MLX5_SET(dr_action_hw_set, ops + i,
+ opcode, MLX5DR_ACTION_MDFY_HW_OP_SET);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_length, 0);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_field_code, MLX5DR_ACTION_MDFY_HW_FLD_L2_1);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_left_shifter, 16);
+ hdr_fld_4b = (MLX5_GET(l2_hdr, l2_hdr, smac_31_0) >> 16 |
+ MLX5_GET(l2_hdr, l2_hdr, smac_47_32) << 16);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ inline_data, hdr_fld_4b);
+ i++;
+
+ /* dmac_15_0 */
+ MLX5_SET(dr_action_hw_set, ops + i,
+ opcode, MLX5DR_ACTION_MDFY_HW_OP_SET);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_length, 16);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_field_code, MLX5DR_ACTION_MDFY_HW_FLD_L2_0);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_left_shifter, 0);
+ hdr_fld_2b = MLX5_GET(l2_hdr, l2_hdr, dmac_15_0);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ inline_data, hdr_fld_2b);
+ i++;
+
+ /* ethertype + (optional) vlan */
+ MLX5_SET(dr_action_hw_set, ops + i,
+ opcode, MLX5DR_ACTION_MDFY_HW_OP_SET);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_field_code, MLX5DR_ACTION_MDFY_HW_FLD_L2_2);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_left_shifter, 32);
+ if (!vlan) {
+ hdr_fld_2b = MLX5_GET(l2_hdr, l2_hdr, ethertype);
+ MLX5_SET(dr_action_hw_set, ops + i, inline_data, hdr_fld_2b);
+ MLX5_SET(dr_action_hw_set, ops + i, destination_length, 16);
+ } else {
+ hdr_fld_2b = MLX5_GET(l2_hdr, l2_hdr, ethertype);
+ vlan_type = hdr_fld_2b == SVLAN_ETHERTYPE ? DR_STE_SVLAN : DR_STE_CVLAN;
+ hdr_fld_2b = MLX5_GET(l2_hdr, l2_hdr, vlan);
+ hdr_fld_4b = (vlan_type << 16) | hdr_fld_2b;
+ MLX5_SET(dr_action_hw_set, ops + i, inline_data, hdr_fld_4b);
+ MLX5_SET(dr_action_hw_set, ops + i, destination_length, 18);
+ }
+ i++;
+
+ /* smac_15_0 */
+ MLX5_SET(dr_action_hw_set, ops + i,
+ opcode, MLX5DR_ACTION_MDFY_HW_OP_SET);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_length, 16);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_field_code, MLX5DR_ACTION_MDFY_HW_FLD_L2_1);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_left_shifter, 0);
+ hdr_fld_2b = MLX5_GET(l2_hdr, l2_hdr, smac_31_0);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ inline_data, hdr_fld_2b);
+ i++;
+
+ if (vlan) {
+ MLX5_SET(dr_action_hw_set, ops + i,
+ opcode, MLX5DR_ACTION_MDFY_HW_OP_SET);
+ hdr_fld_2b = MLX5_GET(l2_hdr, l2_hdr, vlan_type);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ inline_data, hdr_fld_2b);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_length, 16);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_field_code, MLX5DR_ACTION_MDFY_HW_FLD_L2_2);
+ MLX5_SET(dr_action_hw_set, ops + i,
+ destination_left_shifter, 0);
+ i++;
+ }
+
+ action->rewrite.data = (void *)ops;
+ action->rewrite.num_of_actions = i;
+ action->rewrite.chunk->byte_size = i * sizeof(*ops);
+
+ ret = mlx5dr_send_postsend_action(dmn, action);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Writing encapsulation action to ICM failed\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static struct mlx5dr_action *
+dr_action_create_generic(enum mlx5dr_action_type action_type)
+{
+ struct mlx5dr_action *action;
+
+ action = kzalloc(sizeof(*action), GFP_KERNEL);
+ if (!action)
+ return NULL;
+
+ action->action_type = action_type;
+ refcount_set(&action->refcount, 1);
+
+ return action;
+}
+
+struct mlx5dr_action *mlx5dr_action_create_drop(void)
+{
+ return dr_action_create_generic(DR_ACTION_TYP_DROP);
+}
+
+struct mlx5dr_action *
+mlx5dr_action_create_dest_table(struct mlx5dr_table *tbl)
+{
+ struct mlx5dr_action *action;
+
+ refcount_inc(&tbl->refcount);
+
+ action = dr_action_create_generic(DR_ACTION_TYP_FT);
+ if (!action)
+ goto dec_ref;
+
+ action->dest_tbl.tbl = tbl;
+
+ return action;
+
+dec_ref:
+ refcount_dec(&tbl->refcount);
+ return NULL;
+}
+
+struct mlx5dr_action *
+mlx5dr_create_action_dest_flow_fw_table(struct mlx5_flow_table *ft,
+ struct mlx5_core_dev *mdev)
+{
+ struct mlx5dr_action *action;
+
+ action = dr_action_create_generic(DR_ACTION_TYP_FT);
+ if (!action)
+ return NULL;
+
+ action->dest_tbl.is_fw_tbl = 1;
+ action->dest_tbl.fw_tbl.ft = ft;
+ action->dest_tbl.fw_tbl.mdev = mdev;
+
+ return action;
+}
+
+struct mlx5dr_action *
+mlx5dr_action_create_flow_counter(u32 counter_id)
+{
+ struct mlx5dr_action *action;
+
+ action = dr_action_create_generic(DR_ACTION_TYP_CTR);
+ if (!action)
+ return NULL;
+
+ action->ctr.ctr_id = counter_id;
+
+ return action;
+}
+
+struct mlx5dr_action *mlx5dr_action_create_tag(u32 tag_value)
+{
+ struct mlx5dr_action *action;
+
+ action = dr_action_create_generic(DR_ACTION_TYP_TAG);
+ if (!action)
+ return NULL;
+
+ action->flow_tag = tag_value & 0xffffff;
+
+ return action;
+}
+
+static int
+dr_action_verify_reformat_params(enum mlx5dr_action_type reformat_type,
+ struct mlx5dr_domain *dmn,
+ size_t data_sz,
+ void *data)
+{
+ if ((!data && data_sz) || (data && !data_sz) || reformat_type >
+ DR_ACTION_TYP_L2_TO_TNL_L3) {
+ mlx5dr_dbg(dmn, "Invalid reformat parameter!\n");
+ goto out_err;
+ }
+
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB)
+ return 0;
+
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX) {
+ if (reformat_type != DR_ACTION_TYP_TNL_L2_TO_L2 &&
+ reformat_type != DR_ACTION_TYP_TNL_L3_TO_L2) {
+ mlx5dr_dbg(dmn, "Action reformat type not support on RX domain\n");
+ goto out_err;
+ }
+ } else if (dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX) {
+ if (reformat_type != DR_ACTION_TYP_L2_TO_TNL_L2 &&
+ reformat_type != DR_ACTION_TYP_L2_TO_TNL_L3) {
+ mlx5dr_dbg(dmn, "Action reformat type not support on TX domain\n");
+ goto out_err;
+ }
+ }
+
+ return 0;
+
+out_err:
+ return -EINVAL;
+}
+
+#define ACTION_CACHE_LINE_SIZE 64
+
+static int
+dr_action_create_reformat_action(struct mlx5dr_domain *dmn,
+ size_t data_sz, void *data,
+ struct mlx5dr_action *action)
+{
+ u32 reformat_id;
+ int ret;
+
+ switch (action->action_type) {
+ case DR_ACTION_TYP_L2_TO_TNL_L2:
+ case DR_ACTION_TYP_L2_TO_TNL_L3:
+ {
+ enum mlx5_reformat_ctx_type rt;
+
+ if (action->action_type == DR_ACTION_TYP_L2_TO_TNL_L2)
+ rt = MLX5_REFORMAT_TYPE_L2_TO_L2_TUNNEL;
+ else
+ rt = MLX5_REFORMAT_TYPE_L2_TO_L3_TUNNEL;
+
+ ret = mlx5dr_cmd_create_reformat_ctx(dmn->mdev, rt, data_sz, data,
+ &reformat_id);
+ if (ret)
+ return ret;
+
+ action->reformat.reformat_id = reformat_id;
+ action->reformat.reformat_size = data_sz;
+ return 0;
+ }
+ case DR_ACTION_TYP_TNL_L2_TO_L2:
+ {
+ return 0;
+ }
+ case DR_ACTION_TYP_TNL_L3_TO_L2:
+ {
+ /* Only Ethernet frame is supported, with VLAN (18) or without (14) */
+ if (data_sz != HDR_LEN_L2_ONLY && data_sz != HDR_LEN_L2_VLAN)
+ return -EINVAL;
+
+ action->rewrite.chunk = mlx5dr_icm_alloc_chunk(dmn->action_icm_pool,
+ DR_CHUNK_SIZE_8);
+ if (!action->rewrite.chunk)
+ return -ENOMEM;
+
+ action->rewrite.index = (action->rewrite.chunk->icm_addr -
+ dmn->info.caps.hdr_modify_icm_addr) /
+ ACTION_CACHE_LINE_SIZE;
+
+ ret = dr_actions_l2_rewrite(dmn, action, data, data_sz);
+ if (ret) {
+ mlx5dr_icm_free_chunk(action->rewrite.chunk);
+ return ret;
+ }
+ return 0;
+ }
+ default:
+ mlx5dr_info(dmn, "Reformat type is not supported %d\n", action->action_type);
+ return -EINVAL;
+ }
+}
+
+struct mlx5dr_action *mlx5dr_action_create_pop_vlan(void)
+{
+ return dr_action_create_generic(DR_ACTION_TYP_POP_VLAN);
+}
+
+struct mlx5dr_action *mlx5dr_action_create_push_vlan(struct mlx5dr_domain *dmn,
+ __be32 vlan_hdr)
+{
+ u32 vlan_hdr_h = ntohl(vlan_hdr);
+ u16 ethertype = vlan_hdr_h >> 16;
+ struct mlx5dr_action *action;
+
+ if (ethertype != SVLAN_ETHERTYPE && ethertype != CVLAN_ETHERTYPE) {
+ mlx5dr_dbg(dmn, "Invalid vlan ethertype\n");
+ return NULL;
+ }
+
+ action = dr_action_create_generic(DR_ACTION_TYP_PUSH_VLAN);
+ if (!action)
+ return NULL;
+
+ action->push_vlan.vlan_hdr = vlan_hdr_h;
+ return action;
+}
+
+struct mlx5dr_action *
+mlx5dr_action_create_packet_reformat(struct mlx5dr_domain *dmn,
+ enum mlx5dr_action_reformat_type reformat_type,
+ size_t data_sz,
+ void *data)
+{
+ enum mlx5dr_action_type action_type;
+ struct mlx5dr_action *action;
+ int ret;
+
+ refcount_inc(&dmn->refcount);
+
+ /* General checks */
+ ret = dr_action_reformat_to_action_type(reformat_type, &action_type);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Invalid reformat_type provided\n");
+ goto dec_ref;
+ }
+
+ ret = dr_action_verify_reformat_params(action_type, dmn, data_sz, data);
+ if (ret)
+ goto dec_ref;
+
+ action = dr_action_create_generic(action_type);
+ if (!action)
+ goto dec_ref;
+
+ action->reformat.dmn = dmn;
+
+ ret = dr_action_create_reformat_action(dmn,
+ data_sz,
+ data,
+ action);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Failed creating reformat action %d\n", ret);
+ goto free_action;
+ }
+
+ return action;
+
+free_action:
+ kfree(action);
+dec_ref:
+ refcount_dec(&dmn->refcount);
+ return NULL;
+}
+
+static const struct dr_action_modify_field_conv *
+dr_action_modify_get_hw_info(u16 sw_field)
+{
+ const struct dr_action_modify_field_conv *hw_action_info;
+
+ if (sw_field >= ARRAY_SIZE(dr_action_conv_arr))
+ goto not_found;
+
+ hw_action_info = &dr_action_conv_arr[sw_field];
+ if (!hw_action_info->end && !hw_action_info->start)
+ goto not_found;
+
+ return hw_action_info;
+
+not_found:
+ return NULL;
+}
+
+static int
+dr_action_modify_sw_to_hw(struct mlx5dr_domain *dmn,
+ __be64 *sw_action,
+ __be64 *hw_action,
+ const struct dr_action_modify_field_conv **ret_hw_info)
+{
+ const struct dr_action_modify_field_conv *hw_action_info;
+ u8 offset, length, max_length, action;
+ u16 sw_field;
+ u8 hw_opcode;
+ u32 data;
+
+ /* Get SW modify action data */
+ action = MLX5_GET(set_action_in, sw_action, action_type);
+ length = MLX5_GET(set_action_in, sw_action, length);
+ offset = MLX5_GET(set_action_in, sw_action, offset);
+ sw_field = MLX5_GET(set_action_in, sw_action, field);
+ data = MLX5_GET(set_action_in, sw_action, data);
+
+ /* Convert SW data to HW modify action format */
+ hw_action_info = dr_action_modify_get_hw_info(sw_field);
+ if (!hw_action_info) {
+ mlx5dr_dbg(dmn, "Modify action invalid field given\n");
+ return -EINVAL;
+ }
+
+ max_length = hw_action_info->end - hw_action_info->start + 1;
+
+ switch (action) {
+ case MLX5_ACTION_TYPE_SET:
+ hw_opcode = MLX5DR_ACTION_MDFY_HW_OP_SET;
+ /* PRM defines that length zero specific length of 32bits */
+ if (!length)
+ length = 32;
+
+ if (length + offset > max_length) {
+ mlx5dr_dbg(dmn, "Modify action length + offset exceeds limit\n");
+ return -EINVAL;
+ }
+ break;
+
+ case MLX5_ACTION_TYPE_ADD:
+ hw_opcode = MLX5DR_ACTION_MDFY_HW_OP_ADD;
+ offset = 0;
+ length = max_length;
+ break;
+
+ default:
+ mlx5dr_info(dmn, "Unsupported action_type for modify action\n");
+ return -EOPNOTSUPP;
+ }
+
+ MLX5_SET(dr_action_hw_set, hw_action, opcode, hw_opcode);
+
+ MLX5_SET(dr_action_hw_set, hw_action, destination_field_code,
+ hw_action_info->hw_field);
+
+ MLX5_SET(dr_action_hw_set, hw_action, destination_left_shifter,
+ hw_action_info->start + offset);
+
+ MLX5_SET(dr_action_hw_set, hw_action, destination_length,
+ length == 32 ? 0 : length);
+
+ MLX5_SET(dr_action_hw_set, hw_action, inline_data, data);
+
+ *ret_hw_info = hw_action_info;
+
+ return 0;
+}
+
+static int
+dr_action_modify_check_field_limitation(struct mlx5dr_domain *dmn,
+ const __be64 *sw_action)
+{
+ u16 sw_field;
+ u8 action;
+
+ sw_field = MLX5_GET(set_action_in, sw_action, field);
+ action = MLX5_GET(set_action_in, sw_action, action_type);
+
+ /* Check if SW field is supported in current domain (RX/TX) */
+ if (action == MLX5_ACTION_TYPE_SET) {
+ if (sw_field == MLX5_ACTION_IN_FIELD_METADATA_REG_A) {
+ if (dmn->type != MLX5DR_DOMAIN_TYPE_NIC_TX) {
+ mlx5dr_dbg(dmn, "Unsupported field %d for RX/FDB set action\n",
+ sw_field);
+ return -EINVAL;
+ }
+ }
+
+ if (sw_field == MLX5_ACTION_IN_FIELD_METADATA_REG_B) {
+ if (dmn->type != MLX5DR_DOMAIN_TYPE_NIC_RX) {
+ mlx5dr_dbg(dmn, "Unsupported field %d for TX/FDB set action\n",
+ sw_field);
+ return -EINVAL;
+ }
+ }
+ } else if (action == MLX5_ACTION_TYPE_ADD) {
+ if (sw_field != MLX5_ACTION_IN_FIELD_OUT_IP_TTL &&
+ sw_field != MLX5_ACTION_IN_FIELD_OUT_IPV6_HOPLIMIT &&
+ sw_field != MLX5_ACTION_IN_FIELD_OUT_TCP_SEQ_NUM &&
+ sw_field != MLX5_ACTION_IN_FIELD_OUT_TCP_ACK_NUM) {
+ mlx5dr_dbg(dmn, "Unsupported field %d for add action\n", sw_field);
+ return -EINVAL;
+ }
+ } else {
+ mlx5dr_info(dmn, "Unsupported action %d modify action\n", action);
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static bool
+dr_action_modify_check_is_ttl_modify(const u64 *sw_action)
+{
+ u16 sw_field = MLX5_GET(set_action_in, sw_action, field);
+
+ return sw_field == MLX5_ACTION_IN_FIELD_OUT_IP_TTL;
+}
+
+static int dr_actions_convert_modify_header(struct mlx5dr_domain *dmn,
+ u32 max_hw_actions,
+ u32 num_sw_actions,
+ __be64 sw_actions[],
+ __be64 hw_actions[],
+ u32 *num_hw_actions,
+ bool *modify_ttl)
+{
+ const struct dr_action_modify_field_conv *hw_action_info;
+ u16 hw_field = MLX5DR_ACTION_MDFY_HW_FLD_RESERVED;
+ u32 l3_type = MLX5DR_ACTION_MDFY_HW_HDR_L3_NONE;
+ u32 l4_type = MLX5DR_ACTION_MDFY_HW_HDR_L4_NONE;
+ int ret, i, hw_idx = 0;
+ __be64 *sw_action;
+ __be64 hw_action;
+
+ *modify_ttl = false;
+
+ for (i = 0; i < num_sw_actions; i++) {
+ sw_action = &sw_actions[i];
+
+ ret = dr_action_modify_check_field_limitation(dmn, sw_action);
+ if (ret)
+ return ret;
+
+ if (!(*modify_ttl))
+ *modify_ttl = dr_action_modify_check_is_ttl_modify(sw_action);
+
+ /* Convert SW action to HW action */
+ ret = dr_action_modify_sw_to_hw(dmn,
+ sw_action,
+ &hw_action,
+ &hw_action_info);
+ if (ret)
+ return ret;
+
+ /* Due to a HW limitation we cannot modify 2 different L3 types */
+ if (l3_type && hw_action_info->l3_type &&
+ hw_action_info->l3_type != l3_type) {
+ mlx5dr_dbg(dmn, "Action list can't support two different L3 types\n");
+ return -EINVAL;
+ }
+ if (hw_action_info->l3_type)
+ l3_type = hw_action_info->l3_type;
+
+ /* Due to a HW limitation we cannot modify two different L4 types */
+ if (l4_type && hw_action_info->l4_type &&
+ hw_action_info->l4_type != l4_type) {
+ mlx5dr_dbg(dmn, "Action list can't support two different L4 types\n");
+ return -EINVAL;
+ }
+ if (hw_action_info->l4_type)
+ l4_type = hw_action_info->l4_type;
+
+ /* HW reads and executes two actions at once this means we
+ * need to create a gap if two actions access the same field
+ */
+ if ((hw_idx % 2) && hw_field == hw_action_info->hw_field) {
+ /* Check if after gap insertion the total number of HW
+ * modify actions doesn't exceeds the limit
+ */
+ hw_idx++;
+ if ((num_sw_actions + hw_idx - i) >= max_hw_actions) {
+ mlx5dr_dbg(dmn, "Modify header action number exceeds HW limit\n");
+ return -EINVAL;
+ }
+ }
+ hw_field = hw_action_info->hw_field;
+
+ hw_actions[hw_idx] = hw_action;
+ hw_idx++;
+ }
+
+ *num_hw_actions = hw_idx;
+
+ return 0;
+}
+
+static int dr_action_create_modify_action(struct mlx5dr_domain *dmn,
+ size_t actions_sz,
+ __be64 actions[],
+ struct mlx5dr_action *action)
+{
+ struct mlx5dr_icm_chunk *chunk;
+ u32 max_hw_actions;
+ u32 num_hw_actions;
+ u32 num_sw_actions;
+ __be64 *hw_actions;
+ bool modify_ttl;
+ int ret;
+
+ num_sw_actions = actions_sz / DR_MODIFY_ACTION_SIZE;
+ max_hw_actions = mlx5dr_icm_pool_chunk_size_to_entries(DR_CHUNK_SIZE_16);
+
+ if (num_sw_actions > max_hw_actions) {
+ mlx5dr_dbg(dmn, "Max number of actions %d exceeds limit %d\n",
+ num_sw_actions, max_hw_actions);
+ return -EINVAL;
+ }
+
+ chunk = mlx5dr_icm_alloc_chunk(dmn->action_icm_pool, DR_CHUNK_SIZE_16);
+ if (!chunk)
+ return -ENOMEM;
+
+ hw_actions = kcalloc(1, max_hw_actions * DR_MODIFY_ACTION_SIZE, GFP_KERNEL);
+ if (!hw_actions) {
+ ret = -ENOMEM;
+ goto free_chunk;
+ }
+
+ ret = dr_actions_convert_modify_header(dmn,
+ max_hw_actions,
+ num_sw_actions,
+ actions,
+ hw_actions,
+ &num_hw_actions,
+ &modify_ttl);
+ if (ret)
+ goto free_hw_actions;
+
+ action->rewrite.chunk = chunk;
+ action->rewrite.modify_ttl = modify_ttl;
+ action->rewrite.data = (u8 *)hw_actions;
+ action->rewrite.num_of_actions = num_hw_actions;
+ action->rewrite.index = (chunk->icm_addr -
+ dmn->info.caps.hdr_modify_icm_addr) /
+ ACTION_CACHE_LINE_SIZE;
+
+ ret = mlx5dr_send_postsend_action(dmn, action);
+ if (ret)
+ goto free_hw_actions;
+
+ return 0;
+
+free_hw_actions:
+ kfree(hw_actions);
+free_chunk:
+ mlx5dr_icm_free_chunk(chunk);
+ return ret;
+}
+
+struct mlx5dr_action *
+mlx5dr_action_create_modify_header(struct mlx5dr_domain *dmn,
+ u32 flags,
+ size_t actions_sz,
+ __be64 actions[])
+{
+ struct mlx5dr_action *action;
+ int ret = 0;
+
+ refcount_inc(&dmn->refcount);
+
+ if (actions_sz % DR_MODIFY_ACTION_SIZE) {
+ mlx5dr_dbg(dmn, "Invalid modify actions size provided\n");
+ goto dec_ref;
+ }
+
+ action = dr_action_create_generic(DR_ACTION_TYP_MODIFY_HDR);
+ if (!action)
+ goto dec_ref;
+
+ action->rewrite.dmn = dmn;
+
+ ret = dr_action_create_modify_action(dmn,
+ actions_sz,
+ actions,
+ action);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Failed creating modify header action %d\n", ret);
+ goto free_action;
+ }
+
+ return action;
+
+free_action:
+ kfree(action);
+dec_ref:
+ refcount_dec(&dmn->refcount);
+ return NULL;
+}
+
+struct mlx5dr_action *
+mlx5dr_action_create_dest_vport(struct mlx5dr_domain *dmn,
+ u32 vport, u8 vhca_id_valid,
+ u16 vhca_id)
+{
+ struct mlx5dr_cmd_vport_cap *vport_cap;
+ struct mlx5dr_domain *vport_dmn;
+ struct mlx5dr_action *action;
+ u8 peer_vport;
+
+ peer_vport = vhca_id_valid && (vhca_id != dmn->info.caps.gvmi);
+ vport_dmn = peer_vport ? dmn->peer_dmn : dmn;
+ if (!vport_dmn) {
+ mlx5dr_dbg(dmn, "No peer vport domain for given vhca_id\n");
+ return NULL;
+ }
+
+ if (vport_dmn->type != MLX5DR_DOMAIN_TYPE_FDB) {
+ mlx5dr_dbg(dmn, "Domain doesn't support vport actions\n");
+ return NULL;
+ }
+
+ vport_cap = mlx5dr_get_vport_cap(&vport_dmn->info.caps, vport);
+ if (!vport_cap) {
+ mlx5dr_dbg(dmn, "Failed to get vport %d caps\n", vport);
+ return NULL;
+ }
+
+ action = dr_action_create_generic(DR_ACTION_TYP_VPORT);
+ if (!action)
+ return NULL;
+
+ action->vport.dmn = vport_dmn;
+ action->vport.caps = vport_cap;
+
+ return action;
+}
+
+int mlx5dr_action_destroy(struct mlx5dr_action *action)
+{
+ if (refcount_read(&action->refcount) > 1)
+ return -EBUSY;
+
+ switch (action->action_type) {
+ case DR_ACTION_TYP_FT:
+ if (!action->dest_tbl.is_fw_tbl)
+ refcount_dec(&action->dest_tbl.tbl->refcount);
+ break;
+ case DR_ACTION_TYP_TNL_L2_TO_L2:
+ refcount_dec(&action->reformat.dmn->refcount);
+ break;
+ case DR_ACTION_TYP_TNL_L3_TO_L2:
+ mlx5dr_icm_free_chunk(action->rewrite.chunk);
+ refcount_dec(&action->reformat.dmn->refcount);
+ break;
+ case DR_ACTION_TYP_L2_TO_TNL_L2:
+ case DR_ACTION_TYP_L2_TO_TNL_L3:
+ mlx5dr_cmd_destroy_reformat_ctx((action->reformat.dmn)->mdev,
+ action->reformat.reformat_id);
+ refcount_dec(&action->reformat.dmn->refcount);
+ break;
+ case DR_ACTION_TYP_MODIFY_HDR:
+ mlx5dr_icm_free_chunk(action->rewrite.chunk);
+ refcount_dec(&action->rewrite.dmn->refcount);
+ break;
+ default:
+ break;
+ }
+
+ kfree(action);
+ return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
new file mode 100644
index 0000000..41662c4e
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c
@@ -0,0 +1,480 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+int mlx5dr_cmd_query_esw_vport_context(struct mlx5_core_dev *mdev,
+ bool other_vport,
+ u16 vport_number,
+ u64 *icm_address_rx,
+ u64 *icm_address_tx)
+{
+ u32 out[MLX5_ST_SZ_DW(query_esw_vport_context_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(query_esw_vport_context_in)] = {};
+ int err;
+
+ MLX5_SET(query_esw_vport_context_in, in, opcode,
+ MLX5_CMD_OP_QUERY_ESW_VPORT_CONTEXT);
+ MLX5_SET(query_esw_vport_context_in, in, other_vport, other_vport);
+ MLX5_SET(query_esw_vport_context_in, in, vport_number, vport_number);
+
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+ if (err)
+ return err;
+
+ *icm_address_rx =
+ MLX5_GET64(query_esw_vport_context_out, out,
+ esw_vport_context.sw_steering_vport_icm_address_rx);
+ *icm_address_tx =
+ MLX5_GET64(query_esw_vport_context_out, out,
+ esw_vport_context.sw_steering_vport_icm_address_tx);
+ return 0;
+}
+
+int mlx5dr_cmd_query_gvmi(struct mlx5_core_dev *mdev, bool other_vport,
+ u16 vport_number, u16 *gvmi)
+{
+ u32 in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {};
+ int out_size;
+ void *out;
+ int err;
+
+ out_size = MLX5_ST_SZ_BYTES(query_hca_cap_out);
+ out = kzalloc(out_size, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
+ MLX5_SET(query_hca_cap_in, in, other_function, other_vport);
+ MLX5_SET(query_hca_cap_in, in, function_id, vport_number);
+ MLX5_SET(query_hca_cap_in, in, op_mod,
+ MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE << 1 |
+ HCA_CAP_OPMOD_GET_CUR);
+
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, out_size);
+ if (err) {
+ kfree(out);
+ return err;
+ }
+
+ *gvmi = MLX5_GET(query_hca_cap_out, out, capability.cmd_hca_cap.vhca_id);
+
+ kfree(out);
+ return 0;
+}
+
+int mlx5dr_cmd_query_esw_caps(struct mlx5_core_dev *mdev,
+ struct mlx5dr_esw_caps *caps)
+{
+ caps->drop_icm_address_rx =
+ MLX5_CAP64_ESW_FLOWTABLE(mdev,
+ sw_steering_fdb_action_drop_icm_address_rx);
+ caps->drop_icm_address_tx =
+ MLX5_CAP64_ESW_FLOWTABLE(mdev,
+ sw_steering_fdb_action_drop_icm_address_tx);
+ caps->uplink_icm_address_rx =
+ MLX5_CAP64_ESW_FLOWTABLE(mdev,
+ sw_steering_uplink_icm_address_rx);
+ caps->uplink_icm_address_tx =
+ MLX5_CAP64_ESW_FLOWTABLE(mdev,
+ sw_steering_uplink_icm_address_tx);
+ caps->sw_owner =
+ MLX5_CAP_ESW_FLOWTABLE_FDB(mdev,
+ sw_owner);
+
+ return 0;
+}
+
+int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
+ struct mlx5dr_cmd_caps *caps)
+{
+ caps->prio_tag_required = MLX5_CAP_GEN(mdev, prio_tag_required);
+ caps->eswitch_manager = MLX5_CAP_GEN(mdev, eswitch_manager);
+ caps->gvmi = MLX5_CAP_GEN(mdev, vhca_id);
+ caps->flex_protocols = MLX5_CAP_GEN(mdev, flex_parser_protocols);
+
+ if (mlx5dr_matcher_supp_flex_parser_icmp_v4(caps)) {
+ caps->flex_parser_id_icmp_dw0 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw0);
+ caps->flex_parser_id_icmp_dw1 = MLX5_CAP_GEN(mdev, flex_parser_id_icmp_dw1);
+ }
+
+ if (mlx5dr_matcher_supp_flex_parser_icmp_v6(caps)) {
+ caps->flex_parser_id_icmpv6_dw0 =
+ MLX5_CAP_GEN(mdev, flex_parser_id_icmpv6_dw0);
+ caps->flex_parser_id_icmpv6_dw1 =
+ MLX5_CAP_GEN(mdev, flex_parser_id_icmpv6_dw1);
+ }
+
+ caps->nic_rx_drop_address =
+ MLX5_CAP64_FLOWTABLE(mdev, sw_steering_nic_rx_action_drop_icm_address);
+ caps->nic_tx_drop_address =
+ MLX5_CAP64_FLOWTABLE(mdev, sw_steering_nic_tx_action_drop_icm_address);
+ caps->nic_tx_allow_address =
+ MLX5_CAP64_FLOWTABLE(mdev, sw_steering_nic_tx_action_allow_icm_address);
+
+ caps->rx_sw_owner = MLX5_CAP_FLOWTABLE_NIC_RX(mdev, sw_owner);
+ caps->max_ft_level = MLX5_CAP_FLOWTABLE_NIC_RX(mdev, max_ft_level);
+
+ caps->tx_sw_owner = MLX5_CAP_FLOWTABLE_NIC_TX(mdev, sw_owner);
+
+ caps->log_icm_size = MLX5_CAP_DEV_MEM(mdev, log_steering_sw_icm_size);
+ caps->hdr_modify_icm_addr =
+ MLX5_CAP64_DEV_MEM(mdev, header_modify_sw_icm_start_address);
+
+ caps->roce_min_src_udp = MLX5_CAP_ROCE(mdev, r_roce_min_src_udp_port);
+
+ return 0;
+}
+
+int mlx5dr_cmd_query_flow_table(struct mlx5_core_dev *dev,
+ enum fs_flow_table_type type,
+ u32 table_id,
+ struct mlx5dr_cmd_query_flow_table_details *output)
+{
+ u32 out[MLX5_ST_SZ_DW(query_flow_table_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(query_flow_table_in)] = {};
+ int err;
+
+ MLX5_SET(query_flow_table_in, in, opcode,
+ MLX5_CMD_OP_QUERY_FLOW_TABLE);
+
+ MLX5_SET(query_flow_table_in, in, table_type, type);
+ MLX5_SET(query_flow_table_in, in, table_id, table_id);
+
+ err = mlx5_cmd_exec(dev, in, sizeof(in), out, sizeof(out));
+ if (err)
+ return err;
+
+ output->status = MLX5_GET(query_flow_table_out, out, status);
+ output->level = MLX5_GET(query_flow_table_out, out, flow_table_context.level);
+
+ output->sw_owner_icm_root_1 = MLX5_GET64(query_flow_table_out, out,
+ flow_table_context.sw_owner_icm_root_1);
+ output->sw_owner_icm_root_0 = MLX5_GET64(query_flow_table_out, out,
+ flow_table_context.sw_owner_icm_root_0);
+
+ return 0;
+}
+
+int mlx5dr_cmd_sync_steering(struct mlx5_core_dev *mdev)
+{
+ u32 out[MLX5_ST_SZ_DW(sync_steering_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(sync_steering_in)] = {};
+
+ MLX5_SET(sync_steering_in, in, opcode, MLX5_CMD_OP_SYNC_STEERING);
+
+ return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+int mlx5dr_cmd_set_fte_modify_and_vport(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id,
+ u32 group_id,
+ u32 modify_header_id,
+ u32 vport_id)
+{
+ u32 out[MLX5_ST_SZ_DW(set_fte_out)] = {};
+ void *in_flow_context;
+ unsigned int inlen;
+ void *in_dests;
+ u32 *in;
+ int err;
+
+ inlen = MLX5_ST_SZ_BYTES(set_fte_in) +
+ 1 * MLX5_ST_SZ_BYTES(dest_format_struct); /* One destination only */
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ MLX5_SET(set_fte_in, in, opcode, MLX5_CMD_OP_SET_FLOW_TABLE_ENTRY);
+ MLX5_SET(set_fte_in, in, table_type, table_type);
+ MLX5_SET(set_fte_in, in, table_id, table_id);
+
+ in_flow_context = MLX5_ADDR_OF(set_fte_in, in, flow_context);
+ MLX5_SET(flow_context, in_flow_context, group_id, group_id);
+ MLX5_SET(flow_context, in_flow_context, modify_header_id, modify_header_id);
+ MLX5_SET(flow_context, in_flow_context, destination_list_size, 1);
+ MLX5_SET(flow_context, in_flow_context, action,
+ MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_MOD_HDR);
+
+ in_dests = MLX5_ADDR_OF(flow_context, in_flow_context, destination);
+ MLX5_SET(dest_format_struct, in_dests, destination_type,
+ MLX5_FLOW_DESTINATION_TYPE_VPORT);
+ MLX5_SET(dest_format_struct, in_dests, destination_id, vport_id);
+
+ err = mlx5_cmd_exec(mdev, in, inlen, out, sizeof(out));
+ kvfree(in);
+
+ return err;
+}
+
+int mlx5dr_cmd_del_flow_table_entry(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id)
+{
+ u32 out[MLX5_ST_SZ_DW(delete_fte_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(delete_fte_in)] = {};
+
+ MLX5_SET(delete_fte_in, in, opcode, MLX5_CMD_OP_DELETE_FLOW_TABLE_ENTRY);
+ MLX5_SET(delete_fte_in, in, table_type, table_type);
+ MLX5_SET(delete_fte_in, in, table_id, table_id);
+
+ return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+int mlx5dr_cmd_alloc_modify_header(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u8 num_of_actions,
+ u64 *actions,
+ u32 *modify_header_id)
+{
+ u32 out[MLX5_ST_SZ_DW(alloc_modify_header_context_out)] = {};
+ void *p_actions;
+ u32 inlen;
+ u32 *in;
+ int err;
+
+ inlen = MLX5_ST_SZ_BYTES(alloc_modify_header_context_in) +
+ num_of_actions * sizeof(u64);
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ MLX5_SET(alloc_modify_header_context_in, in, opcode,
+ MLX5_CMD_OP_ALLOC_MODIFY_HEADER_CONTEXT);
+ MLX5_SET(alloc_modify_header_context_in, in, table_type, table_type);
+ MLX5_SET(alloc_modify_header_context_in, in, num_of_actions, num_of_actions);
+ p_actions = MLX5_ADDR_OF(alloc_modify_header_context_in, in, actions);
+ memcpy(p_actions, actions, num_of_actions * sizeof(u64));
+
+ err = mlx5_cmd_exec(mdev, in, inlen, out, sizeof(out));
+ if (err)
+ goto out;
+
+ *modify_header_id = MLX5_GET(alloc_modify_header_context_out, out,
+ modify_header_id);
+out:
+ kvfree(in);
+ return err;
+}
+
+int mlx5dr_cmd_dealloc_modify_header(struct mlx5_core_dev *mdev,
+ u32 modify_header_id)
+{
+ u32 out[MLX5_ST_SZ_DW(dealloc_modify_header_context_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(dealloc_modify_header_context_in)] = {};
+
+ MLX5_SET(dealloc_modify_header_context_in, in, opcode,
+ MLX5_CMD_OP_DEALLOC_MODIFY_HEADER_CONTEXT);
+ MLX5_SET(dealloc_modify_header_context_in, in, modify_header_id,
+ modify_header_id);
+
+ return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+int mlx5dr_cmd_create_empty_flow_group(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id,
+ u32 *group_id)
+{
+ u32 out[MLX5_ST_SZ_DW(create_flow_group_out)] = {};
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ u32 *in;
+ int err;
+
+ in = kzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ MLX5_SET(create_flow_group_in, in, opcode, MLX5_CMD_OP_CREATE_FLOW_GROUP);
+ MLX5_SET(create_flow_group_in, in, table_type, table_type);
+ MLX5_SET(create_flow_group_in, in, table_id, table_id);
+
+ err = mlx5_cmd_exec(mdev, in, inlen, out, sizeof(out));
+ if (err)
+ goto out;
+
+ *group_id = MLX5_GET(create_flow_group_out, out, group_id);
+
+out:
+ kfree(in);
+ return err;
+}
+
+int mlx5dr_cmd_destroy_flow_group(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id,
+ u32 group_id)
+{
+ u32 in[MLX5_ST_SZ_DW(destroy_flow_group_in)] = {};
+ u32 out[MLX5_ST_SZ_DW(destroy_flow_group_out)] = {};
+
+ MLX5_SET(create_flow_group_in, in, opcode, MLX5_CMD_OP_DESTROY_FLOW_GROUP);
+ MLX5_SET(destroy_flow_group_in, in, table_type, table_type);
+ MLX5_SET(destroy_flow_group_in, in, table_id, table_id);
+ MLX5_SET(destroy_flow_group_in, in, group_id, group_id);
+
+ return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+int mlx5dr_cmd_create_flow_table(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u64 icm_addr_rx,
+ u64 icm_addr_tx,
+ u8 level,
+ bool sw_owner,
+ bool term_tbl,
+ u64 *fdb_rx_icm_addr,
+ u32 *table_id)
+{
+ u32 out[MLX5_ST_SZ_DW(create_flow_table_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(create_flow_table_in)] = {};
+ void *ft_mdev;
+ int err;
+
+ MLX5_SET(create_flow_table_in, in, opcode, MLX5_CMD_OP_CREATE_FLOW_TABLE);
+ MLX5_SET(create_flow_table_in, in, table_type, table_type);
+
+ ft_mdev = MLX5_ADDR_OF(create_flow_table_in, in, flow_table_context);
+ MLX5_SET(flow_table_context, ft_mdev, termination_table, term_tbl);
+ MLX5_SET(flow_table_context, ft_mdev, sw_owner, sw_owner);
+ MLX5_SET(flow_table_context, ft_mdev, level, level);
+
+ if (sw_owner) {
+ /* icm_addr_0 used for FDB RX / NIC TX / NIC_RX
+ * icm_addr_1 used for FDB TX
+ */
+ if (table_type == MLX5_FLOW_TABLE_TYPE_NIC_RX) {
+ MLX5_SET64(flow_table_context, ft_mdev,
+ sw_owner_icm_root_0, icm_addr_rx);
+ } else if (table_type == MLX5_FLOW_TABLE_TYPE_NIC_TX) {
+ MLX5_SET64(flow_table_context, ft_mdev,
+ sw_owner_icm_root_0, icm_addr_tx);
+ } else if (table_type == MLX5_FLOW_TABLE_TYPE_FDB) {
+ MLX5_SET64(flow_table_context, ft_mdev,
+ sw_owner_icm_root_0, icm_addr_rx);
+ MLX5_SET64(flow_table_context, ft_mdev,
+ sw_owner_icm_root_1, icm_addr_tx);
+ }
+ }
+
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+ if (err)
+ return err;
+
+ *table_id = MLX5_GET(create_flow_table_out, out, table_id);
+ if (!sw_owner && table_type == MLX5_FLOW_TABLE_TYPE_FDB)
+ *fdb_rx_icm_addr =
+ (u64)MLX5_GET(create_flow_table_out, out, icm_address_31_0) |
+ (u64)MLX5_GET(create_flow_table_out, out, icm_address_39_32) << 32 |
+ (u64)MLX5_GET(create_flow_table_out, out, icm_address_63_40) << 40;
+
+ return 0;
+}
+
+int mlx5dr_cmd_destroy_flow_table(struct mlx5_core_dev *mdev,
+ u32 table_id,
+ u32 table_type)
+{
+ u32 out[MLX5_ST_SZ_DW(destroy_flow_table_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(destroy_flow_table_in)] = {};
+
+ MLX5_SET(destroy_flow_table_in, in, opcode,
+ MLX5_CMD_OP_DESTROY_FLOW_TABLE);
+ MLX5_SET(destroy_flow_table_in, in, table_type, table_type);
+ MLX5_SET(destroy_flow_table_in, in, table_id, table_id);
+
+ return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+int mlx5dr_cmd_create_reformat_ctx(struct mlx5_core_dev *mdev,
+ enum mlx5_reformat_ctx_type rt,
+ size_t reformat_size,
+ void *reformat_data,
+ u32 *reformat_id)
+{
+ u32 out[MLX5_ST_SZ_DW(alloc_packet_reformat_context_out)] = {};
+ size_t inlen, cmd_data_sz, cmd_total_sz;
+ void *prctx;
+ void *pdata;
+ void *in;
+ int err;
+
+ cmd_total_sz = MLX5_ST_SZ_BYTES(alloc_packet_reformat_context_in);
+ cmd_data_sz = MLX5_FLD_SZ_BYTES(alloc_packet_reformat_context_in,
+ packet_reformat_context.reformat_data);
+ inlen = ALIGN(cmd_total_sz + reformat_size - cmd_data_sz, 4);
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ MLX5_SET(alloc_packet_reformat_context_in, in, opcode,
+ MLX5_CMD_OP_ALLOC_PACKET_REFORMAT_CONTEXT);
+
+ prctx = MLX5_ADDR_OF(alloc_packet_reformat_context_in, in, packet_reformat_context);
+ pdata = MLX5_ADDR_OF(packet_reformat_context_in, prctx, reformat_data);
+
+ MLX5_SET(packet_reformat_context_in, prctx, reformat_type, rt);
+ MLX5_SET(packet_reformat_context_in, prctx, reformat_data_size, reformat_size);
+ memcpy(pdata, reformat_data, reformat_size);
+
+ err = mlx5_cmd_exec(mdev, in, inlen, out, sizeof(out));
+ if (err)
+ return err;
+
+ *reformat_id = MLX5_GET(alloc_packet_reformat_context_out, out, packet_reformat_id);
+ kvfree(in);
+
+ return err;
+}
+
+void mlx5dr_cmd_destroy_reformat_ctx(struct mlx5_core_dev *mdev,
+ u32 reformat_id)
+{
+ u32 out[MLX5_ST_SZ_DW(dealloc_packet_reformat_context_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(dealloc_packet_reformat_context_in)] = {};
+
+ MLX5_SET(dealloc_packet_reformat_context_in, in, opcode,
+ MLX5_CMD_OP_DEALLOC_PACKET_REFORMAT_CONTEXT);
+ MLX5_SET(dealloc_packet_reformat_context_in, in, packet_reformat_id,
+ reformat_id);
+
+ mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+int mlx5dr_cmd_query_gid(struct mlx5_core_dev *mdev, u8 vhca_port_num,
+ u16 index, struct mlx5dr_cmd_gid_attr *attr)
+{
+ u32 out[MLX5_ST_SZ_DW(query_roce_address_out)] = {};
+ u32 in[MLX5_ST_SZ_DW(query_roce_address_in)] = {};
+ int err;
+
+ MLX5_SET(query_roce_address_in, in, opcode,
+ MLX5_CMD_OP_QUERY_ROCE_ADDRESS);
+
+ MLX5_SET(query_roce_address_in, in, roce_address_index, index);
+ MLX5_SET(query_roce_address_in, in, vhca_port_num, vhca_port_num);
+
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+ if (err)
+ return err;
+
+ memcpy(&attr->gid,
+ MLX5_ADDR_OF(query_roce_address_out,
+ out, roce_address.source_l3_address),
+ sizeof(attr->gid));
+ memcpy(attr->mac,
+ MLX5_ADDR_OF(query_roce_address_out, out,
+ roce_address.source_mac_47_32),
+ sizeof(attr->mac));
+
+ if (MLX5_GET(query_roce_address_out, out,
+ roce_address.roce_version) == MLX5_ROCE_VERSION_2)
+ attr->roce_ver = MLX5_ROCE_VERSION_2;
+ else
+ attr->roce_ver = MLX5_ROCE_VERSION_1;
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_crc32.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_crc32.c
new file mode 100644
index 0000000..9e2eccb
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_crc32.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+/* Copyright (c) 2011-2015 Stephan Brumme. All rights reserved.
+ * Slicing-by-16 contributed by Bulat Ziganshin
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ * In no event will the author be held liable for any damages arising from the
+ * of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it
+ * freely, subject to the following restrictions:
+ *
+ * 1. The origin of this software must not be misrepresented; you must not
+ * claim that you wrote the original software.
+ * 2. If you use this software in a product, an acknowledgment in the product
+ * documentation would be appreciated but is not required.
+ * 3. Altered source versions must be plainly marked as such, and must not be
+ * misrepresented as being the original software.
+ *
+ * Taken from http://create.stephan-brumme.com/crc32/ and adapted.
+ */
+
+#include "dr_types.h"
+
+#define DR_STE_CRC_POLY 0xEDB88320L
+
+static u32 dr_ste_crc_tab32[8][256];
+
+static void dr_crc32_calc_lookup_entry(u32 (*tbl)[256], u8 i, u8 j)
+{
+ tbl[i][j] = (tbl[i - 1][j] >> 8) ^ tbl[0][tbl[i - 1][j] & 0xff];
+}
+
+void mlx5dr_crc32_init_table(void)
+{
+ u32 crc, i, j;
+
+ for (i = 0; i < 256; i++) {
+ crc = i;
+ for (j = 0; j < 8; j++) {
+ if (crc & 0x00000001L)
+ crc = (crc >> 1) ^ DR_STE_CRC_POLY;
+ else
+ crc = crc >> 1;
+ }
+ dr_ste_crc_tab32[0][i] = crc;
+ }
+
+ /* Init CRC lookup tables according to crc_slice_8 algorithm */
+ for (i = 0; i < 256; i++) {
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 1, i);
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 2, i);
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 3, i);
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 4, i);
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 5, i);
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 6, i);
+ dr_crc32_calc_lookup_entry(dr_ste_crc_tab32, 7, i);
+ }
+}
+
+/* Compute CRC32 (Slicing-by-8 algorithm) */
+u32 mlx5dr_crc32_slice8_calc(const void *input_data, size_t length)
+{
+ const u32 *curr = (const u32 *)input_data;
+ const u8 *curr_char;
+ u32 crc = 0, one, two;
+
+ if (!input_data)
+ return 0;
+
+ /* Process eight bytes at once (Slicing-by-8) */
+ while (length >= 8) {
+ one = *curr++ ^ crc;
+ two = *curr++;
+
+ crc = dr_ste_crc_tab32[0][(two >> 24) & 0xff]
+ ^ dr_ste_crc_tab32[1][(two >> 16) & 0xff]
+ ^ dr_ste_crc_tab32[2][(two >> 8) & 0xff]
+ ^ dr_ste_crc_tab32[3][two & 0xff]
+ ^ dr_ste_crc_tab32[4][(one >> 24) & 0xff]
+ ^ dr_ste_crc_tab32[5][(one >> 16) & 0xff]
+ ^ dr_ste_crc_tab32[6][(one >> 8) & 0xff]
+ ^ dr_ste_crc_tab32[7][one & 0xff];
+
+ length -= 8;
+ }
+
+ curr_char = (const u8 *)curr;
+ /* Remaining 1 to 7 bytes (standard algorithm) */
+ while (length-- != 0)
+ crc = (crc >> 8) ^ dr_ste_crc_tab32[0][(crc & 0xff)
+ ^ *curr_char++];
+
+ return ((crc >> 24) & 0xff) | ((crc << 8) & 0xff0000) |
+ ((crc >> 8) & 0xff00) | ((crc << 24) & 0xff000000);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
new file mode 100644
index 0000000..5b24732
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_domain.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include <linux/mlx5/eswitch.h>
+#include "dr_types.h"
+
+static int dr_domain_init_cache(struct mlx5dr_domain *dmn)
+{
+ /* Per vport cached FW FT for checksum recalculation, this
+ * recalculation is needed due to a HW bug.
+ */
+ dmn->cache.recalc_cs_ft = kcalloc(dmn->info.caps.num_vports,
+ sizeof(dmn->cache.recalc_cs_ft[0]),
+ GFP_KERNEL);
+ if (!dmn->cache.recalc_cs_ft)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void dr_domain_uninit_cache(struct mlx5dr_domain *dmn)
+{
+ int i;
+
+ for (i = 0; i < dmn->info.caps.num_vports; i++) {
+ if (!dmn->cache.recalc_cs_ft[i])
+ continue;
+
+ mlx5dr_fw_destroy_recalc_cs_ft(dmn, dmn->cache.recalc_cs_ft[i]);
+ }
+
+ kfree(dmn->cache.recalc_cs_ft);
+}
+
+int mlx5dr_domain_cache_get_recalc_cs_ft_addr(struct mlx5dr_domain *dmn,
+ u32 vport_num,
+ u64 *rx_icm_addr)
+{
+ struct mlx5dr_fw_recalc_cs_ft *recalc_cs_ft;
+
+ recalc_cs_ft = dmn->cache.recalc_cs_ft[vport_num];
+ if (!recalc_cs_ft) {
+ /* Table not in cache, need to allocate a new one */
+ recalc_cs_ft = mlx5dr_fw_create_recalc_cs_ft(dmn, vport_num);
+ if (!recalc_cs_ft)
+ return -EINVAL;
+
+ dmn->cache.recalc_cs_ft[vport_num] = recalc_cs_ft;
+ }
+
+ *rx_icm_addr = recalc_cs_ft->rx_icm_addr;
+
+ return 0;
+}
+
+static int dr_domain_init_resources(struct mlx5dr_domain *dmn)
+{
+ int ret;
+
+ ret = mlx5_core_alloc_pd(dmn->mdev, &dmn->pdn);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Couldn't allocate PD\n");
+ return ret;
+ }
+
+ dmn->uar = mlx5_get_uars_page(dmn->mdev);
+ if (!dmn->uar) {
+ mlx5dr_err(dmn, "Couldn't allocate UAR\n");
+ ret = -ENOMEM;
+ goto clean_pd;
+ }
+
+ dmn->ste_icm_pool = mlx5dr_icm_pool_create(dmn, DR_ICM_TYPE_STE);
+ if (!dmn->ste_icm_pool) {
+ mlx5dr_err(dmn, "Couldn't get icm memory\n");
+ ret = -ENOMEM;
+ goto clean_uar;
+ }
+
+ dmn->action_icm_pool = mlx5dr_icm_pool_create(dmn, DR_ICM_TYPE_MODIFY_ACTION);
+ if (!dmn->action_icm_pool) {
+ mlx5dr_err(dmn, "Couldn't get action icm memory\n");
+ ret = -ENOMEM;
+ goto free_ste_icm_pool;
+ }
+
+ ret = mlx5dr_send_ring_alloc(dmn);
+ if (ret) {
+ mlx5dr_err(dmn, "Couldn't create send-ring\n");
+ goto free_action_icm_pool;
+ }
+
+ return 0;
+
+free_action_icm_pool:
+ mlx5dr_icm_pool_destroy(dmn->action_icm_pool);
+free_ste_icm_pool:
+ mlx5dr_icm_pool_destroy(dmn->ste_icm_pool);
+clean_uar:
+ mlx5_put_uars_page(dmn->mdev, dmn->uar);
+clean_pd:
+ mlx5_core_dealloc_pd(dmn->mdev, dmn->pdn);
+
+ return ret;
+}
+
+static void dr_domain_uninit_resources(struct mlx5dr_domain *dmn)
+{
+ mlx5dr_send_ring_free(dmn, dmn->send_ring);
+ mlx5dr_icm_pool_destroy(dmn->action_icm_pool);
+ mlx5dr_icm_pool_destroy(dmn->ste_icm_pool);
+ mlx5_put_uars_page(dmn->mdev, dmn->uar);
+ mlx5_core_dealloc_pd(dmn->mdev, dmn->pdn);
+}
+
+static int dr_domain_query_vport(struct mlx5dr_domain *dmn,
+ bool other_vport,
+ u16 vport_number)
+{
+ struct mlx5dr_cmd_vport_cap *vport_caps;
+ int ret;
+
+ vport_caps = &dmn->info.caps.vports_caps[vport_number];
+
+ ret = mlx5dr_cmd_query_esw_vport_context(dmn->mdev,
+ other_vport,
+ vport_number,
+ &vport_caps->icm_address_rx,
+ &vport_caps->icm_address_tx);
+ if (ret)
+ return ret;
+
+ ret = mlx5dr_cmd_query_gvmi(dmn->mdev,
+ other_vport,
+ vport_number,
+ &vport_caps->vport_gvmi);
+ if (ret)
+ return ret;
+
+ vport_caps->num = vport_number;
+ vport_caps->vhca_gvmi = dmn->info.caps.gvmi;
+
+ return 0;
+}
+
+static int dr_domain_query_vports(struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_esw_caps *esw_caps = &dmn->info.caps.esw_caps;
+ struct mlx5dr_cmd_vport_cap *wire_vport;
+ int vport;
+ int ret;
+
+ /* Query vports (except wire vport) */
+ for (vport = 0; vport < dmn->info.caps.num_esw_ports - 1; vport++) {
+ ret = dr_domain_query_vport(dmn, !!vport, vport);
+ if (ret)
+ return ret;
+ }
+
+ /* Last vport is the wire port */
+ wire_vport = &dmn->info.caps.vports_caps[vport];
+ wire_vport->num = WIRE_PORT;
+ wire_vport->icm_address_rx = esw_caps->uplink_icm_address_rx;
+ wire_vport->icm_address_tx = esw_caps->uplink_icm_address_tx;
+ wire_vport->vport_gvmi = 0;
+ wire_vport->vhca_gvmi = dmn->info.caps.gvmi;
+
+ return 0;
+}
+
+static int dr_domain_query_fdb_caps(struct mlx5_core_dev *mdev,
+ struct mlx5dr_domain *dmn)
+{
+ int ret;
+
+ if (!dmn->info.caps.eswitch_manager)
+ return -EOPNOTSUPP;
+
+ ret = mlx5dr_cmd_query_esw_caps(mdev, &dmn->info.caps.esw_caps);
+ if (ret)
+ return ret;
+
+ dmn->info.caps.fdb_sw_owner = dmn->info.caps.esw_caps.sw_owner;
+ dmn->info.caps.esw_rx_drop_address = dmn->info.caps.esw_caps.drop_icm_address_rx;
+ dmn->info.caps.esw_tx_drop_address = dmn->info.caps.esw_caps.drop_icm_address_tx;
+
+ dmn->info.caps.vports_caps = kcalloc(dmn->info.caps.num_esw_ports,
+ sizeof(dmn->info.caps.vports_caps[0]),
+ GFP_KERNEL);
+ if (!dmn->info.caps.vports_caps)
+ return -ENOMEM;
+
+ ret = dr_domain_query_vports(dmn);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Failed to query vports caps\n");
+ goto free_vports_caps;
+ }
+
+ dmn->info.caps.num_vports = dmn->info.caps.num_esw_ports - 1;
+
+ return 0;
+
+free_vports_caps:
+ kfree(dmn->info.caps.vports_caps);
+ dmn->info.caps.vports_caps = NULL;
+ return ret;
+}
+
+static int dr_domain_caps_init(struct mlx5_core_dev *mdev,
+ struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_cmd_vport_cap *vport_cap;
+ int ret;
+
+ if (MLX5_CAP_GEN(mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH) {
+ mlx5dr_dbg(dmn, "Failed to allocate domain, bad link type\n");
+ return -EOPNOTSUPP;
+ }
+
+ dmn->info.caps.num_esw_ports = mlx5_eswitch_get_total_vports(mdev);
+
+ ret = mlx5dr_cmd_query_device(mdev, &dmn->info.caps);
+ if (ret)
+ return ret;
+
+ ret = dr_domain_query_fdb_caps(mdev, dmn);
+ if (ret)
+ return ret;
+
+ switch (dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ if (!dmn->info.caps.rx_sw_owner)
+ return -ENOTSUPP;
+
+ dmn->info.supp_sw_steering = true;
+ dmn->info.rx.ste_type = MLX5DR_STE_TYPE_RX;
+ dmn->info.rx.default_icm_addr = dmn->info.caps.nic_rx_drop_address;
+ dmn->info.rx.drop_icm_addr = dmn->info.caps.nic_rx_drop_address;
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ if (!dmn->info.caps.tx_sw_owner)
+ return -ENOTSUPP;
+
+ dmn->info.supp_sw_steering = true;
+ dmn->info.tx.ste_type = MLX5DR_STE_TYPE_TX;
+ dmn->info.tx.default_icm_addr = dmn->info.caps.nic_tx_allow_address;
+ dmn->info.tx.drop_icm_addr = dmn->info.caps.nic_tx_drop_address;
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ if (!dmn->info.caps.eswitch_manager)
+ return -ENOTSUPP;
+
+ if (!dmn->info.caps.fdb_sw_owner)
+ return -ENOTSUPP;
+
+ dmn->info.rx.ste_type = MLX5DR_STE_TYPE_RX;
+ dmn->info.tx.ste_type = MLX5DR_STE_TYPE_TX;
+ vport_cap = mlx5dr_get_vport_cap(&dmn->info.caps, 0);
+ if (!vport_cap) {
+ mlx5dr_dbg(dmn, "Failed to get esw manager vport\n");
+ return -ENOENT;
+ }
+
+ dmn->info.supp_sw_steering = true;
+ dmn->info.tx.default_icm_addr = vport_cap->icm_address_tx;
+ dmn->info.rx.default_icm_addr = vport_cap->icm_address_rx;
+ dmn->info.rx.drop_icm_addr = dmn->info.caps.esw_rx_drop_address;
+ dmn->info.tx.drop_icm_addr = dmn->info.caps.esw_tx_drop_address;
+ break;
+ default:
+ mlx5dr_dbg(dmn, "Invalid domain\n");
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static void dr_domain_caps_uninit(struct mlx5dr_domain *dmn)
+{
+ kfree(dmn->info.caps.vports_caps);
+}
+
+struct mlx5dr_domain *
+mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type)
+{
+ struct mlx5dr_domain *dmn;
+ int ret;
+
+ if (type > MLX5DR_DOMAIN_TYPE_FDB)
+ return NULL;
+
+ dmn = kzalloc(sizeof(*dmn), GFP_KERNEL);
+ if (!dmn)
+ return NULL;
+
+ dmn->mdev = mdev;
+ dmn->type = type;
+ refcount_set(&dmn->refcount, 1);
+ mutex_init(&dmn->mutex);
+
+ if (dr_domain_caps_init(mdev, dmn)) {
+ mlx5dr_dbg(dmn, "Failed init domain, no caps\n");
+ goto free_domain;
+ }
+
+ dmn->info.max_log_action_icm_sz = DR_CHUNK_SIZE_4K;
+ dmn->info.max_log_sw_icm_sz = min_t(u32, DR_CHUNK_SIZE_1024K,
+ dmn->info.caps.log_icm_size);
+
+ if (!dmn->info.supp_sw_steering) {
+ mlx5dr_err(dmn, "SW steering is not supported\n");
+ goto uninit_caps;
+ }
+
+ /* Allocate resources */
+ ret = dr_domain_init_resources(dmn);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed init domain resources\n");
+ goto uninit_caps;
+ }
+
+ ret = dr_domain_init_cache(dmn);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed initialize domain cache\n");
+ goto uninit_resourses;
+ }
+
+ /* Init CRC table for htbl CRC calculation */
+ mlx5dr_crc32_init_table();
+
+ return dmn;
+
+uninit_resourses:
+ dr_domain_uninit_resources(dmn);
+uninit_caps:
+ dr_domain_caps_uninit(dmn);
+free_domain:
+ kfree(dmn);
+ return NULL;
+}
+
+/* Assure synchronization of the device steering tables with updates made by SW
+ * insertion.
+ */
+int mlx5dr_domain_sync(struct mlx5dr_domain *dmn, u32 flags)
+{
+ int ret = 0;
+
+ if (flags & MLX5DR_DOMAIN_SYNC_FLAGS_SW) {
+ mutex_lock(&dmn->mutex);
+ ret = mlx5dr_send_ring_force_drain(dmn);
+ mutex_unlock(&dmn->mutex);
+ if (ret)
+ return ret;
+ }
+
+ if (flags & MLX5DR_DOMAIN_SYNC_FLAGS_HW)
+ ret = mlx5dr_cmd_sync_steering(dmn->mdev);
+
+ return ret;
+}
+
+int mlx5dr_domain_destroy(struct mlx5dr_domain *dmn)
+{
+ if (refcount_read(&dmn->refcount) > 1)
+ return -EBUSY;
+
+ /* make sure resources are not used by the hardware */
+ mlx5dr_cmd_sync_steering(dmn->mdev);
+ dr_domain_uninit_cache(dmn);
+ dr_domain_uninit_resources(dmn);
+ dr_domain_caps_uninit(dmn);
+ mutex_destroy(&dmn->mutex);
+ kfree(dmn);
+ return 0;
+}
+
+void mlx5dr_domain_set_peer(struct mlx5dr_domain *dmn,
+ struct mlx5dr_domain *peer_dmn)
+{
+ mutex_lock(&dmn->mutex);
+
+ if (dmn->peer_dmn)
+ refcount_dec(&dmn->peer_dmn->refcount);
+
+ dmn->peer_dmn = peer_dmn;
+
+ if (dmn->peer_dmn)
+ refcount_inc(&dmn->peer_dmn->refcount);
+
+ mutex_unlock(&dmn->mutex);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_fw.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_fw.c
new file mode 100644
index 0000000..60ef6e6
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_fw.c
@@ -0,0 +1,93 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include <linux/types.h>
+#include "dr_types.h"
+
+struct mlx5dr_fw_recalc_cs_ft *
+mlx5dr_fw_create_recalc_cs_ft(struct mlx5dr_domain *dmn, u32 vport_num)
+{
+ struct mlx5dr_fw_recalc_cs_ft *recalc_cs_ft;
+ u32 table_id, group_id, modify_hdr_id;
+ u64 rx_icm_addr, modify_ttl_action;
+ int ret;
+
+ recalc_cs_ft = kzalloc(sizeof(*recalc_cs_ft), GFP_KERNEL);
+ if (!recalc_cs_ft)
+ return NULL;
+
+ ret = mlx5dr_cmd_create_flow_table(dmn->mdev, MLX5_FLOW_TABLE_TYPE_FDB,
+ 0, 0, dmn->info.caps.max_ft_level - 1,
+ false, true, &rx_icm_addr, &table_id);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed creating TTL W/A FW flow table %d\n", ret);
+ goto free_ttl_tbl;
+ }
+
+ ret = mlx5dr_cmd_create_empty_flow_group(dmn->mdev,
+ MLX5_FLOW_TABLE_TYPE_FDB,
+ table_id, &group_id);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed creating TTL W/A FW flow group %d\n", ret);
+ goto destroy_flow_table;
+ }
+
+ /* Modify TTL action by adding zero to trigger CS recalculation */
+ modify_ttl_action = 0;
+ MLX5_SET(set_action_in, &modify_ttl_action, action_type, MLX5_ACTION_TYPE_ADD);
+ MLX5_SET(set_action_in, &modify_ttl_action, field, MLX5_ACTION_IN_FIELD_OUT_IP_TTL);
+
+ ret = mlx5dr_cmd_alloc_modify_header(dmn->mdev, MLX5_FLOW_TABLE_TYPE_FDB, 1,
+ &modify_ttl_action,
+ &modify_hdr_id);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed modify header TTL %d\n", ret);
+ goto destroy_flow_group;
+ }
+
+ ret = mlx5dr_cmd_set_fte_modify_and_vport(dmn->mdev,
+ MLX5_FLOW_TABLE_TYPE_FDB,
+ table_id, group_id, modify_hdr_id,
+ vport_num);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed setting TTL W/A flow table entry %d\n", ret);
+ goto dealloc_modify_header;
+ }
+
+ recalc_cs_ft->modify_hdr_id = modify_hdr_id;
+ recalc_cs_ft->rx_icm_addr = rx_icm_addr;
+ recalc_cs_ft->table_id = table_id;
+ recalc_cs_ft->group_id = group_id;
+
+ return recalc_cs_ft;
+
+dealloc_modify_header:
+ mlx5dr_cmd_dealloc_modify_header(dmn->mdev, modify_hdr_id);
+destroy_flow_group:
+ mlx5dr_cmd_destroy_flow_group(dmn->mdev,
+ MLX5_FLOW_TABLE_TYPE_FDB,
+ table_id, group_id);
+destroy_flow_table:
+ mlx5dr_cmd_destroy_flow_table(dmn->mdev, table_id, MLX5_FLOW_TABLE_TYPE_FDB);
+free_ttl_tbl:
+ kfree(recalc_cs_ft);
+ return NULL;
+}
+
+void mlx5dr_fw_destroy_recalc_cs_ft(struct mlx5dr_domain *dmn,
+ struct mlx5dr_fw_recalc_cs_ft *recalc_cs_ft)
+{
+ mlx5dr_cmd_del_flow_table_entry(dmn->mdev,
+ MLX5_FLOW_TABLE_TYPE_FDB,
+ recalc_cs_ft->table_id);
+ mlx5dr_cmd_dealloc_modify_header(dmn->mdev, recalc_cs_ft->modify_hdr_id);
+ mlx5dr_cmd_destroy_flow_group(dmn->mdev,
+ MLX5_FLOW_TABLE_TYPE_FDB,
+ recalc_cs_ft->table_id,
+ recalc_cs_ft->group_id);
+ mlx5dr_cmd_destroy_flow_table(dmn->mdev,
+ recalc_cs_ft->table_id,
+ MLX5_FLOW_TABLE_TYPE_FDB);
+
+ kfree(recalc_cs_ft);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c
new file mode 100644
index 0000000..913f1e5
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_icm_pool.c
@@ -0,0 +1,570 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+#define DR_ICM_MODIFY_HDR_ALIGN_BASE 64
+#define DR_ICM_SYNC_THRESHOLD (64 * 1024 * 1024)
+
+struct mlx5dr_icm_pool;
+
+struct mlx5dr_icm_bucket {
+ struct mlx5dr_icm_pool *pool;
+
+ /* Chunks that aren't visible to HW not directly and not in cache */
+ struct list_head free_list;
+ unsigned int free_list_count;
+
+ /* Used chunks, HW may be accessing this memory */
+ struct list_head used_list;
+ unsigned int used_list_count;
+
+ /* HW may be accessing this memory but at some future,
+ * undetermined time, it might cease to do so. Before deciding to call
+ * sync_ste, this list is moved to sync_list
+ */
+ struct list_head hot_list;
+ unsigned int hot_list_count;
+
+ /* Pending sync list, entries from the hot list are moved to this list.
+ * sync_ste is executed and then sync_list is concatenated to the free list
+ */
+ struct list_head sync_list;
+ unsigned int sync_list_count;
+
+ u32 total_chunks;
+ u32 num_of_entries;
+ u32 entry_size;
+ /* protect the ICM bucket */
+ struct mutex mutex;
+};
+
+struct mlx5dr_icm_pool {
+ struct mlx5dr_icm_bucket *buckets;
+ enum mlx5dr_icm_type icm_type;
+ enum mlx5dr_icm_chunk_size max_log_chunk_sz;
+ enum mlx5dr_icm_chunk_size num_of_buckets;
+ struct list_head icm_mr_list;
+ /* protect the ICM MR list */
+ struct mutex mr_mutex;
+ struct mlx5dr_domain *dmn;
+};
+
+struct mlx5dr_icm_dm {
+ u32 obj_id;
+ enum mlx5_sw_icm_type type;
+ phys_addr_t addr;
+ size_t length;
+};
+
+struct mlx5dr_icm_mr {
+ struct mlx5dr_icm_pool *pool;
+ struct mlx5_core_mkey mkey;
+ struct mlx5dr_icm_dm dm;
+ size_t used_length;
+ size_t length;
+ u64 icm_start_addr;
+ struct list_head mr_list;
+};
+
+static int dr_icm_create_dm_mkey(struct mlx5_core_dev *mdev,
+ u32 pd, u64 length, u64 start_addr, int mode,
+ struct mlx5_core_mkey *mkey)
+{
+ u32 inlen = MLX5_ST_SZ_BYTES(create_mkey_in);
+ u32 in[MLX5_ST_SZ_DW(create_mkey_in)] = {};
+ void *mkc;
+
+ mkc = MLX5_ADDR_OF(create_mkey_in, in, memory_key_mkey_entry);
+
+ MLX5_SET(mkc, mkc, access_mode_1_0, mode);
+ MLX5_SET(mkc, mkc, access_mode_4_2, (mode >> 2) & 0x7);
+ MLX5_SET(mkc, mkc, lw, 1);
+ MLX5_SET(mkc, mkc, lr, 1);
+ if (mode == MLX5_MKC_ACCESS_MODE_SW_ICM) {
+ MLX5_SET(mkc, mkc, rw, 1);
+ MLX5_SET(mkc, mkc, rr, 1);
+ }
+
+ MLX5_SET64(mkc, mkc, len, length);
+ MLX5_SET(mkc, mkc, pd, pd);
+ MLX5_SET(mkc, mkc, qpn, 0xffffff);
+ MLX5_SET64(mkc, mkc, start_addr, start_addr);
+
+ return mlx5_core_create_mkey(mdev, mkey, in, inlen);
+}
+
+static struct mlx5dr_icm_mr *
+dr_icm_pool_mr_create(struct mlx5dr_icm_pool *pool,
+ enum mlx5_sw_icm_type type,
+ size_t align_base)
+{
+ struct mlx5_core_dev *mdev = pool->dmn->mdev;
+ struct mlx5dr_icm_mr *icm_mr;
+ size_t align_diff;
+ int err;
+
+ icm_mr = kvzalloc(sizeof(*icm_mr), GFP_KERNEL);
+ if (!icm_mr)
+ return NULL;
+
+ icm_mr->pool = pool;
+ INIT_LIST_HEAD(&icm_mr->mr_list);
+
+ icm_mr->dm.type = type;
+
+ /* 2^log_biggest_table * entry-size * double-for-alignment */
+ icm_mr->dm.length = mlx5dr_icm_pool_chunk_size_to_byte(pool->max_log_chunk_sz,
+ pool->icm_type) * 2;
+
+ err = mlx5_dm_sw_icm_alloc(mdev, icm_mr->dm.type, icm_mr->dm.length, 0,
+ &icm_mr->dm.addr, &icm_mr->dm.obj_id);
+ if (err) {
+ mlx5dr_err(pool->dmn, "Failed to allocate SW ICM memory, err (%d)\n", err);
+ goto free_icm_mr;
+ }
+
+ /* Register device memory */
+ err = dr_icm_create_dm_mkey(mdev, pool->dmn->pdn,
+ icm_mr->dm.length,
+ icm_mr->dm.addr,
+ MLX5_MKC_ACCESS_MODE_SW_ICM,
+ &icm_mr->mkey);
+ if (err) {
+ mlx5dr_err(pool->dmn, "Failed to create SW ICM MKEY, err (%d)\n", err);
+ goto free_dm;
+ }
+
+ icm_mr->icm_start_addr = icm_mr->dm.addr;
+
+ align_diff = icm_mr->icm_start_addr % align_base;
+ if (align_diff)
+ icm_mr->used_length = align_base - align_diff;
+
+ list_add_tail(&icm_mr->mr_list, &pool->icm_mr_list);
+
+ return icm_mr;
+
+free_dm:
+ mlx5_dm_sw_icm_dealloc(mdev, icm_mr->dm.type, icm_mr->dm.length, 0,
+ icm_mr->dm.addr, icm_mr->dm.obj_id);
+free_icm_mr:
+ kvfree(icm_mr);
+ return NULL;
+}
+
+static void dr_icm_pool_mr_destroy(struct mlx5dr_icm_mr *icm_mr)
+{
+ struct mlx5_core_dev *mdev = icm_mr->pool->dmn->mdev;
+ struct mlx5dr_icm_dm *dm = &icm_mr->dm;
+
+ list_del(&icm_mr->mr_list);
+ mlx5_core_destroy_mkey(mdev, &icm_mr->mkey);
+ mlx5_dm_sw_icm_dealloc(mdev, dm->type, dm->length, 0,
+ dm->addr, dm->obj_id);
+ kvfree(icm_mr);
+}
+
+static int dr_icm_chunk_ste_init(struct mlx5dr_icm_chunk *chunk)
+{
+ struct mlx5dr_icm_bucket *bucket = chunk->bucket;
+
+ chunk->ste_arr = kvzalloc(bucket->num_of_entries *
+ sizeof(chunk->ste_arr[0]), GFP_KERNEL);
+ if (!chunk->ste_arr)
+ return -ENOMEM;
+
+ chunk->hw_ste_arr = kvzalloc(bucket->num_of_entries *
+ DR_STE_SIZE_REDUCED, GFP_KERNEL);
+ if (!chunk->hw_ste_arr)
+ goto out_free_ste_arr;
+
+ chunk->miss_list = kvmalloc(bucket->num_of_entries *
+ sizeof(chunk->miss_list[0]), GFP_KERNEL);
+ if (!chunk->miss_list)
+ goto out_free_hw_ste_arr;
+
+ return 0;
+
+out_free_hw_ste_arr:
+ kvfree(chunk->hw_ste_arr);
+out_free_ste_arr:
+ kvfree(chunk->ste_arr);
+ return -ENOMEM;
+}
+
+static int dr_icm_chunks_create(struct mlx5dr_icm_bucket *bucket)
+{
+ size_t mr_free_size, mr_req_size, mr_row_size;
+ struct mlx5dr_icm_pool *pool = bucket->pool;
+ struct mlx5dr_icm_mr *icm_mr = NULL;
+ struct mlx5dr_icm_chunk *chunk;
+ enum mlx5_sw_icm_type dm_type;
+ size_t align_base;
+ int i, err = 0;
+
+ mr_req_size = bucket->num_of_entries * bucket->entry_size;
+ mr_row_size = mlx5dr_icm_pool_chunk_size_to_byte(pool->max_log_chunk_sz,
+ pool->icm_type);
+
+ if (pool->icm_type == DR_ICM_TYPE_STE) {
+ dm_type = MLX5_SW_ICM_TYPE_STEERING;
+ /* Align base is the biggest chunk size / row size */
+ align_base = mr_row_size;
+ } else {
+ dm_type = MLX5_SW_ICM_TYPE_HEADER_MODIFY;
+ /* Align base is 64B */
+ align_base = DR_ICM_MODIFY_HDR_ALIGN_BASE;
+ }
+
+ mutex_lock(&pool->mr_mutex);
+ if (!list_empty(&pool->icm_mr_list)) {
+ icm_mr = list_last_entry(&pool->icm_mr_list,
+ struct mlx5dr_icm_mr, mr_list);
+
+ if (icm_mr)
+ mr_free_size = icm_mr->dm.length - icm_mr->used_length;
+ }
+
+ if (!icm_mr || mr_free_size < mr_row_size) {
+ icm_mr = dr_icm_pool_mr_create(pool, dm_type, align_base);
+ if (!icm_mr) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+ }
+
+ /* Create memory aligned chunks */
+ for (i = 0; i < mr_row_size / mr_req_size; i++) {
+ chunk = kvzalloc(sizeof(*chunk), GFP_KERNEL);
+ if (!chunk) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+
+ chunk->bucket = bucket;
+ chunk->rkey = icm_mr->mkey.key;
+ /* mr start addr is zero based */
+ chunk->mr_addr = icm_mr->used_length;
+ chunk->icm_addr = (uintptr_t)icm_mr->icm_start_addr + icm_mr->used_length;
+ icm_mr->used_length += mr_req_size;
+ chunk->num_of_entries = bucket->num_of_entries;
+ chunk->byte_size = chunk->num_of_entries * bucket->entry_size;
+
+ if (pool->icm_type == DR_ICM_TYPE_STE) {
+ err = dr_icm_chunk_ste_init(chunk);
+ if (err)
+ goto out_free_chunk;
+ }
+
+ INIT_LIST_HEAD(&chunk->chunk_list);
+ list_add(&chunk->chunk_list, &bucket->free_list);
+ bucket->free_list_count++;
+ bucket->total_chunks++;
+ }
+ mutex_unlock(&pool->mr_mutex);
+ return 0;
+
+out_free_chunk:
+ kvfree(chunk);
+out_err:
+ mutex_unlock(&pool->mr_mutex);
+ return err;
+}
+
+static void dr_icm_chunk_ste_cleanup(struct mlx5dr_icm_chunk *chunk)
+{
+ kvfree(chunk->miss_list);
+ kvfree(chunk->hw_ste_arr);
+ kvfree(chunk->ste_arr);
+}
+
+static void dr_icm_chunk_destroy(struct mlx5dr_icm_chunk *chunk)
+{
+ struct mlx5dr_icm_bucket *bucket = chunk->bucket;
+
+ list_del(&chunk->chunk_list);
+ bucket->total_chunks--;
+
+ if (bucket->pool->icm_type == DR_ICM_TYPE_STE)
+ dr_icm_chunk_ste_cleanup(chunk);
+
+ kvfree(chunk);
+}
+
+static void dr_icm_bucket_init(struct mlx5dr_icm_pool *pool,
+ struct mlx5dr_icm_bucket *bucket,
+ enum mlx5dr_icm_chunk_size chunk_size)
+{
+ if (pool->icm_type == DR_ICM_TYPE_STE)
+ bucket->entry_size = DR_STE_SIZE;
+ else
+ bucket->entry_size = DR_MODIFY_ACTION_SIZE;
+
+ bucket->num_of_entries = mlx5dr_icm_pool_chunk_size_to_entries(chunk_size);
+ bucket->pool = pool;
+ mutex_init(&bucket->mutex);
+ INIT_LIST_HEAD(&bucket->free_list);
+ INIT_LIST_HEAD(&bucket->used_list);
+ INIT_LIST_HEAD(&bucket->hot_list);
+ INIT_LIST_HEAD(&bucket->sync_list);
+}
+
+static void dr_icm_bucket_cleanup(struct mlx5dr_icm_bucket *bucket)
+{
+ struct mlx5dr_icm_chunk *chunk, *next;
+
+ mutex_destroy(&bucket->mutex);
+ list_splice_tail_init(&bucket->sync_list, &bucket->free_list);
+ list_splice_tail_init(&bucket->hot_list, &bucket->free_list);
+
+ list_for_each_entry_safe(chunk, next, &bucket->free_list, chunk_list)
+ dr_icm_chunk_destroy(chunk);
+
+ WARN_ON(bucket->total_chunks != 0);
+
+ /* Cleanup of unreturned chunks */
+ list_for_each_entry_safe(chunk, next, &bucket->used_list, chunk_list)
+ dr_icm_chunk_destroy(chunk);
+}
+
+static u64 dr_icm_hot_mem_size(struct mlx5dr_icm_pool *pool)
+{
+ u64 hot_size = 0;
+ int chunk_order;
+
+ for (chunk_order = 0; chunk_order < pool->num_of_buckets; chunk_order++)
+ hot_size += pool->buckets[chunk_order].hot_list_count *
+ mlx5dr_icm_pool_chunk_size_to_byte(chunk_order, pool->icm_type);
+
+ return hot_size;
+}
+
+static bool dr_icm_reuse_hot_entries(struct mlx5dr_icm_pool *pool,
+ struct mlx5dr_icm_bucket *bucket)
+{
+ u64 bytes_for_sync;
+
+ bytes_for_sync = dr_icm_hot_mem_size(pool);
+ if (bytes_for_sync < DR_ICM_SYNC_THRESHOLD || !bucket->hot_list_count)
+ return false;
+
+ return true;
+}
+
+static void dr_icm_chill_bucket_start(struct mlx5dr_icm_bucket *bucket)
+{
+ list_splice_tail_init(&bucket->hot_list, &bucket->sync_list);
+ bucket->sync_list_count += bucket->hot_list_count;
+ bucket->hot_list_count = 0;
+}
+
+static void dr_icm_chill_bucket_end(struct mlx5dr_icm_bucket *bucket)
+{
+ list_splice_tail_init(&bucket->sync_list, &bucket->free_list);
+ bucket->free_list_count += bucket->sync_list_count;
+ bucket->sync_list_count = 0;
+}
+
+static void dr_icm_chill_bucket_abort(struct mlx5dr_icm_bucket *bucket)
+{
+ list_splice_tail_init(&bucket->sync_list, &bucket->hot_list);
+ bucket->hot_list_count += bucket->sync_list_count;
+ bucket->sync_list_count = 0;
+}
+
+static void dr_icm_chill_buckets_start(struct mlx5dr_icm_pool *pool,
+ struct mlx5dr_icm_bucket *cb,
+ bool buckets[DR_CHUNK_SIZE_MAX])
+{
+ struct mlx5dr_icm_bucket *bucket;
+ int i;
+
+ for (i = 0; i < pool->num_of_buckets; i++) {
+ bucket = &pool->buckets[i];
+ if (bucket == cb) {
+ dr_icm_chill_bucket_start(bucket);
+ continue;
+ }
+
+ /* Freeing the mutex is done at the end of that process, after
+ * sync_ste was executed at dr_icm_chill_buckets_end func.
+ */
+ if (mutex_trylock(&bucket->mutex)) {
+ dr_icm_chill_bucket_start(bucket);
+ buckets[i] = true;
+ }
+ }
+}
+
+static void dr_icm_chill_buckets_end(struct mlx5dr_icm_pool *pool,
+ struct mlx5dr_icm_bucket *cb,
+ bool buckets[DR_CHUNK_SIZE_MAX])
+{
+ struct mlx5dr_icm_bucket *bucket;
+ int i;
+
+ for (i = 0; i < pool->num_of_buckets; i++) {
+ bucket = &pool->buckets[i];
+ if (bucket == cb) {
+ dr_icm_chill_bucket_end(bucket);
+ continue;
+ }
+
+ if (!buckets[i])
+ continue;
+
+ dr_icm_chill_bucket_end(bucket);
+ mutex_unlock(&bucket->mutex);
+ }
+}
+
+static void dr_icm_chill_buckets_abort(struct mlx5dr_icm_pool *pool,
+ struct mlx5dr_icm_bucket *cb,
+ bool buckets[DR_CHUNK_SIZE_MAX])
+{
+ struct mlx5dr_icm_bucket *bucket;
+ int i;
+
+ for (i = 0; i < pool->num_of_buckets; i++) {
+ bucket = &pool->buckets[i];
+ if (bucket == cb) {
+ dr_icm_chill_bucket_abort(bucket);
+ continue;
+ }
+
+ if (!buckets[i])
+ continue;
+
+ dr_icm_chill_bucket_abort(bucket);
+ mutex_unlock(&bucket->mutex);
+ }
+}
+
+/* Allocate an ICM chunk, each chunk holds a piece of ICM memory and
+ * also memory used for HW STE management for optimizations.
+ */
+struct mlx5dr_icm_chunk *
+mlx5dr_icm_alloc_chunk(struct mlx5dr_icm_pool *pool,
+ enum mlx5dr_icm_chunk_size chunk_size)
+{
+ struct mlx5dr_icm_chunk *chunk = NULL; /* Fix compilation warning */
+ bool buckets[DR_CHUNK_SIZE_MAX] = {};
+ struct mlx5dr_icm_bucket *bucket;
+ int err;
+
+ if (chunk_size > pool->max_log_chunk_sz)
+ return NULL;
+
+ bucket = &pool->buckets[chunk_size];
+
+ mutex_lock(&bucket->mutex);
+
+ /* Take chunk from pool if available, otherwise allocate new chunks */
+ if (list_empty(&bucket->free_list)) {
+ if (dr_icm_reuse_hot_entries(pool, bucket)) {
+ dr_icm_chill_buckets_start(pool, bucket, buckets);
+ err = mlx5dr_cmd_sync_steering(pool->dmn->mdev);
+ if (err) {
+ dr_icm_chill_buckets_abort(pool, bucket, buckets);
+ mlx5dr_dbg(pool->dmn, "Sync_steering failed\n");
+ chunk = NULL;
+ goto out;
+ }
+ dr_icm_chill_buckets_end(pool, bucket, buckets);
+ } else {
+ dr_icm_chunks_create(bucket);
+ }
+ }
+
+ if (!list_empty(&bucket->free_list)) {
+ chunk = list_last_entry(&bucket->free_list,
+ struct mlx5dr_icm_chunk,
+ chunk_list);
+ if (chunk) {
+ list_del_init(&chunk->chunk_list);
+ list_add_tail(&chunk->chunk_list, &bucket->used_list);
+ bucket->free_list_count--;
+ bucket->used_list_count++;
+ }
+ }
+out:
+ mutex_unlock(&bucket->mutex);
+ return chunk;
+}
+
+void mlx5dr_icm_free_chunk(struct mlx5dr_icm_chunk *chunk)
+{
+ struct mlx5dr_icm_bucket *bucket = chunk->bucket;
+
+ if (bucket->pool->icm_type == DR_ICM_TYPE_STE) {
+ memset(chunk->ste_arr, 0,
+ bucket->num_of_entries * sizeof(chunk->ste_arr[0]));
+ memset(chunk->hw_ste_arr, 0,
+ bucket->num_of_entries * DR_STE_SIZE_REDUCED);
+ }
+
+ mutex_lock(&bucket->mutex);
+ list_del_init(&chunk->chunk_list);
+ list_add_tail(&chunk->chunk_list, &bucket->hot_list);
+ bucket->hot_list_count++;
+ bucket->used_list_count--;
+ mutex_unlock(&bucket->mutex);
+}
+
+struct mlx5dr_icm_pool *mlx5dr_icm_pool_create(struct mlx5dr_domain *dmn,
+ enum mlx5dr_icm_type icm_type)
+{
+ enum mlx5dr_icm_chunk_size max_log_chunk_sz;
+ struct mlx5dr_icm_pool *pool;
+ int i;
+
+ if (icm_type == DR_ICM_TYPE_STE)
+ max_log_chunk_sz = dmn->info.max_log_sw_icm_sz;
+ else
+ max_log_chunk_sz = dmn->info.max_log_action_icm_sz;
+
+ pool = kvzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool)
+ return NULL;
+
+ pool->buckets = kcalloc(max_log_chunk_sz + 1,
+ sizeof(pool->buckets[0]),
+ GFP_KERNEL);
+ if (!pool->buckets)
+ goto free_pool;
+
+ pool->dmn = dmn;
+ pool->icm_type = icm_type;
+ pool->max_log_chunk_sz = max_log_chunk_sz;
+ pool->num_of_buckets = max_log_chunk_sz + 1;
+ INIT_LIST_HEAD(&pool->icm_mr_list);
+
+ for (i = 0; i < pool->num_of_buckets; i++)
+ dr_icm_bucket_init(pool, &pool->buckets[i], i);
+
+ mutex_init(&pool->mr_mutex);
+
+ return pool;
+
+free_pool:
+ kvfree(pool);
+ return NULL;
+}
+
+void mlx5dr_icm_pool_destroy(struct mlx5dr_icm_pool *pool)
+{
+ struct mlx5dr_icm_mr *icm_mr, *next;
+ int i;
+
+ mutex_destroy(&pool->mr_mutex);
+
+ list_for_each_entry_safe(icm_mr, next, &pool->icm_mr_list, mr_list)
+ dr_icm_pool_mr_destroy(icm_mr);
+
+ for (i = 0; i < pool->num_of_buckets; i++)
+ dr_icm_bucket_cleanup(&pool->buckets[i]);
+
+ kfree(pool->buckets);
+ kvfree(pool);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c
new file mode 100644
index 0000000..01008cd
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c
@@ -0,0 +1,770 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+static bool dr_mask_is_smac_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->smac_47_16 || spec->smac_15_0);
+}
+
+static bool dr_mask_is_dmac_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->dmac_47_16 || spec->dmac_15_0);
+}
+
+static bool dr_mask_is_src_addr_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->src_ip_127_96 || spec->src_ip_95_64 ||
+ spec->src_ip_63_32 || spec->src_ip_31_0);
+}
+
+static bool dr_mask_is_dst_addr_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->dst_ip_127_96 || spec->dst_ip_95_64 ||
+ spec->dst_ip_63_32 || spec->dst_ip_31_0);
+}
+
+static bool dr_mask_is_l3_base_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->ip_protocol || spec->frag || spec->tcp_flags ||
+ spec->ip_ecn || spec->ip_dscp);
+}
+
+static bool dr_mask_is_tcp_udp_base_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->tcp_sport || spec->tcp_dport ||
+ spec->udp_sport || spec->udp_dport);
+}
+
+static bool dr_mask_is_ipv4_set(struct mlx5dr_match_spec *spec)
+{
+ return (spec->dst_ip_31_0 || spec->src_ip_31_0);
+}
+
+static bool dr_mask_is_ipv4_5_tuple_set(struct mlx5dr_match_spec *spec)
+{
+ return (dr_mask_is_l3_base_set(spec) ||
+ dr_mask_is_tcp_udp_base_set(spec) ||
+ dr_mask_is_ipv4_set(spec));
+}
+
+static bool dr_mask_is_eth_l2_tnl_set(struct mlx5dr_match_misc *misc)
+{
+ return misc->vxlan_vni;
+}
+
+static bool dr_mask_is_ttl_set(struct mlx5dr_match_spec *spec)
+{
+ return spec->ttl_hoplimit;
+}
+
+#define DR_MASK_IS_L2_DST(_spec, _misc, _inner_outer) (_spec.first_vid || \
+ (_spec).first_cfi || (_spec).first_prio || (_spec).cvlan_tag || \
+ (_spec).svlan_tag || (_spec).dmac_47_16 || (_spec).dmac_15_0 || \
+ (_spec).ethertype || (_spec).ip_version || \
+ (_misc)._inner_outer##_second_vid || \
+ (_misc)._inner_outer##_second_cfi || \
+ (_misc)._inner_outer##_second_prio || \
+ (_misc)._inner_outer##_second_cvlan_tag || \
+ (_misc)._inner_outer##_second_svlan_tag)
+
+#define DR_MASK_IS_ETH_L4_SET(_spec, _misc, _inner_outer) ( \
+ dr_mask_is_l3_base_set(&(_spec)) || \
+ dr_mask_is_tcp_udp_base_set(&(_spec)) || \
+ dr_mask_is_ttl_set(&(_spec)) || \
+ (_misc)._inner_outer##_ipv6_flow_label)
+
+#define DR_MASK_IS_ETH_L4_MISC_SET(_misc3, _inner_outer) ( \
+ (_misc3)._inner_outer##_tcp_seq_num || \
+ (_misc3)._inner_outer##_tcp_ack_num)
+
+#define DR_MASK_IS_FIRST_MPLS_SET(_misc2, _inner_outer) ( \
+ (_misc2)._inner_outer##_first_mpls_label || \
+ (_misc2)._inner_outer##_first_mpls_exp || \
+ (_misc2)._inner_outer##_first_mpls_s_bos || \
+ (_misc2)._inner_outer##_first_mpls_ttl)
+
+static bool dr_mask_is_gre_set(struct mlx5dr_match_misc *misc)
+{
+ return (misc->gre_key_h || misc->gre_key_l ||
+ misc->gre_protocol || misc->gre_c_present ||
+ misc->gre_k_present || misc->gre_s_present);
+}
+
+#define DR_MASK_IS_OUTER_MPLS_OVER_GRE_UDP_SET(_misc2, gre_udp) ( \
+ (_misc2).outer_first_mpls_over_##gre_udp##_label || \
+ (_misc2).outer_first_mpls_over_##gre_udp##_exp || \
+ (_misc2).outer_first_mpls_over_##gre_udp##_s_bos || \
+ (_misc2).outer_first_mpls_over_##gre_udp##_ttl)
+
+#define DR_MASK_IS_FLEX_PARSER_0_SET(_misc2) ( \
+ DR_MASK_IS_OUTER_MPLS_OVER_GRE_UDP_SET((_misc2), gre) || \
+ DR_MASK_IS_OUTER_MPLS_OVER_GRE_UDP_SET((_misc2), udp))
+
+static bool dr_mask_is_flex_parser_tnl_set(struct mlx5dr_match_misc3 *misc3)
+{
+ return (misc3->outer_vxlan_gpe_vni ||
+ misc3->outer_vxlan_gpe_next_protocol ||
+ misc3->outer_vxlan_gpe_flags);
+}
+
+static bool dr_mask_is_flex_parser_icmpv6_set(struct mlx5dr_match_misc3 *misc3)
+{
+ return (misc3->icmpv6_type || misc3->icmpv6_code ||
+ misc3->icmpv6_header_data);
+}
+
+static bool dr_mask_is_wqe_metadata_set(struct mlx5dr_match_misc2 *misc2)
+{
+ return misc2->metadata_reg_a;
+}
+
+static bool dr_mask_is_reg_c_0_3_set(struct mlx5dr_match_misc2 *misc2)
+{
+ return (misc2->metadata_reg_c_0 || misc2->metadata_reg_c_1 ||
+ misc2->metadata_reg_c_2 || misc2->metadata_reg_c_3);
+}
+
+static bool dr_mask_is_reg_c_4_7_set(struct mlx5dr_match_misc2 *misc2)
+{
+ return (misc2->metadata_reg_c_4 || misc2->metadata_reg_c_5 ||
+ misc2->metadata_reg_c_6 || misc2->metadata_reg_c_7);
+}
+
+static bool dr_mask_is_gvmi_or_qpn_set(struct mlx5dr_match_misc *misc)
+{
+ return (misc->source_sqn || misc->source_port);
+}
+
+static bool
+dr_matcher_supp_flex_parser_vxlan_gpe(struct mlx5dr_domain *dmn)
+{
+ return dmn->info.caps.flex_protocols &
+ MLX5_FLEX_PARSER_VXLAN_GPE_ENABLED;
+}
+
+int mlx5dr_matcher_select_builders(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ bool ipv6)
+{
+ if (ipv6) {
+ nic_matcher->ste_builder = nic_matcher->ste_builder6;
+ nic_matcher->num_of_builders = nic_matcher->num_of_builders6;
+ } else {
+ nic_matcher->ste_builder = nic_matcher->ste_builder4;
+ nic_matcher->num_of_builders = nic_matcher->num_of_builders4;
+ }
+
+ if (!nic_matcher->num_of_builders) {
+ mlx5dr_dbg(matcher->tbl->dmn,
+ "Rule not supported on this matcher due to IP related fields\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int dr_matcher_set_ste_builders(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ bool ipv6)
+{
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_match_param mask = {};
+ struct mlx5dr_match_misc3 *misc3;
+ struct mlx5dr_ste_build *sb;
+ u8 *num_of_builders;
+ bool inner, rx;
+ int idx = 0;
+ int ret, i;
+
+ if (ipv6) {
+ sb = nic_matcher->ste_builder6;
+ num_of_builders = &nic_matcher->num_of_builders6;
+ } else {
+ sb = nic_matcher->ste_builder4;
+ num_of_builders = &nic_matcher->num_of_builders4;
+ }
+
+ rx = nic_dmn->ste_type == MLX5DR_STE_TYPE_RX;
+
+ /* Create a temporary mask to track and clear used mask fields */
+ if (matcher->match_criteria & DR_MATCHER_CRITERIA_OUTER)
+ mask.outer = matcher->mask.outer;
+
+ if (matcher->match_criteria & DR_MATCHER_CRITERIA_MISC)
+ mask.misc = matcher->mask.misc;
+
+ if (matcher->match_criteria & DR_MATCHER_CRITERIA_INNER)
+ mask.inner = matcher->mask.inner;
+
+ if (matcher->match_criteria & DR_MATCHER_CRITERIA_MISC2)
+ mask.misc2 = matcher->mask.misc2;
+
+ if (matcher->match_criteria & DR_MATCHER_CRITERIA_MISC3)
+ mask.misc3 = matcher->mask.misc3;
+
+ ret = mlx5dr_ste_build_pre_check(dmn, matcher->match_criteria,
+ &matcher->mask, NULL);
+ if (ret)
+ return ret;
+
+ /* Outer */
+ if (matcher->match_criteria & (DR_MATCHER_CRITERIA_OUTER |
+ DR_MATCHER_CRITERIA_MISC |
+ DR_MATCHER_CRITERIA_MISC2 |
+ DR_MATCHER_CRITERIA_MISC3)) {
+ inner = false;
+
+ if (dr_mask_is_wqe_metadata_set(&mask.misc2))
+ mlx5dr_ste_build_general_purpose(&sb[idx++], &mask, inner, rx);
+
+ if (dr_mask_is_reg_c_0_3_set(&mask.misc2))
+ mlx5dr_ste_build_register_0(&sb[idx++], &mask, inner, rx);
+
+ if (dr_mask_is_reg_c_4_7_set(&mask.misc2))
+ mlx5dr_ste_build_register_1(&sb[idx++], &mask, inner, rx);
+
+ if (dr_mask_is_gvmi_or_qpn_set(&mask.misc) &&
+ (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
+ dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX)) {
+ ret = mlx5dr_ste_build_src_gvmi_qpn(&sb[idx++], &mask,
+ &dmn->info.caps,
+ inner, rx);
+ if (ret)
+ return ret;
+ }
+
+ if (dr_mask_is_smac_set(&mask.outer) &&
+ dr_mask_is_dmac_set(&mask.outer)) {
+ ret = mlx5dr_ste_build_eth_l2_src_des(&sb[idx++], &mask,
+ inner, rx);
+ if (ret)
+ return ret;
+ }
+
+ if (dr_mask_is_smac_set(&mask.outer))
+ mlx5dr_ste_build_eth_l2_src(&sb[idx++], &mask, inner, rx);
+
+ if (DR_MASK_IS_L2_DST(mask.outer, mask.misc, outer))
+ mlx5dr_ste_build_eth_l2_dst(&sb[idx++], &mask, inner, rx);
+
+ if (ipv6) {
+ if (dr_mask_is_dst_addr_set(&mask.outer))
+ mlx5dr_ste_build_eth_l3_ipv6_dst(&sb[idx++], &mask,
+ inner, rx);
+
+ if (dr_mask_is_src_addr_set(&mask.outer))
+ mlx5dr_ste_build_eth_l3_ipv6_src(&sb[idx++], &mask,
+ inner, rx);
+
+ if (DR_MASK_IS_ETH_L4_SET(mask.outer, mask.misc, outer))
+ mlx5dr_ste_build_ipv6_l3_l4(&sb[idx++], &mask,
+ inner, rx);
+ } else {
+ if (dr_mask_is_ipv4_5_tuple_set(&mask.outer))
+ mlx5dr_ste_build_eth_l3_ipv4_5_tuple(&sb[idx++], &mask,
+ inner, rx);
+
+ if (dr_mask_is_ttl_set(&mask.outer))
+ mlx5dr_ste_build_eth_l3_ipv4_misc(&sb[idx++], &mask,
+ inner, rx);
+ }
+
+ if (dr_mask_is_flex_parser_tnl_set(&mask.misc3) &&
+ dr_matcher_supp_flex_parser_vxlan_gpe(dmn))
+ mlx5dr_ste_build_flex_parser_tnl(&sb[idx++], &mask,
+ inner, rx);
+
+ if (DR_MASK_IS_ETH_L4_MISC_SET(mask.misc3, outer))
+ mlx5dr_ste_build_eth_l4_misc(&sb[idx++], &mask, inner, rx);
+
+ if (DR_MASK_IS_FIRST_MPLS_SET(mask.misc2, outer))
+ mlx5dr_ste_build_mpls(&sb[idx++], &mask, inner, rx);
+
+ if (DR_MASK_IS_FLEX_PARSER_0_SET(mask.misc2))
+ mlx5dr_ste_build_flex_parser_0(&sb[idx++], &mask,
+ inner, rx);
+
+ misc3 = &mask.misc3;
+ if ((DR_MASK_IS_FLEX_PARSER_ICMPV4_SET(misc3) &&
+ mlx5dr_matcher_supp_flex_parser_icmp_v4(&dmn->info.caps)) ||
+ (dr_mask_is_flex_parser_icmpv6_set(&mask.misc3) &&
+ mlx5dr_matcher_supp_flex_parser_icmp_v6(&dmn->info.caps))) {
+ ret = mlx5dr_ste_build_flex_parser_1(&sb[idx++],
+ &mask, &dmn->info.caps,
+ inner, rx);
+ if (ret)
+ return ret;
+ }
+ if (dr_mask_is_gre_set(&mask.misc))
+ mlx5dr_ste_build_gre(&sb[idx++], &mask, inner, rx);
+ }
+
+ /* Inner */
+ if (matcher->match_criteria & (DR_MATCHER_CRITERIA_INNER |
+ DR_MATCHER_CRITERIA_MISC |
+ DR_MATCHER_CRITERIA_MISC2 |
+ DR_MATCHER_CRITERIA_MISC3)) {
+ inner = true;
+
+ if (dr_mask_is_eth_l2_tnl_set(&mask.misc))
+ mlx5dr_ste_build_eth_l2_tnl(&sb[idx++], &mask, inner, rx);
+
+ if (dr_mask_is_smac_set(&mask.inner) &&
+ dr_mask_is_dmac_set(&mask.inner)) {
+ ret = mlx5dr_ste_build_eth_l2_src_des(&sb[idx++],
+ &mask, inner, rx);
+ if (ret)
+ return ret;
+ }
+
+ if (dr_mask_is_smac_set(&mask.inner))
+ mlx5dr_ste_build_eth_l2_src(&sb[idx++], &mask, inner, rx);
+
+ if (DR_MASK_IS_L2_DST(mask.inner, mask.misc, inner))
+ mlx5dr_ste_build_eth_l2_dst(&sb[idx++], &mask, inner, rx);
+
+ if (ipv6) {
+ if (dr_mask_is_dst_addr_set(&mask.inner))
+ mlx5dr_ste_build_eth_l3_ipv6_dst(&sb[idx++], &mask,
+ inner, rx);
+
+ if (dr_mask_is_src_addr_set(&mask.inner))
+ mlx5dr_ste_build_eth_l3_ipv6_src(&sb[idx++], &mask,
+ inner, rx);
+
+ if (DR_MASK_IS_ETH_L4_SET(mask.inner, mask.misc, inner))
+ mlx5dr_ste_build_ipv6_l3_l4(&sb[idx++], &mask,
+ inner, rx);
+ } else {
+ if (dr_mask_is_ipv4_5_tuple_set(&mask.inner))
+ mlx5dr_ste_build_eth_l3_ipv4_5_tuple(&sb[idx++], &mask,
+ inner, rx);
+
+ if (dr_mask_is_ttl_set(&mask.inner))
+ mlx5dr_ste_build_eth_l3_ipv4_misc(&sb[idx++], &mask,
+ inner, rx);
+ }
+
+ if (DR_MASK_IS_ETH_L4_MISC_SET(mask.misc3, inner))
+ mlx5dr_ste_build_eth_l4_misc(&sb[idx++], &mask, inner, rx);
+
+ if (DR_MASK_IS_FIRST_MPLS_SET(mask.misc2, inner))
+ mlx5dr_ste_build_mpls(&sb[idx++], &mask, inner, rx);
+
+ if (DR_MASK_IS_FLEX_PARSER_0_SET(mask.misc2))
+ mlx5dr_ste_build_flex_parser_0(&sb[idx++], &mask, inner, rx);
+ }
+ /* Empty matcher, takes all */
+ if (matcher->match_criteria == DR_MATCHER_CRITERIA_EMPTY)
+ mlx5dr_ste_build_empty_always_hit(&sb[idx++], rx);
+
+ if (idx == 0) {
+ mlx5dr_dbg(dmn, "Cannot generate any valid rules from mask\n");
+ return -EINVAL;
+ }
+
+ /* Check that all mask fields were consumed */
+ for (i = 0; i < sizeof(struct mlx5dr_match_param); i++) {
+ if (((u8 *)&mask)[i] != 0) {
+ mlx5dr_info(dmn, "Mask contains unsupported parameters\n");
+ return -EOPNOTSUPP;
+ }
+ }
+
+ *num_of_builders = idx;
+
+ return 0;
+}
+
+static int dr_matcher_connect(struct mlx5dr_domain *dmn,
+ struct mlx5dr_matcher_rx_tx *curr_nic_matcher,
+ struct mlx5dr_matcher_rx_tx *next_nic_matcher,
+ struct mlx5dr_matcher_rx_tx *prev_nic_matcher)
+{
+ struct mlx5dr_table_rx_tx *nic_tbl = curr_nic_matcher->nic_tbl;
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_tbl->nic_dmn;
+ struct mlx5dr_htbl_connect_info info;
+ struct mlx5dr_ste_htbl *prev_htbl;
+ int ret;
+
+ /* Connect end anchor hash table to next_htbl or to the default address */
+ if (next_nic_matcher) {
+ info.type = CONNECT_HIT;
+ info.hit_next_htbl = next_nic_matcher->s_htbl;
+ } else {
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = nic_tbl->default_icm_addr;
+ }
+ ret = mlx5dr_ste_htbl_init_and_postsend(dmn, nic_dmn,
+ curr_nic_matcher->e_anchor,
+ &info, info.type == CONNECT_HIT);
+ if (ret)
+ return ret;
+
+ /* Connect start hash table to end anchor */
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = curr_nic_matcher->e_anchor->chunk->icm_addr;
+ ret = mlx5dr_ste_htbl_init_and_postsend(dmn, nic_dmn,
+ curr_nic_matcher->s_htbl,
+ &info, false);
+ if (ret)
+ return ret;
+
+ /* Connect previous hash table to matcher start hash table */
+ if (prev_nic_matcher)
+ prev_htbl = prev_nic_matcher->e_anchor;
+ else
+ prev_htbl = nic_tbl->s_anchor;
+
+ info.type = CONNECT_HIT;
+ info.hit_next_htbl = curr_nic_matcher->s_htbl;
+ ret = mlx5dr_ste_htbl_init_and_postsend(dmn, nic_dmn, prev_htbl,
+ &info, true);
+ if (ret)
+ return ret;
+
+ /* Update the pointing ste and next hash table */
+ curr_nic_matcher->s_htbl->pointing_ste = prev_htbl->ste_arr;
+ prev_htbl->ste_arr[0].next_htbl = curr_nic_matcher->s_htbl;
+
+ if (next_nic_matcher) {
+ next_nic_matcher->s_htbl->pointing_ste = curr_nic_matcher->e_anchor->ste_arr;
+ curr_nic_matcher->e_anchor->ste_arr[0].next_htbl = next_nic_matcher->s_htbl;
+ }
+
+ return 0;
+}
+
+static int dr_matcher_add_to_tbl(struct mlx5dr_matcher *matcher)
+{
+ struct mlx5dr_matcher *next_matcher, *prev_matcher, *tmp_matcher;
+ struct mlx5dr_table *tbl = matcher->tbl;
+ struct mlx5dr_domain *dmn = tbl->dmn;
+ bool first = true;
+ int ret;
+
+ next_matcher = NULL;
+ if (!list_empty(&tbl->matcher_list))
+ list_for_each_entry(tmp_matcher, &tbl->matcher_list, matcher_list) {
+ if (tmp_matcher->prio >= matcher->prio) {
+ next_matcher = tmp_matcher;
+ break;
+ }
+ first = false;
+ }
+
+ prev_matcher = NULL;
+ if (next_matcher && !first)
+ prev_matcher = list_entry(next_matcher->matcher_list.prev,
+ struct mlx5dr_matcher,
+ matcher_list);
+ else if (!first)
+ prev_matcher = list_entry(tbl->matcher_list.prev,
+ struct mlx5dr_matcher,
+ matcher_list);
+
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
+ dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX) {
+ ret = dr_matcher_connect(dmn, &matcher->rx,
+ next_matcher ? &next_matcher->rx : NULL,
+ prev_matcher ? &prev_matcher->rx : NULL);
+ if (ret)
+ return ret;
+ }
+
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
+ dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX) {
+ ret = dr_matcher_connect(dmn, &matcher->tx,
+ next_matcher ? &next_matcher->tx : NULL,
+ prev_matcher ? &prev_matcher->tx : NULL);
+ if (ret)
+ return ret;
+ }
+
+ if (prev_matcher)
+ list_add(&matcher->matcher_list, &prev_matcher->matcher_list);
+ else if (next_matcher)
+ list_add_tail(&matcher->matcher_list,
+ &next_matcher->matcher_list);
+ else
+ list_add(&matcher->matcher_list, &tbl->matcher_list);
+
+ return 0;
+}
+
+static void dr_matcher_uninit_nic(struct mlx5dr_matcher_rx_tx *nic_matcher)
+{
+ mlx5dr_htbl_put(nic_matcher->s_htbl);
+ mlx5dr_htbl_put(nic_matcher->e_anchor);
+}
+
+static void dr_matcher_uninit_fdb(struct mlx5dr_matcher *matcher)
+{
+ dr_matcher_uninit_nic(&matcher->rx);
+ dr_matcher_uninit_nic(&matcher->tx);
+}
+
+static void dr_matcher_uninit(struct mlx5dr_matcher *matcher)
+{
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+
+ switch (dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ dr_matcher_uninit_nic(&matcher->rx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ dr_matcher_uninit_nic(&matcher->tx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ dr_matcher_uninit_fdb(matcher);
+ break;
+ default:
+ WARN_ON(true);
+ break;
+ }
+}
+
+static int dr_matcher_init_nic(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher)
+{
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ int ret, ret_v4, ret_v6;
+
+ ret_v4 = dr_matcher_set_ste_builders(matcher, nic_matcher, false);
+ ret_v6 = dr_matcher_set_ste_builders(matcher, nic_matcher, true);
+
+ if (ret_v4 && ret_v6) {
+ mlx5dr_dbg(dmn, "Cannot generate IPv4 or IPv6 rules with given mask\n");
+ return -EINVAL;
+ }
+
+ if (!ret_v4)
+ nic_matcher->ste_builder = nic_matcher->ste_builder4;
+ else
+ nic_matcher->ste_builder = nic_matcher->ste_builder6;
+
+ nic_matcher->e_anchor = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
+ DR_CHUNK_SIZE_1,
+ MLX5DR_STE_LU_TYPE_DONT_CARE,
+ 0);
+ if (!nic_matcher->e_anchor)
+ return -ENOMEM;
+
+ nic_matcher->s_htbl = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
+ DR_CHUNK_SIZE_1,
+ nic_matcher->ste_builder[0].lu_type,
+ nic_matcher->ste_builder[0].byte_mask);
+ if (!nic_matcher->s_htbl) {
+ ret = -ENOMEM;
+ goto free_e_htbl;
+ }
+
+ /* make sure the tables exist while empty */
+ mlx5dr_htbl_get(nic_matcher->s_htbl);
+ mlx5dr_htbl_get(nic_matcher->e_anchor);
+
+ return 0;
+
+free_e_htbl:
+ mlx5dr_ste_htbl_free(nic_matcher->e_anchor);
+ return ret;
+}
+
+static int dr_matcher_init_fdb(struct mlx5dr_matcher *matcher)
+{
+ int ret;
+
+ ret = dr_matcher_init_nic(matcher, &matcher->rx);
+ if (ret)
+ return ret;
+
+ ret = dr_matcher_init_nic(matcher, &matcher->tx);
+ if (ret)
+ goto uninit_nic_rx;
+
+ return 0;
+
+uninit_nic_rx:
+ dr_matcher_uninit_nic(&matcher->rx);
+ return ret;
+}
+
+static int dr_matcher_init(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *mask)
+{
+ struct mlx5dr_table *tbl = matcher->tbl;
+ struct mlx5dr_domain *dmn = tbl->dmn;
+ int ret;
+
+ if (matcher->match_criteria >= DR_MATCHER_CRITERIA_MAX) {
+ mlx5dr_info(dmn, "Invalid match criteria attribute\n");
+ return -EINVAL;
+ }
+
+ if (mask) {
+ if (mask->match_sz > sizeof(struct mlx5dr_match_param)) {
+ mlx5dr_info(dmn, "Invalid match size attribute\n");
+ return -EINVAL;
+ }
+ mlx5dr_ste_copy_param(matcher->match_criteria,
+ &matcher->mask, mask);
+ }
+
+ switch (dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ matcher->rx.nic_tbl = &tbl->rx;
+ ret = dr_matcher_init_nic(matcher, &matcher->rx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ matcher->tx.nic_tbl = &tbl->tx;
+ ret = dr_matcher_init_nic(matcher, &matcher->tx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ matcher->rx.nic_tbl = &tbl->rx;
+ matcher->tx.nic_tbl = &tbl->tx;
+ ret = dr_matcher_init_fdb(matcher);
+ break;
+ default:
+ WARN_ON(true);
+ return -EINVAL;
+ }
+
+ return ret;
+}
+
+struct mlx5dr_matcher *
+mlx5dr_matcher_create(struct mlx5dr_table *tbl,
+ u16 priority,
+ u8 match_criteria_enable,
+ struct mlx5dr_match_parameters *mask)
+{
+ struct mlx5dr_matcher *matcher;
+ int ret;
+
+ refcount_inc(&tbl->refcount);
+
+ matcher = kzalloc(sizeof(*matcher), GFP_KERNEL);
+ if (!matcher)
+ goto dec_ref;
+
+ matcher->tbl = tbl;
+ matcher->prio = priority;
+ matcher->match_criteria = match_criteria_enable;
+ refcount_set(&matcher->refcount, 1);
+ INIT_LIST_HEAD(&matcher->matcher_list);
+
+ mutex_lock(&tbl->dmn->mutex);
+
+ ret = dr_matcher_init(matcher, mask);
+ if (ret)
+ goto free_matcher;
+
+ ret = dr_matcher_add_to_tbl(matcher);
+ if (ret)
+ goto matcher_uninit;
+
+ mutex_unlock(&tbl->dmn->mutex);
+
+ return matcher;
+
+matcher_uninit:
+ dr_matcher_uninit(matcher);
+free_matcher:
+ mutex_unlock(&tbl->dmn->mutex);
+ kfree(matcher);
+dec_ref:
+ refcount_dec(&tbl->refcount);
+ return NULL;
+}
+
+static int dr_matcher_disconnect(struct mlx5dr_domain *dmn,
+ struct mlx5dr_table_rx_tx *nic_tbl,
+ struct mlx5dr_matcher_rx_tx *next_nic_matcher,
+ struct mlx5dr_matcher_rx_tx *prev_nic_matcher)
+{
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_tbl->nic_dmn;
+ struct mlx5dr_htbl_connect_info info;
+ struct mlx5dr_ste_htbl *prev_anchor;
+
+ if (prev_nic_matcher)
+ prev_anchor = prev_nic_matcher->e_anchor;
+ else
+ prev_anchor = nic_tbl->s_anchor;
+
+ /* Connect previous anchor hash table to next matcher or to the default address */
+ if (next_nic_matcher) {
+ info.type = CONNECT_HIT;
+ info.hit_next_htbl = next_nic_matcher->s_htbl;
+ next_nic_matcher->s_htbl->pointing_ste = prev_anchor->ste_arr;
+ prev_anchor->ste_arr[0].next_htbl = next_nic_matcher->s_htbl;
+ } else {
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = nic_tbl->default_icm_addr;
+ prev_anchor->ste_arr[0].next_htbl = NULL;
+ }
+
+ return mlx5dr_ste_htbl_init_and_postsend(dmn, nic_dmn, prev_anchor,
+ &info, true);
+}
+
+static int dr_matcher_remove_from_tbl(struct mlx5dr_matcher *matcher)
+{
+ struct mlx5dr_matcher *prev_matcher, *next_matcher;
+ struct mlx5dr_table *tbl = matcher->tbl;
+ struct mlx5dr_domain *dmn = tbl->dmn;
+ int ret = 0;
+
+ if (list_is_last(&matcher->matcher_list, &tbl->matcher_list))
+ next_matcher = NULL;
+ else
+ next_matcher = list_next_entry(matcher, matcher_list);
+
+ if (matcher->matcher_list.prev == &tbl->matcher_list)
+ prev_matcher = NULL;
+ else
+ prev_matcher = list_prev_entry(matcher, matcher_list);
+
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
+ dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX) {
+ ret = dr_matcher_disconnect(dmn, &tbl->rx,
+ next_matcher ? &next_matcher->rx : NULL,
+ prev_matcher ? &prev_matcher->rx : NULL);
+ if (ret)
+ return ret;
+ }
+
+ if (dmn->type == MLX5DR_DOMAIN_TYPE_FDB ||
+ dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX) {
+ ret = dr_matcher_disconnect(dmn, &tbl->tx,
+ next_matcher ? &next_matcher->tx : NULL,
+ prev_matcher ? &prev_matcher->tx : NULL);
+ if (ret)
+ return ret;
+ }
+
+ list_del(&matcher->matcher_list);
+
+ return 0;
+}
+
+int mlx5dr_matcher_destroy(struct mlx5dr_matcher *matcher)
+{
+ struct mlx5dr_table *tbl = matcher->tbl;
+
+ if (refcount_read(&matcher->refcount) > 1)
+ return -EBUSY;
+
+ mutex_lock(&tbl->dmn->mutex);
+
+ dr_matcher_remove_from_tbl(matcher);
+ dr_matcher_uninit(matcher);
+ refcount_dec(&matcher->tbl->refcount);
+
+ mutex_unlock(&tbl->dmn->mutex);
+ kfree(matcher);
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c
new file mode 100644
index 0000000..3bc3f66
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c
@@ -0,0 +1,1243 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+#define DR_RULE_MAX_STE_CHAIN (DR_RULE_MAX_STES + DR_ACTION_MAX_STES)
+
+struct mlx5dr_rule_action_member {
+ struct mlx5dr_action *action;
+ struct list_head list;
+};
+
+static int dr_rule_append_to_miss_list(struct mlx5dr_ste *new_last_ste,
+ struct list_head *miss_list,
+ struct list_head *send_list)
+{
+ struct mlx5dr_ste_send_info *ste_info_last;
+ struct mlx5dr_ste *last_ste;
+
+ /* The new entry will be inserted after the last */
+ last_ste = list_entry(miss_list->prev, struct mlx5dr_ste, miss_list_node);
+ WARN_ON(!last_ste);
+
+ ste_info_last = kzalloc(sizeof(*ste_info_last), GFP_KERNEL);
+ if (!ste_info_last)
+ return -ENOMEM;
+
+ mlx5dr_ste_set_miss_addr(last_ste->hw_ste,
+ mlx5dr_ste_get_icm_addr(new_last_ste));
+ list_add_tail(&new_last_ste->miss_list_node, miss_list);
+
+ mlx5dr_send_fill_and_append_ste_send_info(last_ste, DR_STE_SIZE_REDUCED,
+ 0, last_ste->hw_ste,
+ ste_info_last, send_list, true);
+
+ return 0;
+}
+
+static struct mlx5dr_ste *
+dr_rule_create_collision_htbl(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ u8 *hw_ste)
+{
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_ste_htbl *new_htbl;
+ struct mlx5dr_ste *ste;
+
+ /* Create new table for miss entry */
+ new_htbl = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
+ DR_CHUNK_SIZE_1,
+ MLX5DR_STE_LU_TYPE_DONT_CARE,
+ 0);
+ if (!new_htbl) {
+ mlx5dr_dbg(dmn, "Failed allocating collision table\n");
+ return NULL;
+ }
+
+ /* One and only entry, never grows */
+ ste = new_htbl->ste_arr;
+ mlx5dr_ste_set_miss_addr(hw_ste, nic_matcher->e_anchor->chunk->icm_addr);
+ mlx5dr_htbl_get(new_htbl);
+
+ return ste;
+}
+
+static struct mlx5dr_ste *
+dr_rule_create_collision_entry(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ u8 *hw_ste,
+ struct mlx5dr_ste *orig_ste)
+{
+ struct mlx5dr_ste *ste;
+
+ ste = dr_rule_create_collision_htbl(matcher, nic_matcher, hw_ste);
+ if (!ste) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed creating collision entry\n");
+ return NULL;
+ }
+
+ ste->ste_chain_location = orig_ste->ste_chain_location;
+
+ /* In collision entry, all members share the same miss_list_head */
+ ste->htbl->miss_list = mlx5dr_ste_get_miss_list(orig_ste);
+
+ /* Next table */
+ if (mlx5dr_ste_create_next_htbl(matcher, nic_matcher, ste, hw_ste,
+ DR_CHUNK_SIZE_1)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed allocating table\n");
+ goto free_tbl;
+ }
+
+ return ste;
+
+free_tbl:
+ mlx5dr_ste_free(ste, matcher, nic_matcher);
+ return NULL;
+}
+
+static int
+dr_rule_handle_one_ste_in_update_list(struct mlx5dr_ste_send_info *ste_info,
+ struct mlx5dr_domain *dmn)
+{
+ int ret;
+
+ list_del(&ste_info->send_list);
+ ret = mlx5dr_send_postsend_ste(dmn, ste_info->ste, ste_info->data,
+ ste_info->size, ste_info->offset);
+ if (ret)
+ goto out;
+ /* Copy data to ste, only reduced size, the last 16B (mask)
+ * is already written to the hw.
+ */
+ memcpy(ste_info->ste->hw_ste, ste_info->data, DR_STE_SIZE_REDUCED);
+
+out:
+ kfree(ste_info);
+ return ret;
+}
+
+static int dr_rule_send_update_list(struct list_head *send_ste_list,
+ struct mlx5dr_domain *dmn,
+ bool is_reverse)
+{
+ struct mlx5dr_ste_send_info *ste_info, *tmp_ste_info;
+ int ret;
+
+ if (is_reverse) {
+ list_for_each_entry_safe_reverse(ste_info, tmp_ste_info,
+ send_ste_list, send_list) {
+ ret = dr_rule_handle_one_ste_in_update_list(ste_info,
+ dmn);
+ if (ret)
+ return ret;
+ }
+ } else {
+ list_for_each_entry_safe(ste_info, tmp_ste_info,
+ send_ste_list, send_list) {
+ ret = dr_rule_handle_one_ste_in_update_list(ste_info,
+ dmn);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static struct mlx5dr_ste *
+dr_rule_find_ste_in_miss_list(struct list_head *miss_list, u8 *hw_ste)
+{
+ struct mlx5dr_ste *ste;
+
+ if (list_empty(miss_list))
+ return NULL;
+
+ /* Check if hw_ste is present in the list */
+ list_for_each_entry(ste, miss_list, miss_list_node) {
+ if (mlx5dr_ste_equal_tag(ste->hw_ste, hw_ste))
+ return ste;
+ }
+
+ return NULL;
+}
+
+static struct mlx5dr_ste *
+dr_rule_rehash_handle_collision(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct list_head *update_list,
+ struct mlx5dr_ste *col_ste,
+ u8 *hw_ste)
+{
+ struct mlx5dr_ste *new_ste;
+ int ret;
+
+ new_ste = dr_rule_create_collision_htbl(matcher, nic_matcher, hw_ste);
+ if (!new_ste)
+ return NULL;
+
+ /* In collision entry, all members share the same miss_list_head */
+ new_ste->htbl->miss_list = mlx5dr_ste_get_miss_list(col_ste);
+
+ /* Update the previous from the list */
+ ret = dr_rule_append_to_miss_list(new_ste,
+ mlx5dr_ste_get_miss_list(col_ste),
+ update_list);
+ if (ret) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed update dup entry\n");
+ goto err_exit;
+ }
+
+ return new_ste;
+
+err_exit:
+ mlx5dr_ste_free(new_ste, matcher, nic_matcher);
+ return NULL;
+}
+
+static void dr_rule_rehash_copy_ste_ctrl(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste *cur_ste,
+ struct mlx5dr_ste *new_ste)
+{
+ new_ste->next_htbl = cur_ste->next_htbl;
+ new_ste->ste_chain_location = cur_ste->ste_chain_location;
+
+ if (!mlx5dr_ste_is_last_in_rule(nic_matcher, new_ste->ste_chain_location))
+ new_ste->next_htbl->pointing_ste = new_ste;
+
+ /* We need to copy the refcount since this ste
+ * may have been traversed several times
+ */
+ refcount_set(&new_ste->refcount, refcount_read(&cur_ste->refcount));
+
+ /* Link old STEs rule_mem list to the new ste */
+ mlx5dr_rule_update_rule_member(cur_ste, new_ste);
+ INIT_LIST_HEAD(&new_ste->rule_list);
+ list_splice_tail_init(&cur_ste->rule_list, &new_ste->rule_list);
+}
+
+static struct mlx5dr_ste *
+dr_rule_rehash_copy_ste(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste *cur_ste,
+ struct mlx5dr_ste_htbl *new_htbl,
+ struct list_head *update_list)
+{
+ struct mlx5dr_ste_send_info *ste_info;
+ bool use_update_list = false;
+ u8 hw_ste[DR_STE_SIZE] = {};
+ struct mlx5dr_ste *new_ste;
+ int new_idx;
+ u8 sb_idx;
+
+ /* Copy STE mask from the matcher */
+ sb_idx = cur_ste->ste_chain_location - 1;
+ mlx5dr_ste_set_bit_mask(hw_ste, nic_matcher->ste_builder[sb_idx].bit_mask);
+
+ /* Copy STE control and tag */
+ memcpy(hw_ste, cur_ste->hw_ste, DR_STE_SIZE_REDUCED);
+ mlx5dr_ste_set_miss_addr(hw_ste, nic_matcher->e_anchor->chunk->icm_addr);
+
+ new_idx = mlx5dr_ste_calc_hash_index(hw_ste, new_htbl);
+ new_ste = &new_htbl->ste_arr[new_idx];
+
+ if (mlx5dr_ste_not_used_ste(new_ste)) {
+ mlx5dr_htbl_get(new_htbl);
+ list_add_tail(&new_ste->miss_list_node,
+ mlx5dr_ste_get_miss_list(new_ste));
+ } else {
+ new_ste = dr_rule_rehash_handle_collision(matcher,
+ nic_matcher,
+ update_list,
+ new_ste,
+ hw_ste);
+ if (!new_ste) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed adding collision entry, index: %d\n",
+ new_idx);
+ return NULL;
+ }
+ new_htbl->ctrl.num_of_collisions++;
+ use_update_list = true;
+ }
+
+ memcpy(new_ste->hw_ste, hw_ste, DR_STE_SIZE_REDUCED);
+
+ new_htbl->ctrl.num_of_valid_entries++;
+
+ if (use_update_list) {
+ ste_info = kzalloc(sizeof(*ste_info), GFP_KERNEL);
+ if (!ste_info)
+ goto err_exit;
+
+ mlx5dr_send_fill_and_append_ste_send_info(new_ste, DR_STE_SIZE, 0,
+ hw_ste, ste_info,
+ update_list, true);
+ }
+
+ dr_rule_rehash_copy_ste_ctrl(matcher, nic_matcher, cur_ste, new_ste);
+
+ return new_ste;
+
+err_exit:
+ mlx5dr_ste_free(new_ste, matcher, nic_matcher);
+ return NULL;
+}
+
+static int dr_rule_rehash_copy_miss_list(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct list_head *cur_miss_list,
+ struct mlx5dr_ste_htbl *new_htbl,
+ struct list_head *update_list)
+{
+ struct mlx5dr_ste *tmp_ste, *cur_ste, *new_ste;
+
+ if (list_empty(cur_miss_list))
+ return 0;
+
+ list_for_each_entry_safe(cur_ste, tmp_ste, cur_miss_list, miss_list_node) {
+ new_ste = dr_rule_rehash_copy_ste(matcher,
+ nic_matcher,
+ cur_ste,
+ new_htbl,
+ update_list);
+ if (!new_ste)
+ goto err_insert;
+
+ list_del(&cur_ste->miss_list_node);
+ mlx5dr_htbl_put(cur_ste->htbl);
+ }
+ return 0;
+
+err_insert:
+ mlx5dr_err(matcher->tbl->dmn, "Fatal error during resize\n");
+ WARN_ON(true);
+ return -EINVAL;
+}
+
+static int dr_rule_rehash_copy_htbl(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste_htbl *cur_htbl,
+ struct mlx5dr_ste_htbl *new_htbl,
+ struct list_head *update_list)
+{
+ struct mlx5dr_ste *cur_ste;
+ int cur_entries;
+ int err = 0;
+ int i;
+
+ cur_entries = mlx5dr_icm_pool_chunk_size_to_entries(cur_htbl->chunk_size);
+
+ if (cur_entries < 1) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Invalid number of entries\n");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < cur_entries; i++) {
+ cur_ste = &cur_htbl->ste_arr[i];
+ if (mlx5dr_ste_not_used_ste(cur_ste)) /* Empty, nothing to copy */
+ continue;
+
+ err = dr_rule_rehash_copy_miss_list(matcher,
+ nic_matcher,
+ mlx5dr_ste_get_miss_list(cur_ste),
+ new_htbl,
+ update_list);
+ if (err)
+ goto clean_copy;
+ }
+
+clean_copy:
+ return err;
+}
+
+static struct mlx5dr_ste_htbl *
+dr_rule_rehash_htbl(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule,
+ struct mlx5dr_ste_htbl *cur_htbl,
+ u8 ste_location,
+ struct list_head *update_list,
+ enum mlx5dr_icm_chunk_size new_size)
+{
+ struct mlx5dr_ste_send_info *del_ste_info, *tmp_ste_info;
+ struct mlx5dr_matcher *matcher = rule->matcher;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_matcher_rx_tx *nic_matcher;
+ struct mlx5dr_ste_send_info *ste_info;
+ struct mlx5dr_htbl_connect_info info;
+ struct mlx5dr_domain_rx_tx *nic_dmn;
+ u8 formatted_ste[DR_STE_SIZE] = {};
+ LIST_HEAD(rehash_table_send_list);
+ struct mlx5dr_ste *ste_to_update;
+ struct mlx5dr_ste_htbl *new_htbl;
+ int err;
+
+ nic_matcher = nic_rule->nic_matcher;
+ nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+
+ ste_info = kzalloc(sizeof(*ste_info), GFP_KERNEL);
+ if (!ste_info)
+ return NULL;
+
+ new_htbl = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
+ new_size,
+ cur_htbl->lu_type,
+ cur_htbl->byte_mask);
+ if (!new_htbl) {
+ mlx5dr_err(dmn, "Failed to allocate new hash table\n");
+ goto free_ste_info;
+ }
+
+ /* Write new table to HW */
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = nic_matcher->e_anchor->chunk->icm_addr;
+ mlx5dr_ste_set_formatted_ste(dmn->info.caps.gvmi,
+ nic_dmn,
+ new_htbl,
+ formatted_ste,
+ &info);
+
+ new_htbl->pointing_ste = cur_htbl->pointing_ste;
+ new_htbl->pointing_ste->next_htbl = new_htbl;
+ err = dr_rule_rehash_copy_htbl(matcher,
+ nic_matcher,
+ cur_htbl,
+ new_htbl,
+ &rehash_table_send_list);
+ if (err)
+ goto free_new_htbl;
+
+ if (mlx5dr_send_postsend_htbl(dmn, new_htbl, formatted_ste,
+ nic_matcher->ste_builder[ste_location - 1].bit_mask)) {
+ mlx5dr_err(dmn, "Failed writing table to HW\n");
+ goto free_new_htbl;
+ }
+
+ /* Writing to the hw is done in regular order of rehash_table_send_list,
+ * in order to have the origin data written before the miss address of
+ * collision entries, if exists.
+ */
+ if (dr_rule_send_update_list(&rehash_table_send_list, dmn, false)) {
+ mlx5dr_err(dmn, "Failed updating table to HW\n");
+ goto free_ste_list;
+ }
+
+ /* Connect previous hash table to current */
+ if (ste_location == 1) {
+ /* The previous table is an anchor, anchors size is always one STE */
+ struct mlx5dr_ste_htbl *prev_htbl = cur_htbl->pointing_ste->htbl;
+
+ /* On matcher s_anchor we keep an extra refcount */
+ mlx5dr_htbl_get(new_htbl);
+ mlx5dr_htbl_put(cur_htbl);
+
+ nic_matcher->s_htbl = new_htbl;
+
+ /* It is safe to operate dr_ste_set_hit_addr on the hw_ste here
+ * (48B len) which works only on first 32B
+ */
+ mlx5dr_ste_set_hit_addr(prev_htbl->ste_arr[0].hw_ste,
+ new_htbl->chunk->icm_addr,
+ new_htbl->chunk->num_of_entries);
+
+ ste_to_update = &prev_htbl->ste_arr[0];
+ } else {
+ mlx5dr_ste_set_hit_addr_by_next_htbl(cur_htbl->pointing_ste->hw_ste,
+ new_htbl);
+ ste_to_update = cur_htbl->pointing_ste;
+ }
+
+ mlx5dr_send_fill_and_append_ste_send_info(ste_to_update, DR_STE_SIZE_REDUCED,
+ 0, ste_to_update->hw_ste, ste_info,
+ update_list, false);
+
+ return new_htbl;
+
+free_ste_list:
+ /* Clean all ste_info's from the new table */
+ list_for_each_entry_safe(del_ste_info, tmp_ste_info,
+ &rehash_table_send_list, send_list) {
+ list_del(&del_ste_info->send_list);
+ kfree(del_ste_info);
+ }
+
+free_new_htbl:
+ mlx5dr_ste_htbl_free(new_htbl);
+free_ste_info:
+ kfree(ste_info);
+ mlx5dr_info(dmn, "Failed creating rehash table\n");
+ return NULL;
+}
+
+static struct mlx5dr_ste_htbl *dr_rule_rehash(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule,
+ struct mlx5dr_ste_htbl *cur_htbl,
+ u8 ste_location,
+ struct list_head *update_list)
+{
+ struct mlx5dr_domain *dmn = rule->matcher->tbl->dmn;
+ enum mlx5dr_icm_chunk_size new_size;
+
+ new_size = mlx5dr_icm_next_higher_chunk(cur_htbl->chunk_size);
+ new_size = min_t(u32, new_size, dmn->info.max_log_sw_icm_sz);
+
+ if (new_size == cur_htbl->chunk_size)
+ return NULL; /* Skip rehash, we already at the max size */
+
+ return dr_rule_rehash_htbl(rule, nic_rule, cur_htbl, ste_location,
+ update_list, new_size);
+}
+
+static struct mlx5dr_ste *
+dr_rule_handle_collision(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste *ste,
+ u8 *hw_ste,
+ struct list_head *miss_list,
+ struct list_head *send_list)
+{
+ struct mlx5dr_ste_send_info *ste_info;
+ struct mlx5dr_ste *new_ste;
+
+ ste_info = kzalloc(sizeof(*ste_info), GFP_KERNEL);
+ if (!ste_info)
+ return NULL;
+
+ new_ste = dr_rule_create_collision_entry(matcher, nic_matcher, hw_ste, ste);
+ if (!new_ste)
+ goto free_send_info;
+
+ if (dr_rule_append_to_miss_list(new_ste, miss_list, send_list)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed to update prev miss_list\n");
+ goto err_exit;
+ }
+
+ mlx5dr_send_fill_and_append_ste_send_info(new_ste, DR_STE_SIZE, 0, hw_ste,
+ ste_info, send_list, false);
+
+ ste->htbl->ctrl.num_of_collisions++;
+ ste->htbl->ctrl.num_of_valid_entries++;
+
+ return new_ste;
+
+err_exit:
+ mlx5dr_ste_free(new_ste, matcher, nic_matcher);
+free_send_info:
+ kfree(ste_info);
+ return NULL;
+}
+
+static void dr_rule_remove_action_members(struct mlx5dr_rule *rule)
+{
+ struct mlx5dr_rule_action_member *action_mem;
+ struct mlx5dr_rule_action_member *tmp;
+
+ list_for_each_entry_safe(action_mem, tmp, &rule->rule_actions_list, list) {
+ list_del(&action_mem->list);
+ refcount_dec(&action_mem->action->refcount);
+ kvfree(action_mem);
+ }
+}
+
+static int dr_rule_add_action_members(struct mlx5dr_rule *rule,
+ size_t num_actions,
+ struct mlx5dr_action *actions[])
+{
+ struct mlx5dr_rule_action_member *action_mem;
+ int i;
+
+ for (i = 0; i < num_actions; i++) {
+ action_mem = kvzalloc(sizeof(*action_mem), GFP_KERNEL);
+ if (!action_mem)
+ goto free_action_members;
+
+ action_mem->action = actions[i];
+ INIT_LIST_HEAD(&action_mem->list);
+ list_add_tail(&action_mem->list, &rule->rule_actions_list);
+ refcount_inc(&action_mem->action->refcount);
+ }
+
+ return 0;
+
+free_action_members:
+ dr_rule_remove_action_members(rule);
+ return -ENOMEM;
+}
+
+/* While the pointer of ste is no longer valid, like while moving ste to be
+ * the first in the miss_list, and to be in the origin table,
+ * all rule-members that are attached to this ste should update their ste member
+ * to the new pointer
+ */
+void mlx5dr_rule_update_rule_member(struct mlx5dr_ste *ste,
+ struct mlx5dr_ste *new_ste)
+{
+ struct mlx5dr_rule_member *rule_mem;
+
+ if (!list_empty(&ste->rule_list))
+ list_for_each_entry(rule_mem, &ste->rule_list, use_ste_list)
+ rule_mem->ste = new_ste;
+}
+
+static void dr_rule_clean_rule_members(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule)
+{
+ struct mlx5dr_rule_member *rule_mem;
+ struct mlx5dr_rule_member *tmp_mem;
+
+ if (list_empty(&nic_rule->rule_members_list))
+ return;
+ list_for_each_entry_safe(rule_mem, tmp_mem, &nic_rule->rule_members_list, list) {
+ list_del(&rule_mem->list);
+ list_del(&rule_mem->use_ste_list);
+ mlx5dr_ste_put(rule_mem->ste, rule->matcher, nic_rule->nic_matcher);
+ kvfree(rule_mem);
+ }
+}
+
+static bool dr_rule_need_enlarge_hash(struct mlx5dr_ste_htbl *htbl,
+ struct mlx5dr_domain *dmn,
+ struct mlx5dr_domain_rx_tx *nic_dmn)
+{
+ struct mlx5dr_ste_htbl_ctrl *ctrl = &htbl->ctrl;
+
+ if (dmn->info.max_log_sw_icm_sz <= htbl->chunk_size)
+ return false;
+
+ if (!ctrl->may_grow)
+ return false;
+
+ if (ctrl->num_of_collisions >= ctrl->increase_threshold &&
+ (ctrl->num_of_valid_entries - ctrl->num_of_collisions) >= ctrl->increase_threshold)
+ return true;
+
+ return false;
+}
+
+static int dr_rule_add_member(struct mlx5dr_rule_rx_tx *nic_rule,
+ struct mlx5dr_ste *ste)
+{
+ struct mlx5dr_rule_member *rule_mem;
+
+ rule_mem = kvzalloc(sizeof(*rule_mem), GFP_KERNEL);
+ if (!rule_mem)
+ return -ENOMEM;
+
+ rule_mem->ste = ste;
+ list_add_tail(&rule_mem->list, &nic_rule->rule_members_list);
+
+ list_add_tail(&rule_mem->use_ste_list, &ste->rule_list);
+
+ return 0;
+}
+
+static int dr_rule_handle_action_stes(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule,
+ struct list_head *send_ste_list,
+ struct mlx5dr_ste *last_ste,
+ u8 *hw_ste_arr,
+ u32 new_hw_ste_arr_sz)
+{
+ struct mlx5dr_matcher_rx_tx *nic_matcher = nic_rule->nic_matcher;
+ struct mlx5dr_ste_send_info *ste_info_arr[DR_ACTION_MAX_STES];
+ u8 num_of_builders = nic_matcher->num_of_builders;
+ struct mlx5dr_matcher *matcher = rule->matcher;
+ u8 *curr_hw_ste, *prev_hw_ste;
+ struct mlx5dr_ste *action_ste;
+ int i, k, ret;
+
+ /* Two cases:
+ * 1. num_of_builders is equal to new_hw_ste_arr_sz, the action in the ste
+ * 2. num_of_builders is less then new_hw_ste_arr_sz, new ste was added
+ * to support the action.
+ */
+ if (num_of_builders == new_hw_ste_arr_sz)
+ return 0;
+
+ for (i = num_of_builders, k = 0; i < new_hw_ste_arr_sz; i++, k++) {
+ curr_hw_ste = hw_ste_arr + i * DR_STE_SIZE;
+ prev_hw_ste = (i == 0) ? curr_hw_ste : hw_ste_arr + ((i - 1) * DR_STE_SIZE);
+ action_ste = dr_rule_create_collision_htbl(matcher,
+ nic_matcher,
+ curr_hw_ste);
+ if (!action_ste)
+ return -ENOMEM;
+
+ mlx5dr_ste_get(action_ste);
+
+ /* While free ste we go over the miss list, so add this ste to the list */
+ list_add_tail(&action_ste->miss_list_node,
+ mlx5dr_ste_get_miss_list(action_ste));
+
+ ste_info_arr[k] = kzalloc(sizeof(*ste_info_arr[k]),
+ GFP_KERNEL);
+ if (!ste_info_arr[k])
+ goto err_exit;
+
+ /* Point current ste to the new action */
+ mlx5dr_ste_set_hit_addr_by_next_htbl(prev_hw_ste, action_ste->htbl);
+ ret = dr_rule_add_member(nic_rule, action_ste);
+ if (ret) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed adding rule member\n");
+ goto free_ste_info;
+ }
+ mlx5dr_send_fill_and_append_ste_send_info(action_ste, DR_STE_SIZE, 0,
+ curr_hw_ste,
+ ste_info_arr[k],
+ send_ste_list, false);
+ }
+
+ return 0;
+
+free_ste_info:
+ kfree(ste_info_arr[k]);
+err_exit:
+ mlx5dr_ste_put(action_ste, matcher, nic_matcher);
+ return -ENOMEM;
+}
+
+static int dr_rule_handle_empty_entry(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste_htbl *cur_htbl,
+ struct mlx5dr_ste *ste,
+ u8 ste_location,
+ u8 *hw_ste,
+ struct list_head *miss_list,
+ struct list_head *send_list)
+{
+ struct mlx5dr_ste_send_info *ste_info;
+
+ /* Take ref on table, only on first time this ste is used */
+ mlx5dr_htbl_get(cur_htbl);
+
+ /* new entry -> new branch */
+ list_add_tail(&ste->miss_list_node, miss_list);
+
+ mlx5dr_ste_set_miss_addr(hw_ste, nic_matcher->e_anchor->chunk->icm_addr);
+
+ ste->ste_chain_location = ste_location;
+
+ ste_info = kzalloc(sizeof(*ste_info), GFP_KERNEL);
+ if (!ste_info)
+ goto clean_ste_setting;
+
+ if (mlx5dr_ste_create_next_htbl(matcher,
+ nic_matcher,
+ ste,
+ hw_ste,
+ DR_CHUNK_SIZE_1)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Failed allocating table\n");
+ goto clean_ste_info;
+ }
+
+ cur_htbl->ctrl.num_of_valid_entries++;
+
+ mlx5dr_send_fill_and_append_ste_send_info(ste, DR_STE_SIZE, 0, hw_ste,
+ ste_info, send_list, false);
+
+ return 0;
+
+clean_ste_info:
+ kfree(ste_info);
+clean_ste_setting:
+ list_del_init(&ste->miss_list_node);
+ mlx5dr_htbl_put(cur_htbl);
+
+ return -ENOMEM;
+}
+
+static struct mlx5dr_ste *
+dr_rule_handle_ste_branch(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule,
+ struct list_head *send_ste_list,
+ struct mlx5dr_ste_htbl *cur_htbl,
+ u8 *hw_ste,
+ u8 ste_location,
+ struct mlx5dr_ste_htbl **put_htbl)
+{
+ struct mlx5dr_matcher *matcher = rule->matcher;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_matcher_rx_tx *nic_matcher;
+ struct mlx5dr_domain_rx_tx *nic_dmn;
+ struct mlx5dr_ste_htbl *new_htbl;
+ struct mlx5dr_ste *matched_ste;
+ struct list_head *miss_list;
+ bool skip_rehash = false;
+ struct mlx5dr_ste *ste;
+ int index;
+
+ nic_matcher = nic_rule->nic_matcher;
+ nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+
+again:
+ index = mlx5dr_ste_calc_hash_index(hw_ste, cur_htbl);
+ miss_list = &cur_htbl->chunk->miss_list[index];
+ ste = &cur_htbl->ste_arr[index];
+
+ if (mlx5dr_ste_not_used_ste(ste)) {
+ if (dr_rule_handle_empty_entry(matcher, nic_matcher, cur_htbl,
+ ste, ste_location,
+ hw_ste, miss_list,
+ send_ste_list))
+ return NULL;
+ } else {
+ /* Hash table index in use, check if this ste is in the miss list */
+ matched_ste = dr_rule_find_ste_in_miss_list(miss_list, hw_ste);
+ if (matched_ste) {
+ /* If it is last STE in the chain, and has the same tag
+ * it means that all the previous stes are the same,
+ * if so, this rule is duplicated.
+ */
+ if (mlx5dr_ste_is_last_in_rule(nic_matcher,
+ matched_ste->ste_chain_location)) {
+ mlx5dr_info(dmn, "Duplicate rule inserted, aborting!!\n");
+ return NULL;
+ }
+ return matched_ste;
+ }
+
+ if (!skip_rehash && dr_rule_need_enlarge_hash(cur_htbl, dmn, nic_dmn)) {
+ /* Hash table index in use, try to resize of the hash */
+ skip_rehash = true;
+
+ /* Hold the table till we update.
+ * Release in dr_rule_create_rule()
+ */
+ *put_htbl = cur_htbl;
+ mlx5dr_htbl_get(cur_htbl);
+
+ new_htbl = dr_rule_rehash(rule, nic_rule, cur_htbl,
+ ste_location, send_ste_list);
+ if (!new_htbl) {
+ mlx5dr_htbl_put(cur_htbl);
+ mlx5dr_info(dmn, "failed creating rehash table, htbl-log_size: %d\n",
+ cur_htbl->chunk_size);
+ } else {
+ cur_htbl = new_htbl;
+ }
+ goto again;
+ } else {
+ /* Hash table index in use, add another collision (miss) */
+ ste = dr_rule_handle_collision(matcher,
+ nic_matcher,
+ ste,
+ hw_ste,
+ miss_list,
+ send_ste_list);
+ if (!ste) {
+ mlx5dr_dbg(dmn, "failed adding collision entry, index: %d\n",
+ index);
+ return NULL;
+ }
+ }
+ }
+ return ste;
+}
+
+static bool dr_rule_cmp_value_to_mask(u8 *mask, u8 *value,
+ u32 s_idx, u32 e_idx)
+{
+ u32 i;
+
+ for (i = s_idx; i < e_idx; i++) {
+ if (value[i] & ~mask[i]) {
+ pr_info("Rule parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
+ return true;
+}
+
+static bool dr_rule_verify(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *value,
+ struct mlx5dr_match_param *param)
+{
+ u8 match_criteria = matcher->match_criteria;
+ size_t value_size = value->match_sz;
+ u8 *mask_p = (u8 *)&matcher->mask;
+ u8 *param_p = (u8 *)param;
+ u32 s_idx, e_idx;
+
+ if (!value_size ||
+ (value_size > sizeof(struct mlx5dr_match_param) ||
+ (value_size % sizeof(u32)))) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Rule parameters length is incorrect\n");
+ return false;
+ }
+
+ mlx5dr_ste_copy_param(matcher->match_criteria, param, value);
+
+ if (match_criteria & DR_MATCHER_CRITERIA_OUTER) {
+ s_idx = offsetof(struct mlx5dr_match_param, outer);
+ e_idx = min(s_idx + sizeof(param->outer), value_size);
+
+ if (!dr_rule_cmp_value_to_mask(mask_p, param_p, s_idx, e_idx)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Rule outer parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC) {
+ s_idx = offsetof(struct mlx5dr_match_param, misc);
+ e_idx = min(s_idx + sizeof(param->misc), value_size);
+
+ if (!dr_rule_cmp_value_to_mask(mask_p, param_p, s_idx, e_idx)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Rule misc parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
+
+ if (match_criteria & DR_MATCHER_CRITERIA_INNER) {
+ s_idx = offsetof(struct mlx5dr_match_param, inner);
+ e_idx = min(s_idx + sizeof(param->inner), value_size);
+
+ if (!dr_rule_cmp_value_to_mask(mask_p, param_p, s_idx, e_idx)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Rule inner parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC2) {
+ s_idx = offsetof(struct mlx5dr_match_param, misc2);
+ e_idx = min(s_idx + sizeof(param->misc2), value_size);
+
+ if (!dr_rule_cmp_value_to_mask(mask_p, param_p, s_idx, e_idx)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Rule misc2 parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC3) {
+ s_idx = offsetof(struct mlx5dr_match_param, misc3);
+ e_idx = min(s_idx + sizeof(param->misc3), value_size);
+
+ if (!dr_rule_cmp_value_to_mask(mask_p, param_p, s_idx, e_idx)) {
+ mlx5dr_dbg(matcher->tbl->dmn, "Rule misc3 parameters contains a value not specified by mask\n");
+ return false;
+ }
+ }
+ return true;
+}
+
+static int dr_rule_destroy_rule_nic(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule)
+{
+ dr_rule_clean_rule_members(rule, nic_rule);
+ return 0;
+}
+
+static int dr_rule_destroy_rule_fdb(struct mlx5dr_rule *rule)
+{
+ dr_rule_destroy_rule_nic(rule, &rule->rx);
+ dr_rule_destroy_rule_nic(rule, &rule->tx);
+ return 0;
+}
+
+static int dr_rule_destroy_rule(struct mlx5dr_rule *rule)
+{
+ struct mlx5dr_domain *dmn = rule->matcher->tbl->dmn;
+
+ switch (dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ dr_rule_destroy_rule_nic(rule, &rule->rx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ dr_rule_destroy_rule_nic(rule, &rule->tx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ dr_rule_destroy_rule_fdb(rule);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ dr_rule_remove_action_members(rule);
+ kfree(rule);
+ return 0;
+}
+
+static bool dr_rule_is_ipv6(struct mlx5dr_match_param *param)
+{
+ return (param->outer.ip_version == 6 ||
+ param->inner.ip_version == 6 ||
+ param->outer.ethertype == ETH_P_IPV6 ||
+ param->inner.ethertype == ETH_P_IPV6);
+}
+
+static bool dr_rule_skip(enum mlx5dr_domain_type domain,
+ enum mlx5dr_ste_entry_type ste_type,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_match_param *value)
+{
+ if (domain != MLX5DR_DOMAIN_TYPE_FDB)
+ return false;
+
+ if (mask->misc.source_port) {
+ if (ste_type == MLX5DR_STE_TYPE_RX)
+ if (value->misc.source_port != WIRE_PORT)
+ return true;
+
+ if (ste_type == MLX5DR_STE_TYPE_TX)
+ if (value->misc.source_port == WIRE_PORT)
+ return true;
+ }
+
+ /* Metadata C can be used to describe the source vport */
+ if (mask->misc2.metadata_reg_c_0) {
+ if (ste_type == MLX5DR_STE_TYPE_RX)
+ if ((value->misc2.metadata_reg_c_0 & WIRE_PORT) != WIRE_PORT)
+ return true;
+
+ if (ste_type == MLX5DR_STE_TYPE_TX)
+ if ((value->misc2.metadata_reg_c_0 & WIRE_PORT) == WIRE_PORT)
+ return true;
+ }
+ return false;
+}
+
+static int
+dr_rule_create_rule_nic(struct mlx5dr_rule *rule,
+ struct mlx5dr_rule_rx_tx *nic_rule,
+ struct mlx5dr_match_param *param,
+ size_t num_actions,
+ struct mlx5dr_action *actions[])
+{
+ struct mlx5dr_ste_send_info *ste_info, *tmp_ste_info;
+ struct mlx5dr_matcher *matcher = rule->matcher;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_matcher_rx_tx *nic_matcher;
+ struct mlx5dr_domain_rx_tx *nic_dmn;
+ struct mlx5dr_ste_htbl *htbl = NULL;
+ struct mlx5dr_ste_htbl *cur_htbl;
+ struct mlx5dr_ste *ste = NULL;
+ LIST_HEAD(send_ste_list);
+ u8 *hw_ste_arr = NULL;
+ u32 new_hw_ste_arr_sz;
+ int ret, i;
+
+ nic_matcher = nic_rule->nic_matcher;
+ nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+
+ INIT_LIST_HEAD(&nic_rule->rule_members_list);
+
+ if (dr_rule_skip(dmn->type, nic_dmn->ste_type, &matcher->mask, param))
+ return 0;
+
+ ret = mlx5dr_matcher_select_builders(matcher,
+ nic_matcher,
+ dr_rule_is_ipv6(param));
+ if (ret)
+ goto out_err;
+
+ hw_ste_arr = kzalloc(DR_RULE_MAX_STE_CHAIN * DR_STE_SIZE, GFP_KERNEL);
+ if (!hw_ste_arr) {
+ ret = -ENOMEM;
+ goto out_err;
+ }
+
+ /* Set the tag values inside the ste array */
+ ret = mlx5dr_ste_build_ste_arr(matcher, nic_matcher, param, hw_ste_arr);
+ if (ret)
+ goto free_hw_ste;
+
+ /* Set the actions values/addresses inside the ste array */
+ ret = mlx5dr_actions_build_ste_arr(matcher, nic_matcher, actions,
+ num_actions, hw_ste_arr,
+ &new_hw_ste_arr_sz);
+ if (ret)
+ goto free_hw_ste;
+
+ cur_htbl = nic_matcher->s_htbl;
+
+ /* Go over the array of STEs, and build dr_ste accordingly.
+ * The loop is over only the builders which are equal or less to the
+ * number of stes, in case we have actions that lives in other stes.
+ */
+ for (i = 0; i < nic_matcher->num_of_builders; i++) {
+ /* Calculate CRC and keep new ste entry */
+ u8 *cur_hw_ste_ent = hw_ste_arr + (i * DR_STE_SIZE);
+
+ ste = dr_rule_handle_ste_branch(rule,
+ nic_rule,
+ &send_ste_list,
+ cur_htbl,
+ cur_hw_ste_ent,
+ i + 1,
+ &htbl);
+ if (!ste) {
+ mlx5dr_err(dmn, "Failed creating next branch\n");
+ ret = -ENOENT;
+ goto free_rule;
+ }
+
+ cur_htbl = ste->next_htbl;
+
+ /* Keep all STEs in the rule struct */
+ ret = dr_rule_add_member(nic_rule, ste);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Failed adding rule member index %d\n", i);
+ goto free_ste;
+ }
+
+ mlx5dr_ste_get(ste);
+ }
+
+ /* Connect actions */
+ ret = dr_rule_handle_action_stes(rule, nic_rule, &send_ste_list,
+ ste, hw_ste_arr, new_hw_ste_arr_sz);
+ if (ret) {
+ mlx5dr_dbg(dmn, "Failed apply actions\n");
+ goto free_rule;
+ }
+ ret = dr_rule_send_update_list(&send_ste_list, dmn, true);
+ if (ret) {
+ mlx5dr_err(dmn, "Failed sending ste!\n");
+ goto free_rule;
+ }
+
+ if (htbl)
+ mlx5dr_htbl_put(htbl);
+
+ return 0;
+
+free_ste:
+ mlx5dr_ste_put(ste, matcher, nic_matcher);
+free_rule:
+ dr_rule_clean_rule_members(rule, nic_rule);
+ /* Clean all ste_info's */
+ list_for_each_entry_safe(ste_info, tmp_ste_info, &send_ste_list, send_list) {
+ list_del(&ste_info->send_list);
+ kfree(ste_info);
+ }
+free_hw_ste:
+ kfree(hw_ste_arr);
+out_err:
+ return ret;
+}
+
+static int
+dr_rule_create_rule_fdb(struct mlx5dr_rule *rule,
+ struct mlx5dr_match_param *param,
+ size_t num_actions,
+ struct mlx5dr_action *actions[])
+{
+ struct mlx5dr_match_param copy_param = {};
+ int ret;
+
+ /* Copy match_param since they will be consumed during the first
+ * nic_rule insertion.
+ */
+ memcpy(©_param, param, sizeof(struct mlx5dr_match_param));
+
+ ret = dr_rule_create_rule_nic(rule, &rule->rx, param,
+ num_actions, actions);
+ if (ret)
+ return ret;
+
+ ret = dr_rule_create_rule_nic(rule, &rule->tx, ©_param,
+ num_actions, actions);
+ if (ret)
+ goto destroy_rule_nic_rx;
+
+ return 0;
+
+destroy_rule_nic_rx:
+ dr_rule_destroy_rule_nic(rule, &rule->rx);
+ return ret;
+}
+
+static struct mlx5dr_rule *
+dr_rule_create_rule(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *value,
+ size_t num_actions,
+ struct mlx5dr_action *actions[])
+{
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_match_param param = {};
+ struct mlx5dr_rule *rule;
+ int ret;
+
+ if (!dr_rule_verify(matcher, value, ¶m))
+ return NULL;
+
+ rule = kzalloc(sizeof(*rule), GFP_KERNEL);
+ if (!rule)
+ return NULL;
+
+ rule->matcher = matcher;
+ INIT_LIST_HEAD(&rule->rule_actions_list);
+
+ ret = dr_rule_add_action_members(rule, num_actions, actions);
+ if (ret)
+ goto free_rule;
+
+ switch (dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ rule->rx.nic_matcher = &matcher->rx;
+ ret = dr_rule_create_rule_nic(rule, &rule->rx, ¶m,
+ num_actions, actions);
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ rule->tx.nic_matcher = &matcher->tx;
+ ret = dr_rule_create_rule_nic(rule, &rule->tx, ¶m,
+ num_actions, actions);
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ rule->rx.nic_matcher = &matcher->rx;
+ rule->tx.nic_matcher = &matcher->tx;
+ ret = dr_rule_create_rule_fdb(rule, ¶m,
+ num_actions, actions);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ if (ret)
+ goto remove_action_members;
+
+ return rule;
+
+remove_action_members:
+ dr_rule_remove_action_members(rule);
+free_rule:
+ kfree(rule);
+ mlx5dr_info(dmn, "Failed creating rule\n");
+ return NULL;
+}
+
+struct mlx5dr_rule *mlx5dr_rule_create(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *value,
+ size_t num_actions,
+ struct mlx5dr_action *actions[])
+{
+ struct mlx5dr_rule *rule;
+
+ mutex_lock(&matcher->tbl->dmn->mutex);
+ refcount_inc(&matcher->refcount);
+
+ rule = dr_rule_create_rule(matcher, value, num_actions, actions);
+ if (!rule)
+ refcount_dec(&matcher->refcount);
+
+ mutex_unlock(&matcher->tbl->dmn->mutex);
+
+ return rule;
+}
+
+int mlx5dr_rule_destroy(struct mlx5dr_rule *rule)
+{
+ struct mlx5dr_matcher *matcher = rule->matcher;
+ struct mlx5dr_table *tbl = rule->matcher->tbl;
+ int ret;
+
+ mutex_lock(&tbl->dmn->mutex);
+
+ ret = dr_rule_destroy_rule(rule);
+
+ mutex_unlock(&tbl->dmn->mutex);
+
+ if (!ret)
+ refcount_dec(&matcher->refcount);
+ return ret;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
new file mode 100644
index 0000000..5df8436
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c
@@ -0,0 +1,975 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+#define QUEUE_SIZE 128
+#define SIGNAL_PER_DIV_QUEUE 16
+#define TH_NUMS_TO_DRAIN 2
+
+enum { CQ_OK = 0, CQ_EMPTY = -1, CQ_POLL_ERR = -2 };
+
+struct dr_data_seg {
+ u64 addr;
+ u32 length;
+ u32 lkey;
+ unsigned int send_flags;
+};
+
+struct postsend_info {
+ struct dr_data_seg write;
+ struct dr_data_seg read;
+ u64 remote_addr;
+ u32 rkey;
+};
+
+struct dr_qp_rtr_attr {
+ struct mlx5dr_cmd_gid_attr dgid_attr;
+ enum ib_mtu mtu;
+ u32 qp_num;
+ u16 port_num;
+ u8 min_rnr_timer;
+ u8 sgid_index;
+ u16 udp_src_port;
+};
+
+struct dr_qp_rts_attr {
+ u8 timeout;
+ u8 retry_cnt;
+ u8 rnr_retry;
+};
+
+struct dr_qp_init_attr {
+ u32 cqn;
+ u32 pdn;
+ u32 max_send_wr;
+ struct mlx5_uars_page *uar;
+};
+
+static int dr_parse_cqe(struct mlx5dr_cq *dr_cq, struct mlx5_cqe64 *cqe64)
+{
+ unsigned int idx;
+ u8 opcode;
+
+ opcode = get_cqe_opcode(cqe64);
+ if (opcode == MLX5_CQE_REQ_ERR) {
+ idx = be16_to_cpu(cqe64->wqe_counter) &
+ (dr_cq->qp->sq.wqe_cnt - 1);
+ dr_cq->qp->sq.cc = dr_cq->qp->sq.wqe_head[idx] + 1;
+ } else if (opcode == MLX5_CQE_RESP_ERR) {
+ ++dr_cq->qp->sq.cc;
+ } else {
+ idx = be16_to_cpu(cqe64->wqe_counter) &
+ (dr_cq->qp->sq.wqe_cnt - 1);
+ dr_cq->qp->sq.cc = dr_cq->qp->sq.wqe_head[idx] + 1;
+
+ return CQ_OK;
+ }
+
+ return CQ_POLL_ERR;
+}
+
+static int dr_cq_poll_one(struct mlx5dr_cq *dr_cq)
+{
+ struct mlx5_cqe64 *cqe64;
+ int err;
+
+ cqe64 = mlx5_cqwq_get_cqe(&dr_cq->wq);
+ if (!cqe64)
+ return CQ_EMPTY;
+
+ mlx5_cqwq_pop(&dr_cq->wq);
+ err = dr_parse_cqe(dr_cq, cqe64);
+ mlx5_cqwq_update_db_record(&dr_cq->wq);
+
+ return err;
+}
+
+static int dr_poll_cq(struct mlx5dr_cq *dr_cq, int ne)
+{
+ int npolled;
+ int err = 0;
+
+ for (npolled = 0; npolled < ne; ++npolled) {
+ err = dr_cq_poll_one(dr_cq);
+ if (err != CQ_OK)
+ break;
+ }
+
+ return err == CQ_POLL_ERR ? err : npolled;
+}
+
+static void dr_qp_event(struct mlx5_core_qp *mqp, int event)
+{
+ pr_info("DR QP event %u on QP #%u\n", event, mqp->qpn);
+}
+
+static struct mlx5dr_qp *dr_create_rc_qp(struct mlx5_core_dev *mdev,
+ struct dr_qp_init_attr *attr)
+{
+ u32 temp_qpc[MLX5_ST_SZ_DW(qpc)] = {};
+ struct mlx5_wq_param wqp;
+ struct mlx5dr_qp *dr_qp;
+ int inlen;
+ void *qpc;
+ void *in;
+ int err;
+
+ dr_qp = kzalloc(sizeof(*dr_qp), GFP_KERNEL);
+ if (!dr_qp)
+ return NULL;
+
+ wqp.buf_numa_node = mdev->priv.numa_node;
+ wqp.db_numa_node = mdev->priv.numa_node;
+
+ dr_qp->rq.pc = 0;
+ dr_qp->rq.cc = 0;
+ dr_qp->rq.wqe_cnt = 4;
+ dr_qp->sq.pc = 0;
+ dr_qp->sq.cc = 0;
+ dr_qp->sq.wqe_cnt = roundup_pow_of_two(attr->max_send_wr);
+
+ MLX5_SET(qpc, temp_qpc, log_rq_stride, ilog2(MLX5_SEND_WQE_DS) - 4);
+ MLX5_SET(qpc, temp_qpc, log_rq_size, ilog2(dr_qp->rq.wqe_cnt));
+ MLX5_SET(qpc, temp_qpc, log_sq_size, ilog2(dr_qp->sq.wqe_cnt));
+ err = mlx5_wq_qp_create(mdev, &wqp, temp_qpc, &dr_qp->wq,
+ &dr_qp->wq_ctrl);
+ if (err) {
+ mlx5_core_info(mdev, "Can't create QP WQ\n");
+ goto err_wq;
+ }
+
+ dr_qp->sq.wqe_head = kcalloc(dr_qp->sq.wqe_cnt,
+ sizeof(dr_qp->sq.wqe_head[0]),
+ GFP_KERNEL);
+
+ if (!dr_qp->sq.wqe_head) {
+ mlx5_core_warn(mdev, "Can't allocate wqe head\n");
+ goto err_wqe_head;
+ }
+
+ inlen = MLX5_ST_SZ_BYTES(create_qp_in) +
+ MLX5_FLD_SZ_BYTES(create_qp_in, pas[0]) *
+ dr_qp->wq_ctrl.buf.npages;
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_in;
+ }
+
+ qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
+ MLX5_SET(qpc, qpc, st, MLX5_QP_ST_RC);
+ MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
+ MLX5_SET(qpc, qpc, pd, attr->pdn);
+ MLX5_SET(qpc, qpc, uar_page, attr->uar->index);
+ MLX5_SET(qpc, qpc, log_page_size,
+ dr_qp->wq_ctrl.buf.page_shift - MLX5_ADAPTER_PAGE_SHIFT);
+ MLX5_SET(qpc, qpc, fre, 1);
+ MLX5_SET(qpc, qpc, rlky, 1);
+ MLX5_SET(qpc, qpc, cqn_snd, attr->cqn);
+ MLX5_SET(qpc, qpc, cqn_rcv, attr->cqn);
+ MLX5_SET(qpc, qpc, log_rq_stride, ilog2(MLX5_SEND_WQE_DS) - 4);
+ MLX5_SET(qpc, qpc, log_rq_size, ilog2(dr_qp->rq.wqe_cnt));
+ MLX5_SET(qpc, qpc, rq_type, MLX5_NON_ZERO_RQ);
+ MLX5_SET(qpc, qpc, log_sq_size, ilog2(dr_qp->sq.wqe_cnt));
+ MLX5_SET64(qpc, qpc, dbr_addr, dr_qp->wq_ctrl.db.dma);
+ if (MLX5_CAP_GEN(mdev, cqe_version) == 1)
+ MLX5_SET(qpc, qpc, user_index, 0xFFFFFF);
+ mlx5_fill_page_frag_array(&dr_qp->wq_ctrl.buf,
+ (__be64 *)MLX5_ADDR_OF(create_qp_in,
+ in, pas));
+
+ err = mlx5_core_create_qp(mdev, &dr_qp->mqp, in, inlen);
+ kfree(in);
+
+ if (err) {
+ mlx5_core_warn(mdev, " Can't create QP\n");
+ goto err_in;
+ }
+ dr_qp->mqp.event = dr_qp_event;
+ dr_qp->uar = attr->uar;
+
+ return dr_qp;
+
+err_in:
+ kfree(dr_qp->sq.wqe_head);
+err_wqe_head:
+ mlx5_wq_destroy(&dr_qp->wq_ctrl);
+err_wq:
+ kfree(dr_qp);
+ return NULL;
+}
+
+static void dr_destroy_qp(struct mlx5_core_dev *mdev,
+ struct mlx5dr_qp *dr_qp)
+{
+ mlx5_core_destroy_qp(mdev, &dr_qp->mqp);
+ kfree(dr_qp->sq.wqe_head);
+ mlx5_wq_destroy(&dr_qp->wq_ctrl);
+ kfree(dr_qp);
+}
+
+static void dr_cmd_notify_hw(struct mlx5dr_qp *dr_qp, void *ctrl)
+{
+ dma_wmb();
+ *dr_qp->wq.sq.db = cpu_to_be32(dr_qp->sq.pc & 0xfffff);
+
+ /* After wmb() the hw aware of new work */
+ wmb();
+
+ mlx5_write64(ctrl, dr_qp->uar->map + MLX5_BF_OFFSET);
+}
+
+static void dr_rdma_segments(struct mlx5dr_qp *dr_qp, u64 remote_addr,
+ u32 rkey, struct dr_data_seg *data_seg,
+ u32 opcode, int nreq)
+{
+ struct mlx5_wqe_raddr_seg *wq_raddr;
+ struct mlx5_wqe_ctrl_seg *wq_ctrl;
+ struct mlx5_wqe_data_seg *wq_dseg;
+ unsigned int size;
+ unsigned int idx;
+
+ size = sizeof(*wq_ctrl) / 16 + sizeof(*wq_dseg) / 16 +
+ sizeof(*wq_raddr) / 16;
+
+ idx = dr_qp->sq.pc & (dr_qp->sq.wqe_cnt - 1);
+
+ wq_ctrl = mlx5_wq_cyc_get_wqe(&dr_qp->wq.sq, idx);
+ wq_ctrl->imm = 0;
+ wq_ctrl->fm_ce_se = (data_seg->send_flags) ?
+ MLX5_WQE_CTRL_CQ_UPDATE : 0;
+ wq_ctrl->opmod_idx_opcode = cpu_to_be32(((dr_qp->sq.pc & 0xffff) << 8) |
+ opcode);
+ wq_ctrl->qpn_ds = cpu_to_be32(size | dr_qp->mqp.qpn << 8);
+ wq_raddr = (void *)(wq_ctrl + 1);
+ wq_raddr->raddr = cpu_to_be64(remote_addr);
+ wq_raddr->rkey = cpu_to_be32(rkey);
+ wq_raddr->reserved = 0;
+
+ wq_dseg = (void *)(wq_raddr + 1);
+ wq_dseg->byte_count = cpu_to_be32(data_seg->length);
+ wq_dseg->lkey = cpu_to_be32(data_seg->lkey);
+ wq_dseg->addr = cpu_to_be64(data_seg->addr);
+
+ dr_qp->sq.wqe_head[idx] = dr_qp->sq.pc++;
+
+ if (nreq)
+ dr_cmd_notify_hw(dr_qp, wq_ctrl);
+}
+
+static void dr_post_send(struct mlx5dr_qp *dr_qp, struct postsend_info *send_info)
+{
+ dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
+ &send_info->write, MLX5_OPCODE_RDMA_WRITE, 0);
+ dr_rdma_segments(dr_qp, send_info->remote_addr, send_info->rkey,
+ &send_info->read, MLX5_OPCODE_RDMA_READ, 1);
+}
+
+/**
+ * mlx5dr_send_fill_and_append_ste_send_info: Add data to be sent
+ * with send_list parameters:
+ *
+ * @ste: The data that attached to this specific ste
+ * @size: of data to write
+ * @offset: of the data from start of the hw_ste entry
+ * @data: data
+ * @ste_info: ste to be sent with send_list
+ * @send_list: to append into it
+ * @copy_data: if true indicates that the data should be kept because
+ * it's not backuped any where (like in re-hash).
+ * if false, it lets the data to be updated after
+ * it was added to the list.
+ */
+void mlx5dr_send_fill_and_append_ste_send_info(struct mlx5dr_ste *ste, u16 size,
+ u16 offset, u8 *data,
+ struct mlx5dr_ste_send_info *ste_info,
+ struct list_head *send_list,
+ bool copy_data)
+{
+ ste_info->size = size;
+ ste_info->ste = ste;
+ ste_info->offset = offset;
+
+ if (copy_data) {
+ memcpy(ste_info->data_cont, data, size);
+ ste_info->data = ste_info->data_cont;
+ } else {
+ ste_info->data = data;
+ }
+
+ list_add_tail(&ste_info->send_list, send_list);
+}
+
+/* The function tries to consume one wc each time, unless the queue is full, in
+ * that case, which means that the hw is behind the sw in a full queue len
+ * the function will drain the cq till it empty.
+ */
+static int dr_handle_pending_wc(struct mlx5dr_domain *dmn,
+ struct mlx5dr_send_ring *send_ring)
+{
+ bool is_drain = false;
+ int ne;
+
+ if (send_ring->pending_wqe < send_ring->signal_th)
+ return 0;
+
+ /* Queue is full start drain it */
+ if (send_ring->pending_wqe >=
+ dmn->send_ring->signal_th * TH_NUMS_TO_DRAIN)
+ is_drain = true;
+
+ do {
+ ne = dr_poll_cq(send_ring->cq, 1);
+ if (ne < 0)
+ return ne;
+ else if (ne == 1)
+ send_ring->pending_wqe -= send_ring->signal_th;
+ } while (is_drain && send_ring->pending_wqe);
+
+ return 0;
+}
+
+static void dr_fill_data_segs(struct mlx5dr_send_ring *send_ring,
+ struct postsend_info *send_info)
+{
+ send_ring->pending_wqe++;
+
+ if (send_ring->pending_wqe % send_ring->signal_th == 0)
+ send_info->write.send_flags |= IB_SEND_SIGNALED;
+
+ send_ring->pending_wqe++;
+ send_info->read.length = send_info->write.length;
+ /* Read into the same write area */
+ send_info->read.addr = (uintptr_t)send_info->write.addr;
+ send_info->read.lkey = send_ring->mr->mkey.key;
+
+ if (send_ring->pending_wqe % send_ring->signal_th == 0)
+ send_info->read.send_flags = IB_SEND_SIGNALED;
+ else
+ send_info->read.send_flags = 0;
+}
+
+static int dr_postsend_icm_data(struct mlx5dr_domain *dmn,
+ struct postsend_info *send_info)
+{
+ struct mlx5dr_send_ring *send_ring = dmn->send_ring;
+ u32 buff_offset;
+ int ret;
+
+ ret = dr_handle_pending_wc(dmn, send_ring);
+ if (ret)
+ return ret;
+
+ if (send_info->write.length > dmn->info.max_inline_size) {
+ buff_offset = (send_ring->tx_head &
+ (dmn->send_ring->signal_th - 1)) *
+ send_ring->max_post_send_size;
+ /* Copy to ring mr */
+ memcpy(send_ring->buf + buff_offset,
+ (void *)(uintptr_t)send_info->write.addr,
+ send_info->write.length);
+ send_info->write.addr = (uintptr_t)send_ring->mr->dma_addr + buff_offset;
+ send_info->write.lkey = send_ring->mr->mkey.key;
+ }
+
+ send_ring->tx_head++;
+ dr_fill_data_segs(send_ring, send_info);
+ dr_post_send(send_ring->qp, send_info);
+
+ return 0;
+}
+
+static int dr_get_tbl_copy_details(struct mlx5dr_domain *dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 **data,
+ u32 *byte_size,
+ int *iterations,
+ int *num_stes)
+{
+ int alloc_size;
+
+ if (htbl->chunk->byte_size > dmn->send_ring->max_post_send_size) {
+ *iterations = htbl->chunk->byte_size /
+ dmn->send_ring->max_post_send_size;
+ *byte_size = dmn->send_ring->max_post_send_size;
+ alloc_size = *byte_size;
+ *num_stes = *byte_size / DR_STE_SIZE;
+ } else {
+ *iterations = 1;
+ *num_stes = htbl->chunk->num_of_entries;
+ alloc_size = *num_stes * DR_STE_SIZE;
+ }
+
+ *data = kzalloc(alloc_size, GFP_KERNEL);
+ if (!*data)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * mlx5dr_send_postsend_ste: write size bytes into offset from the hw cm.
+ *
+ * @dmn: Domain
+ * @ste: The ste struct that contains the data (at
+ * least part of it)
+ * @data: The real data to send size data
+ * @size: for writing.
+ * @offset: The offset from the icm mapped data to
+ * start write to this for write only part of the
+ * buffer.
+ *
+ * Return: 0 on success.
+ */
+int mlx5dr_send_postsend_ste(struct mlx5dr_domain *dmn, struct mlx5dr_ste *ste,
+ u8 *data, u16 size, u16 offset)
+{
+ struct postsend_info send_info = {};
+
+ send_info.write.addr = (uintptr_t)data;
+ send_info.write.length = size;
+ send_info.write.lkey = 0;
+ send_info.remote_addr = mlx5dr_ste_get_mr_addr(ste) + offset;
+ send_info.rkey = ste->htbl->chunk->rkey;
+
+ return dr_postsend_icm_data(dmn, &send_info);
+}
+
+int mlx5dr_send_postsend_htbl(struct mlx5dr_domain *dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 *formatted_ste, u8 *mask)
+{
+ u32 byte_size = htbl->chunk->byte_size;
+ int num_stes_per_iter;
+ int iterations;
+ u8 *data;
+ int ret;
+ int i;
+ int j;
+
+ ret = dr_get_tbl_copy_details(dmn, htbl, &data, &byte_size,
+ &iterations, &num_stes_per_iter);
+ if (ret)
+ return ret;
+
+ /* Send the data iteration times */
+ for (i = 0; i < iterations; i++) {
+ u32 ste_index = i * (byte_size / DR_STE_SIZE);
+ struct postsend_info send_info = {};
+
+ /* Copy all ste's on the data buffer
+ * need to add the bit_mask
+ */
+ for (j = 0; j < num_stes_per_iter; j++) {
+ u8 *hw_ste = htbl->ste_arr[ste_index + j].hw_ste;
+ u32 ste_off = j * DR_STE_SIZE;
+
+ if (mlx5dr_ste_is_not_valid_entry(hw_ste)) {
+ memcpy(data + ste_off,
+ formatted_ste, DR_STE_SIZE);
+ } else {
+ /* Copy data */
+ memcpy(data + ste_off,
+ htbl->ste_arr[ste_index + j].hw_ste,
+ DR_STE_SIZE_REDUCED);
+ /* Copy bit_mask */
+ memcpy(data + ste_off + DR_STE_SIZE_REDUCED,
+ mask, DR_STE_SIZE_MASK);
+ }
+ }
+
+ send_info.write.addr = (uintptr_t)data;
+ send_info.write.length = byte_size;
+ send_info.write.lkey = 0;
+ send_info.remote_addr =
+ mlx5dr_ste_get_mr_addr(htbl->ste_arr + ste_index);
+ send_info.rkey = htbl->chunk->rkey;
+
+ ret = dr_postsend_icm_data(dmn, &send_info);
+ if (ret)
+ goto out_free;
+ }
+
+out_free:
+ kfree(data);
+ return ret;
+}
+
+/* Initialize htble with default STEs */
+int mlx5dr_send_postsend_formatted_htbl(struct mlx5dr_domain *dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 *ste_init_data,
+ bool update_hw_ste)
+{
+ u32 byte_size = htbl->chunk->byte_size;
+ int iterations;
+ int num_stes;
+ u8 *data;
+ int ret;
+ int i;
+
+ ret = dr_get_tbl_copy_details(dmn, htbl, &data, &byte_size,
+ &iterations, &num_stes);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < num_stes; i++) {
+ u8 *copy_dst;
+
+ /* Copy the same ste on the data buffer */
+ copy_dst = data + i * DR_STE_SIZE;
+ memcpy(copy_dst, ste_init_data, DR_STE_SIZE);
+
+ if (update_hw_ste) {
+ /* Copy the reduced ste to hash table ste_arr */
+ copy_dst = htbl->hw_ste_arr + i * DR_STE_SIZE_REDUCED;
+ memcpy(copy_dst, ste_init_data, DR_STE_SIZE_REDUCED);
+ }
+ }
+
+ /* Send the data iteration times */
+ for (i = 0; i < iterations; i++) {
+ u8 ste_index = i * (byte_size / DR_STE_SIZE);
+ struct postsend_info send_info = {};
+
+ send_info.write.addr = (uintptr_t)data;
+ send_info.write.length = byte_size;
+ send_info.write.lkey = 0;
+ send_info.remote_addr =
+ mlx5dr_ste_get_mr_addr(htbl->ste_arr + ste_index);
+ send_info.rkey = htbl->chunk->rkey;
+
+ ret = dr_postsend_icm_data(dmn, &send_info);
+ if (ret)
+ goto out_free;
+ }
+
+out_free:
+ kfree(data);
+ return ret;
+}
+
+int mlx5dr_send_postsend_action(struct mlx5dr_domain *dmn,
+ struct mlx5dr_action *action)
+{
+ struct postsend_info send_info = {};
+ int ret;
+
+ send_info.write.addr = (uintptr_t)action->rewrite.data;
+ send_info.write.length = action->rewrite.chunk->byte_size;
+ send_info.write.lkey = 0;
+ send_info.remote_addr = action->rewrite.chunk->mr_addr;
+ send_info.rkey = action->rewrite.chunk->rkey;
+
+ mutex_lock(&dmn->mutex);
+ ret = dr_postsend_icm_data(dmn, &send_info);
+ mutex_unlock(&dmn->mutex);
+
+ return ret;
+}
+
+static int dr_modify_qp_rst2init(struct mlx5_core_dev *mdev,
+ struct mlx5dr_qp *dr_qp,
+ int port)
+{
+ u32 in[MLX5_ST_SZ_DW(rst2init_qp_in)] = {};
+ void *qpc;
+
+ qpc = MLX5_ADDR_OF(rst2init_qp_in, in, qpc);
+
+ MLX5_SET(qpc, qpc, primary_address_path.vhca_port_num, port);
+ MLX5_SET(qpc, qpc, pm_state, MLX5_QPC_PM_STATE_MIGRATED);
+ MLX5_SET(qpc, qpc, rre, 1);
+ MLX5_SET(qpc, qpc, rwe, 1);
+
+ return mlx5_core_qp_modify(mdev, MLX5_CMD_OP_RST2INIT_QP, 0, qpc,
+ &dr_qp->mqp);
+}
+
+static int dr_cmd_modify_qp_rtr2rts(struct mlx5_core_dev *mdev,
+ struct mlx5dr_qp *dr_qp,
+ struct dr_qp_rts_attr *attr)
+{
+ u32 in[MLX5_ST_SZ_DW(rtr2rts_qp_in)] = {};
+ void *qpc;
+
+ qpc = MLX5_ADDR_OF(rtr2rts_qp_in, in, qpc);
+
+ MLX5_SET(rtr2rts_qp_in, in, qpn, dr_qp->mqp.qpn);
+
+ MLX5_SET(qpc, qpc, log_ack_req_freq, 0);
+ MLX5_SET(qpc, qpc, retry_count, attr->retry_cnt);
+ MLX5_SET(qpc, qpc, rnr_retry, attr->rnr_retry);
+
+ return mlx5_core_qp_modify(mdev, MLX5_CMD_OP_RTR2RTS_QP, 0, qpc,
+ &dr_qp->mqp);
+}
+
+static int dr_cmd_modify_qp_init2rtr(struct mlx5_core_dev *mdev,
+ struct mlx5dr_qp *dr_qp,
+ struct dr_qp_rtr_attr *attr)
+{
+ u32 in[MLX5_ST_SZ_DW(init2rtr_qp_in)] = {};
+ void *qpc;
+
+ qpc = MLX5_ADDR_OF(init2rtr_qp_in, in, qpc);
+
+ MLX5_SET(init2rtr_qp_in, in, qpn, dr_qp->mqp.qpn);
+
+ MLX5_SET(qpc, qpc, mtu, attr->mtu);
+ MLX5_SET(qpc, qpc, log_msg_max, DR_CHUNK_SIZE_MAX - 1);
+ MLX5_SET(qpc, qpc, remote_qpn, attr->qp_num);
+ memcpy(MLX5_ADDR_OF(qpc, qpc, primary_address_path.rmac_47_32),
+ attr->dgid_attr.mac, sizeof(attr->dgid_attr.mac));
+ memcpy(MLX5_ADDR_OF(qpc, qpc, primary_address_path.rgid_rip),
+ attr->dgid_attr.gid, sizeof(attr->dgid_attr.gid));
+ MLX5_SET(qpc, qpc, primary_address_path.src_addr_index,
+ attr->sgid_index);
+
+ if (attr->dgid_attr.roce_ver == MLX5_ROCE_VERSION_2)
+ MLX5_SET(qpc, qpc, primary_address_path.udp_sport,
+ attr->udp_src_port);
+
+ MLX5_SET(qpc, qpc, primary_address_path.vhca_port_num, attr->port_num);
+ MLX5_SET(qpc, qpc, min_rnr_nak, 1);
+
+ return mlx5_core_qp_modify(mdev, MLX5_CMD_OP_INIT2RTR_QP, 0, qpc,
+ &dr_qp->mqp);
+}
+
+static int dr_prepare_qp_to_rts(struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_qp *dr_qp = dmn->send_ring->qp;
+ struct dr_qp_rts_attr rts_attr = {};
+ struct dr_qp_rtr_attr rtr_attr = {};
+ enum ib_mtu mtu = IB_MTU_1024;
+ u16 gid_index = 0;
+ int port = 1;
+ int ret;
+
+ /* Init */
+ ret = dr_modify_qp_rst2init(dmn->mdev, dr_qp, port);
+ if (ret)
+ return ret;
+
+ /* RTR */
+ ret = mlx5dr_cmd_query_gid(dmn->mdev, port, gid_index, &rtr_attr.dgid_attr);
+ if (ret)
+ return ret;
+
+ rtr_attr.mtu = mtu;
+ rtr_attr.qp_num = dr_qp->mqp.qpn;
+ rtr_attr.min_rnr_timer = 12;
+ rtr_attr.port_num = port;
+ rtr_attr.sgid_index = gid_index;
+ rtr_attr.udp_src_port = dmn->info.caps.roce_min_src_udp;
+
+ ret = dr_cmd_modify_qp_init2rtr(dmn->mdev, dr_qp, &rtr_attr);
+ if (ret)
+ return ret;
+
+ /* RTS */
+ rts_attr.timeout = 14;
+ rts_attr.retry_cnt = 7;
+ rts_attr.rnr_retry = 7;
+
+ ret = dr_cmd_modify_qp_rtr2rts(dmn->mdev, dr_qp, &rts_attr);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void dr_cq_event(struct mlx5_core_cq *mcq,
+ enum mlx5_event event)
+{
+ pr_info("CQ event %u on CQ #%u\n", event, mcq->cqn);
+}
+
+static struct mlx5dr_cq *dr_create_cq(struct mlx5_core_dev *mdev,
+ struct mlx5_uars_page *uar,
+ size_t ncqe)
+{
+ u32 temp_cqc[MLX5_ST_SZ_DW(cqc)] = {};
+ u32 out[MLX5_ST_SZ_DW(create_cq_out)];
+ struct mlx5_wq_param wqp;
+ struct mlx5_cqe64 *cqe;
+ struct mlx5dr_cq *cq;
+ int inlen, err, eqn;
+ unsigned int irqn;
+ void *cqc, *in;
+ __be64 *pas;
+ u32 i;
+
+ cq = kzalloc(sizeof(*cq), GFP_KERNEL);
+ if (!cq)
+ return NULL;
+
+ ncqe = roundup_pow_of_two(ncqe);
+ MLX5_SET(cqc, temp_cqc, log_cq_size, ilog2(ncqe));
+
+ wqp.buf_numa_node = mdev->priv.numa_node;
+ wqp.db_numa_node = mdev->priv.numa_node;
+
+ err = mlx5_cqwq_create(mdev, &wqp, temp_cqc, &cq->wq,
+ &cq->wq_ctrl);
+ if (err)
+ goto out;
+
+ for (i = 0; i < mlx5_cqwq_get_size(&cq->wq); i++) {
+ cqe = mlx5_cqwq_get_wqe(&cq->wq, i);
+ cqe->op_own = MLX5_CQE_INVALID << 4 | MLX5_CQE_OWNER_MASK;
+ }
+
+ inlen = MLX5_ST_SZ_BYTES(create_cq_in) +
+ sizeof(u64) * cq->wq_ctrl.buf.npages;
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ goto err_cqwq;
+
+ err = mlx5_vector2eqn(mdev, smp_processor_id(), &eqn, &irqn);
+ if (err) {
+ kvfree(in);
+ goto err_cqwq;
+ }
+
+ cqc = MLX5_ADDR_OF(create_cq_in, in, cq_context);
+ MLX5_SET(cqc, cqc, log_cq_size, ilog2(ncqe));
+ MLX5_SET(cqc, cqc, c_eqn, eqn);
+ MLX5_SET(cqc, cqc, uar_page, uar->index);
+ MLX5_SET(cqc, cqc, log_page_size, cq->wq_ctrl.buf.page_shift -
+ MLX5_ADAPTER_PAGE_SHIFT);
+ MLX5_SET64(cqc, cqc, dbr_addr, cq->wq_ctrl.db.dma);
+
+ pas = (__be64 *)MLX5_ADDR_OF(create_cq_in, in, pas);
+ mlx5_fill_page_frag_array(&cq->wq_ctrl.buf, pas);
+
+ cq->mcq.event = dr_cq_event;
+
+ err = mlx5_core_create_cq(mdev, &cq->mcq, in, inlen, out, sizeof(out));
+ kvfree(in);
+
+ if (err)
+ goto err_cqwq;
+
+ cq->mcq.cqe_sz = 64;
+ cq->mcq.set_ci_db = cq->wq_ctrl.db.db;
+ cq->mcq.arm_db = cq->wq_ctrl.db.db + 1;
+ *cq->mcq.set_ci_db = 0;
+ *cq->mcq.arm_db = 0;
+ cq->mcq.vector = 0;
+ cq->mcq.irqn = irqn;
+ cq->mcq.uar = uar;
+
+ return cq;
+
+err_cqwq:
+ mlx5_wq_destroy(&cq->wq_ctrl);
+out:
+ kfree(cq);
+ return NULL;
+}
+
+static void dr_destroy_cq(struct mlx5_core_dev *mdev, struct mlx5dr_cq *cq)
+{
+ mlx5_core_destroy_cq(mdev, &cq->mcq);
+ mlx5_wq_destroy(&cq->wq_ctrl);
+ kfree(cq);
+}
+
+static int
+dr_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, struct mlx5_core_mkey *mkey)
+{
+ u32 in[MLX5_ST_SZ_DW(create_mkey_in)] = {};
+ void *mkc;
+
+ mkc = MLX5_ADDR_OF(create_mkey_in, in, memory_key_mkey_entry);
+ MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_PA);
+ MLX5_SET(mkc, mkc, a, 1);
+ MLX5_SET(mkc, mkc, rw, 1);
+ MLX5_SET(mkc, mkc, rr, 1);
+ MLX5_SET(mkc, mkc, lw, 1);
+ MLX5_SET(mkc, mkc, lr, 1);
+
+ MLX5_SET(mkc, mkc, pd, pdn);
+ MLX5_SET(mkc, mkc, length64, 1);
+ MLX5_SET(mkc, mkc, qpn, 0xffffff);
+
+ return mlx5_core_create_mkey(mdev, mkey, in, sizeof(in));
+}
+
+static struct mlx5dr_mr *dr_reg_mr(struct mlx5_core_dev *mdev,
+ u32 pdn, void *buf, size_t size)
+{
+ struct mlx5dr_mr *mr = kzalloc(sizeof(*mr), GFP_KERNEL);
+ struct device *dma_device;
+ dma_addr_t dma_addr;
+ int err;
+
+ if (!mr)
+ return NULL;
+
+ dma_device = &mdev->pdev->dev;
+ dma_addr = dma_map_single(dma_device, buf, size,
+ DMA_BIDIRECTIONAL);
+ err = dma_mapping_error(dma_device, dma_addr);
+ if (err) {
+ mlx5_core_warn(mdev, "Can't dma buf\n");
+ kfree(mr);
+ return NULL;
+ }
+
+ err = dr_create_mkey(mdev, pdn, &mr->mkey);
+ if (err) {
+ mlx5_core_warn(mdev, "Can't create mkey\n");
+ dma_unmap_single(dma_device, dma_addr, size,
+ DMA_BIDIRECTIONAL);
+ kfree(mr);
+ return NULL;
+ }
+
+ mr->dma_addr = dma_addr;
+ mr->size = size;
+ mr->addr = buf;
+
+ return mr;
+}
+
+static void dr_dereg_mr(struct mlx5_core_dev *mdev, struct mlx5dr_mr *mr)
+{
+ mlx5_core_destroy_mkey(mdev, &mr->mkey);
+ dma_unmap_single(&mdev->pdev->dev, mr->dma_addr, mr->size,
+ DMA_BIDIRECTIONAL);
+ kfree(mr);
+}
+
+int mlx5dr_send_ring_alloc(struct mlx5dr_domain *dmn)
+{
+ struct dr_qp_init_attr init_attr = {};
+ int cq_size;
+ int size;
+ int ret;
+
+ dmn->send_ring = kzalloc(sizeof(*dmn->send_ring), GFP_KERNEL);
+ if (!dmn->send_ring)
+ return -ENOMEM;
+
+ cq_size = QUEUE_SIZE + 1;
+ dmn->send_ring->cq = dr_create_cq(dmn->mdev, dmn->uar, cq_size);
+ if (!dmn->send_ring->cq) {
+ ret = -ENOMEM;
+ goto free_send_ring;
+ }
+
+ init_attr.cqn = dmn->send_ring->cq->mcq.cqn;
+ init_attr.pdn = dmn->pdn;
+ init_attr.uar = dmn->uar;
+ init_attr.max_send_wr = QUEUE_SIZE;
+
+ dmn->send_ring->qp = dr_create_rc_qp(dmn->mdev, &init_attr);
+ if (!dmn->send_ring->qp) {
+ ret = -ENOMEM;
+ goto clean_cq;
+ }
+
+ dmn->send_ring->cq->qp = dmn->send_ring->qp;
+
+ dmn->info.max_send_wr = QUEUE_SIZE;
+ dmn->info.max_inline_size = min(dmn->send_ring->qp->max_inline_data,
+ DR_STE_SIZE);
+
+ dmn->send_ring->signal_th = dmn->info.max_send_wr /
+ SIGNAL_PER_DIV_QUEUE;
+
+ /* Prepare qp to be used */
+ ret = dr_prepare_qp_to_rts(dmn);
+ if (ret)
+ goto clean_qp;
+
+ dmn->send_ring->max_post_send_size =
+ mlx5dr_icm_pool_chunk_size_to_byte(DR_CHUNK_SIZE_1K,
+ DR_ICM_TYPE_STE);
+
+ /* Allocating the max size as a buffer for writing */
+ size = dmn->send_ring->signal_th * dmn->send_ring->max_post_send_size;
+ dmn->send_ring->buf = kzalloc(size, GFP_KERNEL);
+ if (!dmn->send_ring->buf) {
+ ret = -ENOMEM;
+ goto clean_qp;
+ }
+
+ dmn->send_ring->buf_size = size;
+
+ dmn->send_ring->mr = dr_reg_mr(dmn->mdev,
+ dmn->pdn, dmn->send_ring->buf, size);
+ if (!dmn->send_ring->mr) {
+ ret = -ENOMEM;
+ goto free_mem;
+ }
+
+ dmn->send_ring->sync_mr = dr_reg_mr(dmn->mdev,
+ dmn->pdn, dmn->send_ring->sync_buff,
+ MIN_READ_SYNC);
+ if (!dmn->send_ring->sync_mr) {
+ ret = -ENOMEM;
+ goto clean_mr;
+ }
+
+ return 0;
+
+clean_mr:
+ dr_dereg_mr(dmn->mdev, dmn->send_ring->mr);
+free_mem:
+ kfree(dmn->send_ring->buf);
+clean_qp:
+ dr_destroy_qp(dmn->mdev, dmn->send_ring->qp);
+clean_cq:
+ dr_destroy_cq(dmn->mdev, dmn->send_ring->cq);
+free_send_ring:
+ kfree(dmn->send_ring);
+
+ return ret;
+}
+
+void mlx5dr_send_ring_free(struct mlx5dr_domain *dmn,
+ struct mlx5dr_send_ring *send_ring)
+{
+ dr_destroy_qp(dmn->mdev, send_ring->qp);
+ dr_destroy_cq(dmn->mdev, send_ring->cq);
+ dr_dereg_mr(dmn->mdev, send_ring->sync_mr);
+ dr_dereg_mr(dmn->mdev, send_ring->mr);
+ kfree(send_ring->buf);
+ kfree(send_ring);
+}
+
+int mlx5dr_send_ring_force_drain(struct mlx5dr_domain *dmn)
+{
+ struct mlx5dr_send_ring *send_ring = dmn->send_ring;
+ struct postsend_info send_info = {};
+ u8 data[DR_STE_SIZE];
+ int num_of_sends_req;
+ int ret;
+ int i;
+
+ /* Sending this amount of requests makes sure we will get drain */
+ num_of_sends_req = send_ring->signal_th * TH_NUMS_TO_DRAIN / 2;
+
+ /* Send fake requests forcing the last to be signaled */
+ send_info.write.addr = (uintptr_t)data;
+ send_info.write.length = DR_STE_SIZE;
+ send_info.write.lkey = 0;
+ /* Using the sync_mr in order to write/read */
+ send_info.remote_addr = (uintptr_t)send_ring->sync_mr->addr;
+ send_info.rkey = send_ring->sync_mr->mkey.key;
+
+ for (i = 0; i < num_of_sends_req; i++) {
+ ret = dr_postsend_icm_data(dmn, &send_info);
+ if (ret)
+ return ret;
+ }
+
+ ret = dr_handle_pending_wc(dmn, send_ring);
+
+ return ret;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
new file mode 100644
index 0000000..6b0af64
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c
@@ -0,0 +1,2308 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include <linux/types.h>
+#include "dr_types.h"
+
+#define DR_STE_CRC_POLY 0xEDB88320L
+#define STE_IPV4 0x1
+#define STE_IPV6 0x2
+#define STE_TCP 0x1
+#define STE_UDP 0x2
+#define STE_SPI 0x3
+#define IP_VERSION_IPV4 0x4
+#define IP_VERSION_IPV6 0x6
+#define STE_SVLAN 0x1
+#define STE_CVLAN 0x2
+
+#define DR_STE_ENABLE_FLOW_TAG BIT(31)
+
+/* Set to STE a specific value using DR_STE_SET */
+#define DR_STE_SET_VAL(lookup_type, tag, t_fname, spec, s_fname, value) do { \
+ if ((spec)->s_fname) { \
+ MLX5_SET(ste_##lookup_type, tag, t_fname, value); \
+ (spec)->s_fname = 0; \
+ } \
+} while (0)
+
+/* Set to STE spec->s_fname to tag->t_fname */
+#define DR_STE_SET_TAG(lookup_type, tag, t_fname, spec, s_fname) \
+ DR_STE_SET_VAL(lookup_type, tag, t_fname, spec, s_fname, spec->s_fname)
+
+/* Set to STE -1 to bit_mask->bm_fname and set spec->s_fname as used */
+#define DR_STE_SET_MASK(lookup_type, bit_mask, bm_fname, spec, s_fname) \
+ DR_STE_SET_VAL(lookup_type, bit_mask, bm_fname, spec, s_fname, -1)
+
+/* Set to STE spec->s_fname to bit_mask->bm_fname and set spec->s_fname as used */
+#define DR_STE_SET_MASK_V(lookup_type, bit_mask, bm_fname, spec, s_fname) \
+ DR_STE_SET_VAL(lookup_type, bit_mask, bm_fname, spec, s_fname, (spec)->s_fname)
+
+#define DR_STE_SET_TCP_FLAGS(lookup_type, tag, spec) do { \
+ MLX5_SET(ste_##lookup_type, tag, tcp_ns, !!((spec)->tcp_flags & (1 << 8))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_cwr, !!((spec)->tcp_flags & (1 << 7))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_ece, !!((spec)->tcp_flags & (1 << 6))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_urg, !!((spec)->tcp_flags & (1 << 5))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_ack, !!((spec)->tcp_flags & (1 << 4))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_psh, !!((spec)->tcp_flags & (1 << 3))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_rst, !!((spec)->tcp_flags & (1 << 2))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_syn, !!((spec)->tcp_flags & (1 << 1))); \
+ MLX5_SET(ste_##lookup_type, tag, tcp_fin, !!((spec)->tcp_flags & (1 << 0))); \
+} while (0)
+
+#define DR_STE_SET_MPLS_MASK(lookup_type, mask, in_out, bit_mask) do { \
+ DR_STE_SET_MASK_V(lookup_type, mask, mpls0_label, mask, \
+ in_out##_first_mpls_label);\
+ DR_STE_SET_MASK_V(lookup_type, mask, mpls0_s_bos, mask, \
+ in_out##_first_mpls_s_bos); \
+ DR_STE_SET_MASK_V(lookup_type, mask, mpls0_exp, mask, \
+ in_out##_first_mpls_exp); \
+ DR_STE_SET_MASK_V(lookup_type, mask, mpls0_ttl, mask, \
+ in_out##_first_mpls_ttl); \
+} while (0)
+
+#define DR_STE_SET_MPLS_TAG(lookup_type, mask, in_out, tag) do { \
+ DR_STE_SET_TAG(lookup_type, tag, mpls0_label, mask, \
+ in_out##_first_mpls_label);\
+ DR_STE_SET_TAG(lookup_type, tag, mpls0_s_bos, mask, \
+ in_out##_first_mpls_s_bos); \
+ DR_STE_SET_TAG(lookup_type, tag, mpls0_exp, mask, \
+ in_out##_first_mpls_exp); \
+ DR_STE_SET_TAG(lookup_type, tag, mpls0_ttl, mask, \
+ in_out##_first_mpls_ttl); \
+} while (0)
+
+#define DR_STE_IS_OUTER_MPLS_OVER_GRE_SET(_misc) (\
+ (_misc)->outer_first_mpls_over_gre_label || \
+ (_misc)->outer_first_mpls_over_gre_exp || \
+ (_misc)->outer_first_mpls_over_gre_s_bos || \
+ (_misc)->outer_first_mpls_over_gre_ttl)
+#define DR_STE_IS_OUTER_MPLS_OVER_UDP_SET(_misc) (\
+ (_misc)->outer_first_mpls_over_udp_label || \
+ (_misc)->outer_first_mpls_over_udp_exp || \
+ (_misc)->outer_first_mpls_over_udp_s_bos || \
+ (_misc)->outer_first_mpls_over_udp_ttl)
+
+#define DR_STE_CALC_LU_TYPE(lookup_type, rx, inner) \
+ ((inner) ? MLX5DR_STE_LU_TYPE_##lookup_type##_I : \
+ (rx) ? MLX5DR_STE_LU_TYPE_##lookup_type##_D : \
+ MLX5DR_STE_LU_TYPE_##lookup_type##_O)
+
+enum dr_ste_tunl_action {
+ DR_STE_TUNL_ACTION_NONE = 0,
+ DR_STE_TUNL_ACTION_ENABLE = 1,
+ DR_STE_TUNL_ACTION_DECAP = 2,
+ DR_STE_TUNL_ACTION_L3_DECAP = 3,
+ DR_STE_TUNL_ACTION_POP_VLAN = 4,
+};
+
+enum dr_ste_action_type {
+ DR_STE_ACTION_TYPE_PUSH_VLAN = 1,
+ DR_STE_ACTION_TYPE_ENCAP_L3 = 3,
+ DR_STE_ACTION_TYPE_ENCAP = 4,
+};
+
+struct dr_hw_ste_format {
+ u8 ctrl[DR_STE_SIZE_CTRL];
+ u8 tag[DR_STE_SIZE_TAG];
+ u8 mask[DR_STE_SIZE_MASK];
+};
+
+u32 mlx5dr_ste_calc_hash_index(u8 *hw_ste_p, struct mlx5dr_ste_htbl *htbl)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ u8 masked[DR_STE_SIZE_TAG] = {};
+ u32 crc32, index;
+ u16 bit;
+ int i;
+
+ /* Don't calculate CRC if the result is predicted */
+ if (htbl->chunk->num_of_entries == 1 || htbl->byte_mask == 0)
+ return 0;
+
+ /* Mask tag using byte mask, bit per byte */
+ bit = 1 << (DR_STE_SIZE_TAG - 1);
+ for (i = 0; i < DR_STE_SIZE_TAG; i++) {
+ if (htbl->byte_mask & bit)
+ masked[i] = hw_ste->tag[i];
+
+ bit = bit >> 1;
+ }
+
+ crc32 = mlx5dr_crc32_slice8_calc(masked, DR_STE_SIZE_TAG);
+ index = crc32 & (htbl->chunk->num_of_entries - 1);
+
+ return index;
+}
+
+static u16 dr_ste_conv_bit_to_byte_mask(u8 *bit_mask)
+{
+ u16 byte_mask = 0;
+ int i;
+
+ for (i = 0; i < DR_STE_SIZE_MASK; i++) {
+ byte_mask = byte_mask << 1;
+ if (bit_mask[i] == 0xff)
+ byte_mask |= 1;
+ }
+ return byte_mask;
+}
+
+void mlx5dr_ste_set_bit_mask(u8 *hw_ste_p, u8 *bit_mask)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+
+ memcpy(hw_ste->mask, bit_mask, DR_STE_SIZE_MASK);
+}
+
+void mlx5dr_ste_rx_set_flow_tag(u8 *hw_ste_p, u32 flow_tag)
+{
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, qp_list_pointer,
+ DR_STE_ENABLE_FLOW_TAG | flow_tag);
+}
+
+void mlx5dr_ste_set_counter_id(u8 *hw_ste_p, u32 ctr_id)
+{
+ /* This can be used for both rx_steering_mult and for sx_transmit */
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, counter_trigger_15_0, ctr_id);
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, counter_trigger_23_16, ctr_id >> 16);
+}
+
+void mlx5dr_ste_set_go_back_bit(u8 *hw_ste_p)
+{
+ MLX5_SET(ste_sx_transmit, hw_ste_p, go_back, 1);
+}
+
+void mlx5dr_ste_set_tx_push_vlan(u8 *hw_ste_p, u32 vlan_hdr,
+ bool go_back)
+{
+ MLX5_SET(ste_sx_transmit, hw_ste_p, action_type,
+ DR_STE_ACTION_TYPE_PUSH_VLAN);
+ MLX5_SET(ste_sx_transmit, hw_ste_p, encap_pointer_vlan_data, vlan_hdr);
+ /* Due to HW limitation we need to set this bit, otherwise reforamt +
+ * push vlan will not work.
+ */
+ if (go_back)
+ mlx5dr_ste_set_go_back_bit(hw_ste_p);
+}
+
+void mlx5dr_ste_set_tx_encap(void *hw_ste_p, u32 reformat_id, int size, bool encap_l3)
+{
+ MLX5_SET(ste_sx_transmit, hw_ste_p, action_type,
+ encap_l3 ? DR_STE_ACTION_TYPE_ENCAP_L3 : DR_STE_ACTION_TYPE_ENCAP);
+ /* The hardware expects here size in words (2 byte) */
+ MLX5_SET(ste_sx_transmit, hw_ste_p, action_description, size / 2);
+ MLX5_SET(ste_sx_transmit, hw_ste_p, encap_pointer_vlan_data, reformat_id);
+}
+
+void mlx5dr_ste_set_rx_decap(u8 *hw_ste_p)
+{
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, tunneling_action,
+ DR_STE_TUNL_ACTION_DECAP);
+}
+
+void mlx5dr_ste_set_rx_pop_vlan(u8 *hw_ste_p)
+{
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, tunneling_action,
+ DR_STE_TUNL_ACTION_POP_VLAN);
+}
+
+void mlx5dr_ste_set_rx_decap_l3(u8 *hw_ste_p, bool vlan)
+{
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, tunneling_action,
+ DR_STE_TUNL_ACTION_L3_DECAP);
+ MLX5_SET(ste_modify_packet, hw_ste_p, action_description, vlan ? 1 : 0);
+}
+
+void mlx5dr_ste_set_entry_type(u8 *hw_ste_p, u8 entry_type)
+{
+ MLX5_SET(ste_general, hw_ste_p, entry_type, entry_type);
+}
+
+u8 mlx5dr_ste_get_entry_type(u8 *hw_ste_p)
+{
+ return MLX5_GET(ste_general, hw_ste_p, entry_type);
+}
+
+void mlx5dr_ste_set_rewrite_actions(u8 *hw_ste_p, u16 num_of_actions,
+ u32 re_write_index)
+{
+ MLX5_SET(ste_modify_packet, hw_ste_p, number_of_re_write_actions,
+ num_of_actions);
+ MLX5_SET(ste_modify_packet, hw_ste_p, header_re_write_actions_pointer,
+ re_write_index);
+}
+
+void mlx5dr_ste_set_hit_gvmi(u8 *hw_ste_p, u16 gvmi)
+{
+ MLX5_SET(ste_general, hw_ste_p, next_table_base_63_48, gvmi);
+}
+
+void mlx5dr_ste_init(u8 *hw_ste_p, u8 lu_type, u8 entry_type,
+ u16 gvmi)
+{
+ MLX5_SET(ste_general, hw_ste_p, entry_type, entry_type);
+ MLX5_SET(ste_general, hw_ste_p, entry_sub_type, lu_type);
+ MLX5_SET(ste_general, hw_ste_p, next_lu_type, MLX5DR_STE_LU_TYPE_DONT_CARE);
+
+ /* Set GVMI once, this is the same for RX/TX
+ * bits 63_48 of next table base / miss address encode the next GVMI
+ */
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, gvmi, gvmi);
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, next_table_base_63_48, gvmi);
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, miss_address_63_48, gvmi);
+}
+
+static void dr_ste_set_always_hit(struct dr_hw_ste_format *hw_ste)
+{
+ memset(&hw_ste->tag, 0, sizeof(hw_ste->tag));
+ memset(&hw_ste->mask, 0, sizeof(hw_ste->mask));
+}
+
+static void dr_ste_set_always_miss(struct dr_hw_ste_format *hw_ste)
+{
+ hw_ste->tag[0] = 0xdc;
+ hw_ste->mask[0] = 0;
+}
+
+u64 mlx5dr_ste_get_miss_addr(u8 *hw_ste)
+{
+ u64 index =
+ (MLX5_GET(ste_rx_steering_mult, hw_ste, miss_address_31_6) |
+ MLX5_GET(ste_rx_steering_mult, hw_ste, miss_address_39_32) << 26);
+
+ return index << 6;
+}
+
+void mlx5dr_ste_set_hit_addr(u8 *hw_ste, u64 icm_addr, u32 ht_size)
+{
+ u64 index = (icm_addr >> 5) | ht_size;
+
+ MLX5_SET(ste_general, hw_ste, next_table_base_39_32_size, index >> 27);
+ MLX5_SET(ste_general, hw_ste, next_table_base_31_5_size, index);
+}
+
+u64 mlx5dr_ste_get_icm_addr(struct mlx5dr_ste *ste)
+{
+ u32 index = ste - ste->htbl->ste_arr;
+
+ return ste->htbl->chunk->icm_addr + DR_STE_SIZE * index;
+}
+
+u64 mlx5dr_ste_get_mr_addr(struct mlx5dr_ste *ste)
+{
+ u32 index = ste - ste->htbl->ste_arr;
+
+ return ste->htbl->chunk->mr_addr + DR_STE_SIZE * index;
+}
+
+struct list_head *mlx5dr_ste_get_miss_list(struct mlx5dr_ste *ste)
+{
+ u32 index = ste - ste->htbl->ste_arr;
+
+ return &ste->htbl->miss_list[index];
+}
+
+static void dr_ste_always_hit_htbl(struct mlx5dr_ste *ste,
+ struct mlx5dr_ste_htbl *next_htbl)
+{
+ struct mlx5dr_icm_chunk *chunk = next_htbl->chunk;
+ u8 *hw_ste = ste->hw_ste;
+
+ MLX5_SET(ste_general, hw_ste, byte_mask, next_htbl->byte_mask);
+ MLX5_SET(ste_general, hw_ste, next_lu_type, next_htbl->lu_type);
+ mlx5dr_ste_set_hit_addr(hw_ste, chunk->icm_addr, chunk->num_of_entries);
+
+ dr_ste_set_always_hit((struct dr_hw_ste_format *)ste->hw_ste);
+}
+
+bool mlx5dr_ste_is_last_in_rule(struct mlx5dr_matcher_rx_tx *nic_matcher,
+ u8 ste_location)
+{
+ return ste_location == nic_matcher->num_of_builders;
+}
+
+/* Replace relevant fields, except of:
+ * htbl - keep the origin htbl
+ * miss_list + list - already took the src from the list.
+ * icm_addr/mr_addr - depends on the hosting table.
+ *
+ * Before:
+ * | a | -> | b | -> | c | ->
+ *
+ * After:
+ * | a | -> | c | ->
+ * While the data that was in b copied to a.
+ */
+static void dr_ste_replace(struct mlx5dr_ste *dst, struct mlx5dr_ste *src)
+{
+ memcpy(dst->hw_ste, src->hw_ste, DR_STE_SIZE_REDUCED);
+ dst->next_htbl = src->next_htbl;
+ if (dst->next_htbl)
+ dst->next_htbl->pointing_ste = dst;
+
+ refcount_set(&dst->refcount, refcount_read(&src->refcount));
+
+ INIT_LIST_HEAD(&dst->rule_list);
+ list_splice_tail_init(&src->rule_list, &dst->rule_list);
+}
+
+/* Free ste which is the head and the only one in miss_list */
+static void
+dr_ste_remove_head_ste(struct mlx5dr_ste *ste,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste_send_info *ste_info_head,
+ struct list_head *send_ste_list,
+ struct mlx5dr_ste_htbl *stats_tbl)
+{
+ u8 tmp_data_ste[DR_STE_SIZE] = {};
+ struct mlx5dr_ste tmp_ste = {};
+ u64 miss_addr;
+
+ tmp_ste.hw_ste = tmp_data_ste;
+
+ /* Use temp ste because dr_ste_always_miss_addr
+ * touches bit_mask area which doesn't exist at ste->hw_ste.
+ */
+ memcpy(tmp_ste.hw_ste, ste->hw_ste, DR_STE_SIZE_REDUCED);
+ miss_addr = nic_matcher->e_anchor->chunk->icm_addr;
+ mlx5dr_ste_always_miss_addr(&tmp_ste, miss_addr);
+ memcpy(ste->hw_ste, tmp_ste.hw_ste, DR_STE_SIZE_REDUCED);
+
+ list_del_init(&ste->miss_list_node);
+
+ /* Write full STE size in order to have "always_miss" */
+ mlx5dr_send_fill_and_append_ste_send_info(ste, DR_STE_SIZE,
+ 0, tmp_data_ste,
+ ste_info_head,
+ send_ste_list,
+ true /* Copy data */);
+
+ stats_tbl->ctrl.num_of_valid_entries--;
+}
+
+/* Free ste which is the head but NOT the only one in miss_list:
+ * |_ste_| --> |_next_ste_| -->|__| -->|__| -->/0
+ */
+static void
+dr_ste_replace_head_ste(struct mlx5dr_ste *ste, struct mlx5dr_ste *next_ste,
+ struct mlx5dr_ste_send_info *ste_info_head,
+ struct list_head *send_ste_list,
+ struct mlx5dr_ste_htbl *stats_tbl)
+
+{
+ struct mlx5dr_ste_htbl *next_miss_htbl;
+
+ next_miss_htbl = next_ste->htbl;
+
+ /* Remove from the miss_list the next_ste before copy */
+ list_del_init(&next_ste->miss_list_node);
+
+ /* All rule-members that use next_ste should know about that */
+ mlx5dr_rule_update_rule_member(next_ste, ste);
+
+ /* Move data from next into ste */
+ dr_ste_replace(ste, next_ste);
+
+ /* Del the htbl that contains the next_ste.
+ * The origin htbl stay with the same number of entries.
+ */
+ mlx5dr_htbl_put(next_miss_htbl);
+
+ mlx5dr_send_fill_and_append_ste_send_info(ste, DR_STE_SIZE_REDUCED,
+ 0, ste->hw_ste,
+ ste_info_head,
+ send_ste_list,
+ true /* Copy data */);
+
+ stats_tbl->ctrl.num_of_collisions--;
+ stats_tbl->ctrl.num_of_valid_entries--;
+}
+
+/* Free ste that is located in the middle of the miss list:
+ * |__| -->|_prev_ste_|->|_ste_|-->|_next_ste_|
+ */
+static void dr_ste_remove_middle_ste(struct mlx5dr_ste *ste,
+ struct mlx5dr_ste_send_info *ste_info,
+ struct list_head *send_ste_list,
+ struct mlx5dr_ste_htbl *stats_tbl)
+{
+ struct mlx5dr_ste *prev_ste;
+ u64 miss_addr;
+
+ prev_ste = list_entry(mlx5dr_ste_get_miss_list(ste)->prev, struct mlx5dr_ste,
+ miss_list_node);
+ if (!prev_ste) {
+ WARN_ON(true);
+ return;
+ }
+
+ miss_addr = mlx5dr_ste_get_miss_addr(ste->hw_ste);
+ mlx5dr_ste_set_miss_addr(prev_ste->hw_ste, miss_addr);
+
+ mlx5dr_send_fill_and_append_ste_send_info(prev_ste, DR_STE_SIZE_REDUCED, 0,
+ prev_ste->hw_ste, ste_info,
+ send_ste_list, true /* Copy data*/);
+
+ list_del_init(&ste->miss_list_node);
+
+ stats_tbl->ctrl.num_of_valid_entries--;
+ stats_tbl->ctrl.num_of_collisions--;
+}
+
+void mlx5dr_ste_free(struct mlx5dr_ste *ste,
+ struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher)
+{
+ struct mlx5dr_ste_send_info *cur_ste_info, *tmp_ste_info;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_ste_send_info ste_info_head;
+ struct mlx5dr_ste *next_ste, *first_ste;
+ bool put_on_origin_table = true;
+ struct mlx5dr_ste_htbl *stats_tbl;
+ LIST_HEAD(send_ste_list);
+
+ first_ste = list_entry(mlx5dr_ste_get_miss_list(ste)->next,
+ struct mlx5dr_ste, miss_list_node);
+ stats_tbl = first_ste->htbl;
+
+ /* Two options:
+ * 1. ste is head:
+ * a. head ste is the only ste in the miss list
+ * b. head ste is not the only ste in the miss-list
+ * 2. ste is not head
+ */
+ if (first_ste == ste) { /* Ste is the head */
+ struct mlx5dr_ste *last_ste;
+
+ last_ste = list_last_entry(mlx5dr_ste_get_miss_list(ste),
+ struct mlx5dr_ste, miss_list_node);
+ if (last_ste == first_ste)
+ next_ste = NULL;
+ else
+ next_ste = list_entry(ste->miss_list_node.next,
+ struct mlx5dr_ste, miss_list_node);
+
+ if (!next_ste) {
+ /* One and only entry in the list */
+ dr_ste_remove_head_ste(ste, nic_matcher,
+ &ste_info_head,
+ &send_ste_list,
+ stats_tbl);
+ } else {
+ /* First but not only entry in the list */
+ dr_ste_replace_head_ste(ste, next_ste, &ste_info_head,
+ &send_ste_list, stats_tbl);
+ put_on_origin_table = false;
+ }
+ } else { /* Ste in the middle of the list */
+ dr_ste_remove_middle_ste(ste, &ste_info_head, &send_ste_list, stats_tbl);
+ }
+
+ /* Update HW */
+ list_for_each_entry_safe(cur_ste_info, tmp_ste_info,
+ &send_ste_list, send_list) {
+ list_del(&cur_ste_info->send_list);
+ mlx5dr_send_postsend_ste(dmn, cur_ste_info->ste,
+ cur_ste_info->data, cur_ste_info->size,
+ cur_ste_info->offset);
+ }
+
+ if (put_on_origin_table)
+ mlx5dr_htbl_put(ste->htbl);
+}
+
+bool mlx5dr_ste_equal_tag(void *src, void *dst)
+{
+ struct dr_hw_ste_format *s_hw_ste = (struct dr_hw_ste_format *)src;
+ struct dr_hw_ste_format *d_hw_ste = (struct dr_hw_ste_format *)dst;
+
+ return !memcmp(s_hw_ste->tag, d_hw_ste->tag, DR_STE_SIZE_TAG);
+}
+
+void mlx5dr_ste_set_hit_addr_by_next_htbl(u8 *hw_ste,
+ struct mlx5dr_ste_htbl *next_htbl)
+{
+ struct mlx5dr_icm_chunk *chunk = next_htbl->chunk;
+
+ mlx5dr_ste_set_hit_addr(hw_ste, chunk->icm_addr, chunk->num_of_entries);
+}
+
+void mlx5dr_ste_set_miss_addr(u8 *hw_ste_p, u64 miss_addr)
+{
+ u64 index = miss_addr >> 6;
+
+ /* Miss address for TX and RX STEs located in the same offsets */
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, miss_address_39_32, index >> 26);
+ MLX5_SET(ste_rx_steering_mult, hw_ste_p, miss_address_31_6, index);
+}
+
+void mlx5dr_ste_always_miss_addr(struct mlx5dr_ste *ste, u64 miss_addr)
+{
+ u8 *hw_ste = ste->hw_ste;
+
+ MLX5_SET(ste_rx_steering_mult, hw_ste, next_lu_type, MLX5DR_STE_LU_TYPE_DONT_CARE);
+ mlx5dr_ste_set_miss_addr(hw_ste, miss_addr);
+ dr_ste_set_always_miss((struct dr_hw_ste_format *)ste->hw_ste);
+}
+
+/* The assumption here is that we don't update the ste->hw_ste if it is not
+ * used ste, so it will be all zero, checking the next_lu_type.
+ */
+bool mlx5dr_ste_is_not_valid_entry(u8 *p_hw_ste)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)p_hw_ste;
+
+ if (MLX5_GET(ste_general, hw_ste, next_lu_type) ==
+ MLX5DR_STE_LU_TYPE_NOP)
+ return true;
+
+ return false;
+}
+
+bool mlx5dr_ste_not_used_ste(struct mlx5dr_ste *ste)
+{
+ return !refcount_read(&ste->refcount);
+}
+
+static u16 get_bits_per_mask(u16 byte_mask)
+{
+ u16 bits = 0;
+
+ while (byte_mask) {
+ byte_mask = byte_mask & (byte_mask - 1);
+ bits++;
+ }
+
+ return bits;
+}
+
+/* Init one ste as a pattern for ste data array */
+void mlx5dr_ste_set_formatted_ste(u16 gvmi,
+ struct mlx5dr_domain_rx_tx *nic_dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 *formatted_ste,
+ struct mlx5dr_htbl_connect_info *connect_info)
+{
+ struct mlx5dr_ste ste = {};
+
+ mlx5dr_ste_init(formatted_ste, htbl->lu_type, nic_dmn->ste_type, gvmi);
+ ste.hw_ste = formatted_ste;
+
+ if (connect_info->type == CONNECT_HIT)
+ dr_ste_always_hit_htbl(&ste, connect_info->hit_next_htbl);
+ else
+ mlx5dr_ste_always_miss_addr(&ste, connect_info->miss_icm_addr);
+}
+
+int mlx5dr_ste_htbl_init_and_postsend(struct mlx5dr_domain *dmn,
+ struct mlx5dr_domain_rx_tx *nic_dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ struct mlx5dr_htbl_connect_info *connect_info,
+ bool update_hw_ste)
+{
+ u8 formatted_ste[DR_STE_SIZE] = {};
+
+ mlx5dr_ste_set_formatted_ste(dmn->info.caps.gvmi,
+ nic_dmn,
+ htbl,
+ formatted_ste,
+ connect_info);
+
+ return mlx5dr_send_postsend_formatted_htbl(dmn, htbl, formatted_ste, update_hw_ste);
+}
+
+int mlx5dr_ste_create_next_htbl(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste *ste,
+ u8 *cur_hw_ste,
+ enum mlx5dr_icm_chunk_size log_table_size)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)cur_hw_ste;
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_htbl_connect_info info;
+ struct mlx5dr_ste_htbl *next_htbl;
+
+ if (!mlx5dr_ste_is_last_in_rule(nic_matcher, ste->ste_chain_location)) {
+ u32 bits_in_mask;
+ u8 next_lu_type;
+ u16 byte_mask;
+
+ next_lu_type = MLX5_GET(ste_general, hw_ste, next_lu_type);
+ byte_mask = MLX5_GET(ste_general, hw_ste, byte_mask);
+
+ /* Don't allocate table more than required,
+ * the size of the table defined via the byte_mask, so no need
+ * to allocate more than that.
+ */
+ bits_in_mask = get_bits_per_mask(byte_mask) * BITS_PER_BYTE;
+ log_table_size = min(log_table_size, bits_in_mask);
+
+ next_htbl = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
+ log_table_size,
+ next_lu_type,
+ byte_mask);
+ if (!next_htbl) {
+ mlx5dr_dbg(dmn, "Failed allocating table\n");
+ return -ENOMEM;
+ }
+
+ /* Write new table to HW */
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = nic_matcher->e_anchor->chunk->icm_addr;
+ if (mlx5dr_ste_htbl_init_and_postsend(dmn, nic_dmn, next_htbl,
+ &info, false)) {
+ mlx5dr_info(dmn, "Failed writing table to HW\n");
+ goto free_table;
+ }
+
+ mlx5dr_ste_set_hit_addr_by_next_htbl(cur_hw_ste, next_htbl);
+ ste->next_htbl = next_htbl;
+ next_htbl->pointing_ste = ste;
+ }
+
+ return 0;
+
+free_table:
+ mlx5dr_ste_htbl_free(next_htbl);
+ return -ENOENT;
+}
+
+static void dr_ste_set_ctrl(struct mlx5dr_ste_htbl *htbl)
+{
+ struct mlx5dr_ste_htbl_ctrl *ctrl = &htbl->ctrl;
+ int num_of_entries;
+
+ htbl->ctrl.may_grow = true;
+
+ if (htbl->chunk_size == DR_CHUNK_SIZE_MAX - 1)
+ htbl->ctrl.may_grow = false;
+
+ /* Threshold is 50%, one is added to table of size 1 */
+ num_of_entries = mlx5dr_icm_pool_chunk_size_to_entries(htbl->chunk_size);
+ ctrl->increase_threshold = (num_of_entries + 1) / 2;
+}
+
+struct mlx5dr_ste_htbl *mlx5dr_ste_htbl_alloc(struct mlx5dr_icm_pool *pool,
+ enum mlx5dr_icm_chunk_size chunk_size,
+ u8 lu_type, u16 byte_mask)
+{
+ struct mlx5dr_icm_chunk *chunk;
+ struct mlx5dr_ste_htbl *htbl;
+ int i;
+
+ htbl = kzalloc(sizeof(*htbl), GFP_KERNEL);
+ if (!htbl)
+ return NULL;
+
+ chunk = mlx5dr_icm_alloc_chunk(pool, chunk_size);
+ if (!chunk)
+ goto out_free_htbl;
+
+ htbl->chunk = chunk;
+ htbl->lu_type = lu_type;
+ htbl->byte_mask = byte_mask;
+ htbl->ste_arr = chunk->ste_arr;
+ htbl->hw_ste_arr = chunk->hw_ste_arr;
+ htbl->miss_list = chunk->miss_list;
+ refcount_set(&htbl->refcount, 0);
+
+ for (i = 0; i < chunk->num_of_entries; i++) {
+ struct mlx5dr_ste *ste = &htbl->ste_arr[i];
+
+ ste->hw_ste = htbl->hw_ste_arr + i * DR_STE_SIZE_REDUCED;
+ ste->htbl = htbl;
+ refcount_set(&ste->refcount, 0);
+ INIT_LIST_HEAD(&ste->miss_list_node);
+ INIT_LIST_HEAD(&htbl->miss_list[i]);
+ INIT_LIST_HEAD(&ste->rule_list);
+ }
+
+ htbl->chunk_size = chunk_size;
+ dr_ste_set_ctrl(htbl);
+ return htbl;
+
+out_free_htbl:
+ kfree(htbl);
+ return NULL;
+}
+
+int mlx5dr_ste_htbl_free(struct mlx5dr_ste_htbl *htbl)
+{
+ if (refcount_read(&htbl->refcount))
+ return -EBUSY;
+
+ mlx5dr_icm_free_chunk(htbl->chunk);
+ kfree(htbl);
+ return 0;
+}
+
+int mlx5dr_ste_build_pre_check(struct mlx5dr_domain *dmn,
+ u8 match_criteria,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_match_param *value)
+{
+ if (!value && (match_criteria & DR_MATCHER_CRITERIA_MISC)) {
+ if (mask->misc.source_port && mask->misc.source_port != 0xffff) {
+ mlx5dr_dbg(dmn, "Partial mask source_port is not supported\n");
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+int mlx5dr_ste_build_ste_arr(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_match_param *value,
+ u8 *ste_arr)
+{
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_matcher->nic_tbl->nic_dmn;
+ struct mlx5dr_domain *dmn = matcher->tbl->dmn;
+ struct mlx5dr_ste_build *sb;
+ int ret, i;
+
+ ret = mlx5dr_ste_build_pre_check(dmn, matcher->match_criteria,
+ &matcher->mask, value);
+ if (ret)
+ return ret;
+
+ sb = nic_matcher->ste_builder;
+ for (i = 0; i < nic_matcher->num_of_builders; i++) {
+ mlx5dr_ste_init(ste_arr,
+ sb->lu_type,
+ nic_dmn->ste_type,
+ dmn->info.caps.gvmi);
+
+ mlx5dr_ste_set_bit_mask(ste_arr, sb->bit_mask);
+
+ ret = sb->ste_build_tag_func(value, sb, ste_arr);
+ if (ret)
+ return ret;
+
+ /* Connect the STEs */
+ if (i < (nic_matcher->num_of_builders - 1)) {
+ /* Need the next builder for these fields,
+ * not relevant for the last ste in the chain.
+ */
+ sb++;
+ MLX5_SET(ste_general, ste_arr, next_lu_type, sb->lu_type);
+ MLX5_SET(ste_general, ste_arr, byte_mask, sb->byte_mask);
+ }
+ ste_arr += DR_STE_SIZE;
+ }
+ return 0;
+}
+
+static int dr_ste_build_eth_l2_src_des_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l2_src_dst, bit_mask, dmac_47_16, mask, dmac_47_16);
+ DR_STE_SET_MASK_V(eth_l2_src_dst, bit_mask, dmac_15_0, mask, dmac_15_0);
+
+ if (mask->smac_47_16 || mask->smac_15_0) {
+ MLX5_SET(ste_eth_l2_src_dst, bit_mask, smac_47_32,
+ mask->smac_47_16 >> 16);
+ MLX5_SET(ste_eth_l2_src_dst, bit_mask, smac_31_0,
+ mask->smac_47_16 << 16 | mask->smac_15_0);
+ mask->smac_47_16 = 0;
+ mask->smac_15_0 = 0;
+ }
+
+ DR_STE_SET_MASK_V(eth_l2_src_dst, bit_mask, first_vlan_id, mask, first_vid);
+ DR_STE_SET_MASK_V(eth_l2_src_dst, bit_mask, first_cfi, mask, first_cfi);
+ DR_STE_SET_MASK_V(eth_l2_src_dst, bit_mask, first_priority, mask, first_prio);
+ DR_STE_SET_MASK(eth_l2_src_dst, bit_mask, l3_type, mask, ip_version);
+
+ if (mask->cvlan_tag) {
+ MLX5_SET(ste_eth_l2_src_dst, bit_mask, first_vlan_qualifier, -1);
+ mask->cvlan_tag = 0;
+ } else if (mask->svlan_tag) {
+ MLX5_SET(ste_eth_l2_src_dst, bit_mask, first_vlan_qualifier, -1);
+ mask->svlan_tag = 0;
+ }
+
+ if (mask->cvlan_tag || mask->svlan_tag) {
+ pr_info("Invalid c/svlan mask configuration\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void dr_ste_copy_mask_misc(char *mask, struct mlx5dr_match_misc *spec)
+{
+ spec->gre_c_present = MLX5_GET(fte_match_set_misc, mask, gre_c_present);
+ spec->gre_k_present = MLX5_GET(fte_match_set_misc, mask, gre_k_present);
+ spec->gre_s_present = MLX5_GET(fte_match_set_misc, mask, gre_s_present);
+ spec->source_vhca_port = MLX5_GET(fte_match_set_misc, mask, source_vhca_port);
+ spec->source_sqn = MLX5_GET(fte_match_set_misc, mask, source_sqn);
+
+ spec->source_port = MLX5_GET(fte_match_set_misc, mask, source_port);
+
+ spec->outer_second_prio = MLX5_GET(fte_match_set_misc, mask, outer_second_prio);
+ spec->outer_second_cfi = MLX5_GET(fte_match_set_misc, mask, outer_second_cfi);
+ spec->outer_second_vid = MLX5_GET(fte_match_set_misc, mask, outer_second_vid);
+ spec->inner_second_prio = MLX5_GET(fte_match_set_misc, mask, inner_second_prio);
+ spec->inner_second_cfi = MLX5_GET(fte_match_set_misc, mask, inner_second_cfi);
+ spec->inner_second_vid = MLX5_GET(fte_match_set_misc, mask, inner_second_vid);
+
+ spec->outer_second_cvlan_tag =
+ MLX5_GET(fte_match_set_misc, mask, outer_second_cvlan_tag);
+ spec->inner_second_cvlan_tag =
+ MLX5_GET(fte_match_set_misc, mask, inner_second_cvlan_tag);
+ spec->outer_second_svlan_tag =
+ MLX5_GET(fte_match_set_misc, mask, outer_second_svlan_tag);
+ spec->inner_second_svlan_tag =
+ MLX5_GET(fte_match_set_misc, mask, inner_second_svlan_tag);
+
+ spec->gre_protocol = MLX5_GET(fte_match_set_misc, mask, gre_protocol);
+
+ spec->gre_key_h = MLX5_GET(fte_match_set_misc, mask, gre_key.nvgre.hi);
+ spec->gre_key_l = MLX5_GET(fte_match_set_misc, mask, gre_key.nvgre.lo);
+
+ spec->vxlan_vni = MLX5_GET(fte_match_set_misc, mask, vxlan_vni);
+
+ spec->geneve_vni = MLX5_GET(fte_match_set_misc, mask, geneve_vni);
+ spec->geneve_oam = MLX5_GET(fte_match_set_misc, mask, geneve_oam);
+
+ spec->outer_ipv6_flow_label =
+ MLX5_GET(fte_match_set_misc, mask, outer_ipv6_flow_label);
+
+ spec->inner_ipv6_flow_label =
+ MLX5_GET(fte_match_set_misc, mask, inner_ipv6_flow_label);
+
+ spec->geneve_opt_len = MLX5_GET(fte_match_set_misc, mask, geneve_opt_len);
+ spec->geneve_protocol_type =
+ MLX5_GET(fte_match_set_misc, mask, geneve_protocol_type);
+
+ spec->bth_dst_qp = MLX5_GET(fte_match_set_misc, mask, bth_dst_qp);
+}
+
+static void dr_ste_copy_mask_spec(char *mask, struct mlx5dr_match_spec *spec)
+{
+ u32 raw_ip[4];
+
+ spec->smac_47_16 = MLX5_GET(fte_match_set_lyr_2_4, mask, smac_47_16);
+
+ spec->smac_15_0 = MLX5_GET(fte_match_set_lyr_2_4, mask, smac_15_0);
+ spec->ethertype = MLX5_GET(fte_match_set_lyr_2_4, mask, ethertype);
+
+ spec->dmac_47_16 = MLX5_GET(fte_match_set_lyr_2_4, mask, dmac_47_16);
+
+ spec->dmac_15_0 = MLX5_GET(fte_match_set_lyr_2_4, mask, dmac_15_0);
+ spec->first_prio = MLX5_GET(fte_match_set_lyr_2_4, mask, first_prio);
+ spec->first_cfi = MLX5_GET(fte_match_set_lyr_2_4, mask, first_cfi);
+ spec->first_vid = MLX5_GET(fte_match_set_lyr_2_4, mask, first_vid);
+
+ spec->ip_protocol = MLX5_GET(fte_match_set_lyr_2_4, mask, ip_protocol);
+ spec->ip_dscp = MLX5_GET(fte_match_set_lyr_2_4, mask, ip_dscp);
+ spec->ip_ecn = MLX5_GET(fte_match_set_lyr_2_4, mask, ip_ecn);
+ spec->cvlan_tag = MLX5_GET(fte_match_set_lyr_2_4, mask, cvlan_tag);
+ spec->svlan_tag = MLX5_GET(fte_match_set_lyr_2_4, mask, svlan_tag);
+ spec->frag = MLX5_GET(fte_match_set_lyr_2_4, mask, frag);
+ spec->ip_version = MLX5_GET(fte_match_set_lyr_2_4, mask, ip_version);
+ spec->tcp_flags = MLX5_GET(fte_match_set_lyr_2_4, mask, tcp_flags);
+ spec->tcp_sport = MLX5_GET(fte_match_set_lyr_2_4, mask, tcp_sport);
+ spec->tcp_dport = MLX5_GET(fte_match_set_lyr_2_4, mask, tcp_dport);
+
+ spec->ttl_hoplimit = MLX5_GET(fte_match_set_lyr_2_4, mask, ttl_hoplimit);
+
+ spec->udp_sport = MLX5_GET(fte_match_set_lyr_2_4, mask, udp_sport);
+ spec->udp_dport = MLX5_GET(fte_match_set_lyr_2_4, mask, udp_dport);
+
+ memcpy(raw_ip, MLX5_ADDR_OF(fte_match_set_lyr_2_4, mask,
+ src_ipv4_src_ipv6.ipv6_layout.ipv6),
+ sizeof(raw_ip));
+
+ spec->src_ip_127_96 = be32_to_cpu(raw_ip[0]);
+ spec->src_ip_95_64 = be32_to_cpu(raw_ip[1]);
+ spec->src_ip_63_32 = be32_to_cpu(raw_ip[2]);
+ spec->src_ip_31_0 = be32_to_cpu(raw_ip[3]);
+
+ memcpy(raw_ip, MLX5_ADDR_OF(fte_match_set_lyr_2_4, mask,
+ dst_ipv4_dst_ipv6.ipv6_layout.ipv6),
+ sizeof(raw_ip));
+
+ spec->dst_ip_127_96 = be32_to_cpu(raw_ip[0]);
+ spec->dst_ip_95_64 = be32_to_cpu(raw_ip[1]);
+ spec->dst_ip_63_32 = be32_to_cpu(raw_ip[2]);
+ spec->dst_ip_31_0 = be32_to_cpu(raw_ip[3]);
+}
+
+static void dr_ste_copy_mask_misc2(char *mask, struct mlx5dr_match_misc2 *spec)
+{
+ spec->outer_first_mpls_label =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls.mpls_label);
+ spec->outer_first_mpls_exp =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls.mpls_exp);
+ spec->outer_first_mpls_s_bos =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls.mpls_s_bos);
+ spec->outer_first_mpls_ttl =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls.mpls_ttl);
+ spec->inner_first_mpls_label =
+ MLX5_GET(fte_match_set_misc2, mask, inner_first_mpls.mpls_label);
+ spec->inner_first_mpls_exp =
+ MLX5_GET(fte_match_set_misc2, mask, inner_first_mpls.mpls_exp);
+ spec->inner_first_mpls_s_bos =
+ MLX5_GET(fte_match_set_misc2, mask, inner_first_mpls.mpls_s_bos);
+ spec->inner_first_mpls_ttl =
+ MLX5_GET(fte_match_set_misc2, mask, inner_first_mpls.mpls_ttl);
+ spec->outer_first_mpls_over_gre_label =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_gre.mpls_label);
+ spec->outer_first_mpls_over_gre_exp =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_gre.mpls_exp);
+ spec->outer_first_mpls_over_gre_s_bos =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_gre.mpls_s_bos);
+ spec->outer_first_mpls_over_gre_ttl =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_gre.mpls_ttl);
+ spec->outer_first_mpls_over_udp_label =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_udp.mpls_label);
+ spec->outer_first_mpls_over_udp_exp =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_udp.mpls_exp);
+ spec->outer_first_mpls_over_udp_s_bos =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_udp.mpls_s_bos);
+ spec->outer_first_mpls_over_udp_ttl =
+ MLX5_GET(fte_match_set_misc2, mask, outer_first_mpls_over_udp.mpls_ttl);
+ spec->metadata_reg_c_7 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_7);
+ spec->metadata_reg_c_6 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_6);
+ spec->metadata_reg_c_5 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_5);
+ spec->metadata_reg_c_4 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_4);
+ spec->metadata_reg_c_3 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_3);
+ spec->metadata_reg_c_2 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_2);
+ spec->metadata_reg_c_1 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_1);
+ spec->metadata_reg_c_0 = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_c_0);
+ spec->metadata_reg_a = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_a);
+ spec->metadata_reg_b = MLX5_GET(fte_match_set_misc2, mask, metadata_reg_b);
+}
+
+static void dr_ste_copy_mask_misc3(char *mask, struct mlx5dr_match_misc3 *spec)
+{
+ spec->inner_tcp_seq_num = MLX5_GET(fte_match_set_misc3, mask, inner_tcp_seq_num);
+ spec->outer_tcp_seq_num = MLX5_GET(fte_match_set_misc3, mask, outer_tcp_seq_num);
+ spec->inner_tcp_ack_num = MLX5_GET(fte_match_set_misc3, mask, inner_tcp_ack_num);
+ spec->outer_tcp_ack_num = MLX5_GET(fte_match_set_misc3, mask, outer_tcp_ack_num);
+ spec->outer_vxlan_gpe_vni =
+ MLX5_GET(fte_match_set_misc3, mask, outer_vxlan_gpe_vni);
+ spec->outer_vxlan_gpe_next_protocol =
+ MLX5_GET(fte_match_set_misc3, mask, outer_vxlan_gpe_next_protocol);
+ spec->outer_vxlan_gpe_flags =
+ MLX5_GET(fte_match_set_misc3, mask, outer_vxlan_gpe_flags);
+ spec->icmpv4_header_data = MLX5_GET(fte_match_set_misc3, mask, icmp_header_data);
+ spec->icmpv6_header_data =
+ MLX5_GET(fte_match_set_misc3, mask, icmpv6_header_data);
+ spec->icmpv4_type = MLX5_GET(fte_match_set_misc3, mask, icmp_type);
+ spec->icmpv4_code = MLX5_GET(fte_match_set_misc3, mask, icmp_code);
+ spec->icmpv6_type = MLX5_GET(fte_match_set_misc3, mask, icmpv6_type);
+ spec->icmpv6_code = MLX5_GET(fte_match_set_misc3, mask, icmpv6_code);
+}
+
+void mlx5dr_ste_copy_param(u8 match_criteria,
+ struct mlx5dr_match_param *set_param,
+ struct mlx5dr_match_parameters *mask)
+{
+ u8 tail_param[MLX5_ST_SZ_BYTES(fte_match_set_lyr_2_4)] = {};
+ u8 *data = (u8 *)mask->match_buf;
+ size_t param_location;
+ void *buff;
+
+ if (match_criteria & DR_MATCHER_CRITERIA_OUTER) {
+ if (mask->match_sz < sizeof(struct mlx5dr_match_spec)) {
+ memcpy(tail_param, data, mask->match_sz);
+ buff = tail_param;
+ } else {
+ buff = mask->match_buf;
+ }
+ dr_ste_copy_mask_spec(buff, &set_param->outer);
+ }
+ param_location = sizeof(struct mlx5dr_match_spec);
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC) {
+ if (mask->match_sz < param_location +
+ sizeof(struct mlx5dr_match_misc)) {
+ memcpy(tail_param, data + param_location,
+ mask->match_sz - param_location);
+ buff = tail_param;
+ } else {
+ buff = data + param_location;
+ }
+ dr_ste_copy_mask_misc(buff, &set_param->misc);
+ }
+ param_location += sizeof(struct mlx5dr_match_misc);
+
+ if (match_criteria & DR_MATCHER_CRITERIA_INNER) {
+ if (mask->match_sz < param_location +
+ sizeof(struct mlx5dr_match_spec)) {
+ memcpy(tail_param, data + param_location,
+ mask->match_sz - param_location);
+ buff = tail_param;
+ } else {
+ buff = data + param_location;
+ }
+ dr_ste_copy_mask_spec(buff, &set_param->inner);
+ }
+ param_location += sizeof(struct mlx5dr_match_spec);
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC2) {
+ if (mask->match_sz < param_location +
+ sizeof(struct mlx5dr_match_misc2)) {
+ memcpy(tail_param, data + param_location,
+ mask->match_sz - param_location);
+ buff = tail_param;
+ } else {
+ buff = data + param_location;
+ }
+ dr_ste_copy_mask_misc2(buff, &set_param->misc2);
+ }
+
+ param_location += sizeof(struct mlx5dr_match_misc2);
+
+ if (match_criteria & DR_MATCHER_CRITERIA_MISC3) {
+ if (mask->match_sz < param_location +
+ sizeof(struct mlx5dr_match_misc3)) {
+ memcpy(tail_param, data + param_location,
+ mask->match_sz - param_location);
+ buff = tail_param;
+ } else {
+ buff = data + param_location;
+ }
+ dr_ste_copy_mask_misc3(buff, &set_param->misc3);
+ }
+}
+
+static int dr_ste_build_eth_l2_src_des_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l2_src_dst, tag, dmac_47_16, spec, dmac_47_16);
+ DR_STE_SET_TAG(eth_l2_src_dst, tag, dmac_15_0, spec, dmac_15_0);
+
+ if (spec->smac_47_16 || spec->smac_15_0) {
+ MLX5_SET(ste_eth_l2_src_dst, tag, smac_47_32,
+ spec->smac_47_16 >> 16);
+ MLX5_SET(ste_eth_l2_src_dst, tag, smac_31_0,
+ spec->smac_47_16 << 16 | spec->smac_15_0);
+ spec->smac_47_16 = 0;
+ spec->smac_15_0 = 0;
+ }
+
+ if (spec->ip_version) {
+ if (spec->ip_version == IP_VERSION_IPV4) {
+ MLX5_SET(ste_eth_l2_src_dst, tag, l3_type, STE_IPV4);
+ spec->ip_version = 0;
+ } else if (spec->ip_version == IP_VERSION_IPV6) {
+ MLX5_SET(ste_eth_l2_src_dst, tag, l3_type, STE_IPV6);
+ spec->ip_version = 0;
+ } else {
+ pr_info("Unsupported ip_version value\n");
+ return -EINVAL;
+ }
+ }
+
+ DR_STE_SET_TAG(eth_l2_src_dst, tag, first_vlan_id, spec, first_vid);
+ DR_STE_SET_TAG(eth_l2_src_dst, tag, first_cfi, spec, first_cfi);
+ DR_STE_SET_TAG(eth_l2_src_dst, tag, first_priority, spec, first_prio);
+
+ if (spec->cvlan_tag) {
+ MLX5_SET(ste_eth_l2_src_dst, tag, first_vlan_qualifier, DR_STE_CVLAN);
+ spec->cvlan_tag = 0;
+ } else if (spec->svlan_tag) {
+ MLX5_SET(ste_eth_l2_src_dst, tag, first_vlan_qualifier, DR_STE_SVLAN);
+ spec->svlan_tag = 0;
+ }
+ return 0;
+}
+
+int mlx5dr_ste_build_eth_l2_src_des(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ int ret;
+
+ ret = dr_ste_build_eth_l2_src_des_bit_mask(mask, inner, sb->bit_mask);
+ if (ret)
+ return ret;
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL2_SRC_DST, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l2_src_des_tag;
+
+ return 0;
+}
+
+static void dr_ste_build_eth_l3_ipv6_dst_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l3_ipv6_dst, bit_mask, dst_ip_127_96, mask, dst_ip_127_96);
+ DR_STE_SET_MASK_V(eth_l3_ipv6_dst, bit_mask, dst_ip_95_64, mask, dst_ip_95_64);
+ DR_STE_SET_MASK_V(eth_l3_ipv6_dst, bit_mask, dst_ip_63_32, mask, dst_ip_63_32);
+ DR_STE_SET_MASK_V(eth_l3_ipv6_dst, bit_mask, dst_ip_31_0, mask, dst_ip_31_0);
+}
+
+static int dr_ste_build_eth_l3_ipv6_dst_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l3_ipv6_dst, tag, dst_ip_127_96, spec, dst_ip_127_96);
+ DR_STE_SET_TAG(eth_l3_ipv6_dst, tag, dst_ip_95_64, spec, dst_ip_95_64);
+ DR_STE_SET_TAG(eth_l3_ipv6_dst, tag, dst_ip_63_32, spec, dst_ip_63_32);
+ DR_STE_SET_TAG(eth_l3_ipv6_dst, tag, dst_ip_31_0, spec, dst_ip_31_0);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_eth_l3_ipv6_dst(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l3_ipv6_dst_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL3_IPV6_DST, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l3_ipv6_dst_tag;
+}
+
+static void dr_ste_build_eth_l3_ipv6_src_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l3_ipv6_src, bit_mask, src_ip_127_96, mask, src_ip_127_96);
+ DR_STE_SET_MASK_V(eth_l3_ipv6_src, bit_mask, src_ip_95_64, mask, src_ip_95_64);
+ DR_STE_SET_MASK_V(eth_l3_ipv6_src, bit_mask, src_ip_63_32, mask, src_ip_63_32);
+ DR_STE_SET_MASK_V(eth_l3_ipv6_src, bit_mask, src_ip_31_0, mask, src_ip_31_0);
+}
+
+static int dr_ste_build_eth_l3_ipv6_src_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l3_ipv6_src, tag, src_ip_127_96, spec, src_ip_127_96);
+ DR_STE_SET_TAG(eth_l3_ipv6_src, tag, src_ip_95_64, spec, src_ip_95_64);
+ DR_STE_SET_TAG(eth_l3_ipv6_src, tag, src_ip_63_32, spec, src_ip_63_32);
+ DR_STE_SET_TAG(eth_l3_ipv6_src, tag, src_ip_31_0, spec, src_ip_31_0);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_eth_l3_ipv6_src(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l3_ipv6_src_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL3_IPV6_SRC, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l3_ipv6_src_tag;
+}
+
+static void dr_ste_build_eth_l3_ipv4_5_tuple_bit_mask(struct mlx5dr_match_param *value,
+ bool inner,
+ u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ destination_address, mask, dst_ip_31_0);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ source_address, mask, src_ip_31_0);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ destination_port, mask, tcp_dport);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ destination_port, mask, udp_dport);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ source_port, mask, tcp_sport);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ source_port, mask, udp_sport);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ protocol, mask, ip_protocol);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ fragmented, mask, frag);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ dscp, mask, ip_dscp);
+ DR_STE_SET_MASK_V(eth_l3_ipv4_5_tuple, bit_mask,
+ ecn, mask, ip_ecn);
+
+ if (mask->tcp_flags) {
+ DR_STE_SET_TCP_FLAGS(eth_l3_ipv4_5_tuple, bit_mask, mask);
+ mask->tcp_flags = 0;
+ }
+}
+
+static int dr_ste_build_eth_l3_ipv4_5_tuple_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, destination_address, spec, dst_ip_31_0);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, source_address, spec, src_ip_31_0);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, destination_port, spec, tcp_dport);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, destination_port, spec, udp_dport);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, source_port, spec, tcp_sport);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, source_port, spec, udp_sport);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, protocol, spec, ip_protocol);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, fragmented, spec, frag);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, dscp, spec, ip_dscp);
+ DR_STE_SET_TAG(eth_l3_ipv4_5_tuple, tag, ecn, spec, ip_ecn);
+
+ if (spec->tcp_flags) {
+ DR_STE_SET_TCP_FLAGS(eth_l3_ipv4_5_tuple, tag, spec);
+ spec->tcp_flags = 0;
+ }
+
+ return 0;
+}
+
+void mlx5dr_ste_build_eth_l3_ipv4_5_tuple(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l3_ipv4_5_tuple_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL3_IPV4_5_TUPLE, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l3_ipv4_5_tuple_tag;
+}
+
+static void
+dr_ste_build_eth_l2_src_or_dst_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+ struct mlx5dr_match_misc *misc_mask = &value->misc;
+
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, first_vlan_id, mask, first_vid);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, first_cfi, mask, first_cfi);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, first_priority, mask, first_prio);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, ip_fragmented, mask, frag);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, l3_ethertype, mask, ethertype);
+ DR_STE_SET_MASK(eth_l2_src, bit_mask, l3_type, mask, ip_version);
+
+ if (mask->svlan_tag || mask->cvlan_tag) {
+ MLX5_SET(ste_eth_l2_src, bit_mask, first_vlan_qualifier, -1);
+ mask->cvlan_tag = 0;
+ mask->svlan_tag = 0;
+ }
+
+ if (inner) {
+ if (misc_mask->inner_second_cvlan_tag ||
+ misc_mask->inner_second_svlan_tag) {
+ MLX5_SET(ste_eth_l2_src, bit_mask, second_vlan_qualifier, -1);
+ misc_mask->inner_second_cvlan_tag = 0;
+ misc_mask->inner_second_svlan_tag = 0;
+ }
+
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask,
+ second_vlan_id, misc_mask, inner_second_vid);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask,
+ second_cfi, misc_mask, inner_second_cfi);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask,
+ second_priority, misc_mask, inner_second_prio);
+ } else {
+ if (misc_mask->outer_second_cvlan_tag ||
+ misc_mask->outer_second_svlan_tag) {
+ MLX5_SET(ste_eth_l2_src, bit_mask, second_vlan_qualifier, -1);
+ misc_mask->outer_second_cvlan_tag = 0;
+ misc_mask->outer_second_svlan_tag = 0;
+ }
+
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask,
+ second_vlan_id, misc_mask, outer_second_vid);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask,
+ second_cfi, misc_mask, outer_second_cfi);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask,
+ second_priority, misc_mask, outer_second_prio);
+ }
+}
+
+static int dr_ste_build_eth_l2_src_or_dst_tag(struct mlx5dr_match_param *value,
+ bool inner, u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_spec *spec = inner ? &value->inner : &value->outer;
+ struct mlx5dr_match_misc *misc_spec = &value->misc;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l2_src, tag, first_vlan_id, spec, first_vid);
+ DR_STE_SET_TAG(eth_l2_src, tag, first_cfi, spec, first_cfi);
+ DR_STE_SET_TAG(eth_l2_src, tag, first_priority, spec, first_prio);
+ DR_STE_SET_TAG(eth_l2_src, tag, ip_fragmented, spec, frag);
+ DR_STE_SET_TAG(eth_l2_src, tag, l3_ethertype, spec, ethertype);
+
+ if (spec->ip_version) {
+ if (spec->ip_version == IP_VERSION_IPV4) {
+ MLX5_SET(ste_eth_l2_src, tag, l3_type, STE_IPV4);
+ spec->ip_version = 0;
+ } else if (spec->ip_version == IP_VERSION_IPV6) {
+ MLX5_SET(ste_eth_l2_src, tag, l3_type, STE_IPV6);
+ spec->ip_version = 0;
+ } else {
+ pr_info("Unsupported ip_version value\n");
+ return -EINVAL;
+ }
+ }
+
+ if (spec->cvlan_tag) {
+ MLX5_SET(ste_eth_l2_src, tag, first_vlan_qualifier, DR_STE_CVLAN);
+ spec->cvlan_tag = 0;
+ } else if (spec->svlan_tag) {
+ MLX5_SET(ste_eth_l2_src, tag, first_vlan_qualifier, DR_STE_SVLAN);
+ spec->svlan_tag = 0;
+ }
+
+ if (inner) {
+ if (misc_spec->inner_second_cvlan_tag) {
+ MLX5_SET(ste_eth_l2_src, tag, second_vlan_qualifier, DR_STE_CVLAN);
+ misc_spec->inner_second_cvlan_tag = 0;
+ } else if (misc_spec->inner_second_svlan_tag) {
+ MLX5_SET(ste_eth_l2_src, tag, second_vlan_qualifier, DR_STE_SVLAN);
+ misc_spec->inner_second_svlan_tag = 0;
+ }
+
+ DR_STE_SET_TAG(eth_l2_src, tag, second_vlan_id, misc_spec, inner_second_vid);
+ DR_STE_SET_TAG(eth_l2_src, tag, second_cfi, misc_spec, inner_second_cfi);
+ DR_STE_SET_TAG(eth_l2_src, tag, second_priority, misc_spec, inner_second_prio);
+ } else {
+ if (misc_spec->outer_second_cvlan_tag) {
+ MLX5_SET(ste_eth_l2_src, tag, second_vlan_qualifier, DR_STE_CVLAN);
+ misc_spec->outer_second_cvlan_tag = 0;
+ } else if (misc_spec->outer_second_svlan_tag) {
+ MLX5_SET(ste_eth_l2_src, tag, second_vlan_qualifier, DR_STE_SVLAN);
+ misc_spec->outer_second_svlan_tag = 0;
+ }
+ DR_STE_SET_TAG(eth_l2_src, tag, second_vlan_id, misc_spec, outer_second_vid);
+ DR_STE_SET_TAG(eth_l2_src, tag, second_cfi, misc_spec, outer_second_cfi);
+ DR_STE_SET_TAG(eth_l2_src, tag, second_priority, misc_spec, outer_second_prio);
+ }
+
+ return 0;
+}
+
+static void dr_ste_build_eth_l2_src_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, smac_47_16, mask, smac_47_16);
+ DR_STE_SET_MASK_V(eth_l2_src, bit_mask, smac_15_0, mask, smac_15_0);
+
+ dr_ste_build_eth_l2_src_or_dst_bit_mask(value, inner, bit_mask);
+}
+
+static int dr_ste_build_eth_l2_src_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l2_src, tag, smac_47_16, spec, smac_47_16);
+ DR_STE_SET_TAG(eth_l2_src, tag, smac_15_0, spec, smac_15_0);
+
+ return dr_ste_build_eth_l2_src_or_dst_tag(value, sb->inner, hw_ste_p);
+}
+
+void mlx5dr_ste_build_eth_l2_src(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l2_src_bit_mask(mask, inner, sb->bit_mask);
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL2_SRC, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l2_src_tag;
+}
+
+static void dr_ste_build_eth_l2_dst_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l2_dst, bit_mask, dmac_47_16, mask, dmac_47_16);
+ DR_STE_SET_MASK_V(eth_l2_dst, bit_mask, dmac_15_0, mask, dmac_15_0);
+
+ dr_ste_build_eth_l2_src_or_dst_bit_mask(value, inner, bit_mask);
+}
+
+static int dr_ste_build_eth_l2_dst_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l2_dst, tag, dmac_47_16, spec, dmac_47_16);
+ DR_STE_SET_TAG(eth_l2_dst, tag, dmac_15_0, spec, dmac_15_0);
+
+ return dr_ste_build_eth_l2_src_or_dst_tag(value, sb->inner, hw_ste_p);
+}
+
+void mlx5dr_ste_build_eth_l2_dst(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l2_dst_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL2_DST, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l2_dst_tag;
+}
+
+static void dr_ste_build_eth_l2_tnl_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+ struct mlx5dr_match_misc *misc = &value->misc;
+
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, dmac_47_16, mask, dmac_47_16);
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, dmac_15_0, mask, dmac_15_0);
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, first_vlan_id, mask, first_vid);
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, first_cfi, mask, first_cfi);
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, first_priority, mask, first_prio);
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, ip_fragmented, mask, frag);
+ DR_STE_SET_MASK_V(eth_l2_tnl, bit_mask, l3_ethertype, mask, ethertype);
+ DR_STE_SET_MASK(eth_l2_tnl, bit_mask, l3_type, mask, ip_version);
+
+ if (misc->vxlan_vni) {
+ MLX5_SET(ste_eth_l2_tnl, bit_mask,
+ l2_tunneling_network_id, (misc->vxlan_vni << 8));
+ misc->vxlan_vni = 0;
+ }
+
+ if (mask->svlan_tag || mask->cvlan_tag) {
+ MLX5_SET(ste_eth_l2_tnl, bit_mask, first_vlan_qualifier, -1);
+ mask->cvlan_tag = 0;
+ mask->svlan_tag = 0;
+ }
+}
+
+static int dr_ste_build_eth_l2_tnl_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc *misc = &value->misc;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l2_tnl, tag, dmac_47_16, spec, dmac_47_16);
+ DR_STE_SET_TAG(eth_l2_tnl, tag, dmac_15_0, spec, dmac_15_0);
+ DR_STE_SET_TAG(eth_l2_tnl, tag, first_vlan_id, spec, first_vid);
+ DR_STE_SET_TAG(eth_l2_tnl, tag, first_cfi, spec, first_cfi);
+ DR_STE_SET_TAG(eth_l2_tnl, tag, ip_fragmented, spec, frag);
+ DR_STE_SET_TAG(eth_l2_tnl, tag, first_priority, spec, first_prio);
+ DR_STE_SET_TAG(eth_l2_tnl, tag, l3_ethertype, spec, ethertype);
+
+ if (misc->vxlan_vni) {
+ MLX5_SET(ste_eth_l2_tnl, tag, l2_tunneling_network_id,
+ (misc->vxlan_vni << 8));
+ misc->vxlan_vni = 0;
+ }
+
+ if (spec->cvlan_tag) {
+ MLX5_SET(ste_eth_l2_tnl, tag, first_vlan_qualifier, DR_STE_CVLAN);
+ spec->cvlan_tag = 0;
+ } else if (spec->svlan_tag) {
+ MLX5_SET(ste_eth_l2_tnl, tag, first_vlan_qualifier, DR_STE_SVLAN);
+ spec->svlan_tag = 0;
+ }
+
+ if (spec->ip_version) {
+ if (spec->ip_version == IP_VERSION_IPV4) {
+ MLX5_SET(ste_eth_l2_tnl, tag, l3_type, STE_IPV4);
+ spec->ip_version = 0;
+ } else if (spec->ip_version == IP_VERSION_IPV6) {
+ MLX5_SET(ste_eth_l2_tnl, tag, l3_type, STE_IPV6);
+ spec->ip_version = 0;
+ } else {
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+void mlx5dr_ste_build_eth_l2_tnl(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask, bool inner, bool rx)
+{
+ dr_ste_build_eth_l2_tnl_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_ETHL2_TUNNELING_I;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l2_tnl_tag;
+}
+
+static void dr_ste_build_eth_l3_ipv4_misc_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l3_ipv4_misc, bit_mask, time_to_live, mask, ttl_hoplimit);
+}
+
+static int dr_ste_build_eth_l3_ipv4_misc_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l3_ipv4_misc, tag, time_to_live, spec, ttl_hoplimit);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_eth_l3_ipv4_misc(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l3_ipv4_misc_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL3_IPV4_MISC, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l3_ipv4_misc_tag;
+}
+
+static void dr_ste_build_ipv6_l3_l4_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_spec *mask = inner ? &value->inner : &value->outer;
+
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, dst_port, mask, tcp_dport);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, src_port, mask, tcp_sport);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, dst_port, mask, udp_dport);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, src_port, mask, udp_sport);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, protocol, mask, ip_protocol);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, fragmented, mask, frag);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, dscp, mask, ip_dscp);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, ecn, mask, ip_ecn);
+ DR_STE_SET_MASK_V(eth_l4, bit_mask, ipv6_hop_limit, mask, ttl_hoplimit);
+
+ if (mask->tcp_flags) {
+ DR_STE_SET_TCP_FLAGS(eth_l4, bit_mask, mask);
+ mask->tcp_flags = 0;
+ }
+}
+
+static int dr_ste_build_ipv6_l3_l4_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct mlx5dr_match_spec *spec = sb->inner ? &value->inner : &value->outer;
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(eth_l4, tag, dst_port, spec, tcp_dport);
+ DR_STE_SET_TAG(eth_l4, tag, src_port, spec, tcp_sport);
+ DR_STE_SET_TAG(eth_l4, tag, dst_port, spec, udp_dport);
+ DR_STE_SET_TAG(eth_l4, tag, src_port, spec, udp_sport);
+ DR_STE_SET_TAG(eth_l4, tag, protocol, spec, ip_protocol);
+ DR_STE_SET_TAG(eth_l4, tag, fragmented, spec, frag);
+ DR_STE_SET_TAG(eth_l4, tag, dscp, spec, ip_dscp);
+ DR_STE_SET_TAG(eth_l4, tag, ecn, spec, ip_ecn);
+ DR_STE_SET_TAG(eth_l4, tag, ipv6_hop_limit, spec, ttl_hoplimit);
+
+ if (spec->tcp_flags) {
+ DR_STE_SET_TCP_FLAGS(eth_l4, tag, spec);
+ spec->tcp_flags = 0;
+ }
+
+ return 0;
+}
+
+void mlx5dr_ste_build_ipv6_l3_l4(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_ipv6_l3_l4_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL4, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_ipv6_l3_l4_tag;
+}
+
+static int dr_ste_build_empty_always_hit_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ return 0;
+}
+
+void mlx5dr_ste_build_empty_always_hit(struct mlx5dr_ste_build *sb, bool rx)
+{
+ sb->rx = rx;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_DONT_CARE;
+ sb->byte_mask = 0;
+ sb->ste_build_tag_func = &dr_ste_build_empty_always_hit_tag;
+}
+
+static void dr_ste_build_mpls_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_misc2 *misc2_mask = &value->misc2;
+
+ if (inner)
+ DR_STE_SET_MPLS_MASK(mpls, misc2_mask, inner, bit_mask);
+ else
+ DR_STE_SET_MPLS_MASK(mpls, misc2_mask, outer, bit_mask);
+}
+
+static int dr_ste_build_mpls_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc2 *misc2_mask = &value->misc2;
+ u8 *tag = hw_ste->tag;
+
+ if (sb->inner)
+ DR_STE_SET_MPLS_TAG(mpls, misc2_mask, inner, tag);
+ else
+ DR_STE_SET_MPLS_TAG(mpls, misc2_mask, outer, tag);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_mpls(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_mpls_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(MPLS_FIRST, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_mpls_tag;
+}
+
+static void dr_ste_build_gre_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_misc *misc_mask = &value->misc;
+
+ DR_STE_SET_MASK_V(gre, bit_mask, gre_protocol, misc_mask, gre_protocol);
+ DR_STE_SET_MASK_V(gre, bit_mask, gre_k_present, misc_mask, gre_k_present);
+ DR_STE_SET_MASK_V(gre, bit_mask, gre_key_h, misc_mask, gre_key_h);
+ DR_STE_SET_MASK_V(gre, bit_mask, gre_key_l, misc_mask, gre_key_l);
+
+ DR_STE_SET_MASK_V(gre, bit_mask, gre_c_present, misc_mask, gre_c_present);
+ DR_STE_SET_MASK_V(gre, bit_mask, gre_s_present, misc_mask, gre_s_present);
+}
+
+static int dr_ste_build_gre_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc *misc = &value->misc;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(gre, tag, gre_protocol, misc, gre_protocol);
+
+ DR_STE_SET_TAG(gre, tag, gre_k_present, misc, gre_k_present);
+ DR_STE_SET_TAG(gre, tag, gre_key_h, misc, gre_key_h);
+ DR_STE_SET_TAG(gre, tag, gre_key_l, misc, gre_key_l);
+
+ DR_STE_SET_TAG(gre, tag, gre_c_present, misc, gre_c_present);
+
+ DR_STE_SET_TAG(gre, tag, gre_s_present, misc, gre_s_present);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_gre(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask, bool inner, bool rx)
+{
+ dr_ste_build_gre_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_GRE;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_gre_tag;
+}
+
+static void dr_ste_build_flex_parser_0_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_misc2 *misc_2_mask = &value->misc2;
+
+ if (DR_STE_IS_OUTER_MPLS_OVER_GRE_SET(misc_2_mask)) {
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_label,
+ misc_2_mask, outer_first_mpls_over_gre_label);
+
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_exp,
+ misc_2_mask, outer_first_mpls_over_gre_exp);
+
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_s_bos,
+ misc_2_mask, outer_first_mpls_over_gre_s_bos);
+
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_ttl,
+ misc_2_mask, outer_first_mpls_over_gre_ttl);
+ } else {
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_label,
+ misc_2_mask, outer_first_mpls_over_udp_label);
+
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_exp,
+ misc_2_mask, outer_first_mpls_over_udp_exp);
+
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_s_bos,
+ misc_2_mask, outer_first_mpls_over_udp_s_bos);
+
+ DR_STE_SET_MASK_V(flex_parser_0, bit_mask, parser_3_ttl,
+ misc_2_mask, outer_first_mpls_over_udp_ttl);
+ }
+}
+
+static int dr_ste_build_flex_parser_0_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc2 *misc_2_mask = &value->misc2;
+ u8 *tag = hw_ste->tag;
+
+ if (DR_STE_IS_OUTER_MPLS_OVER_GRE_SET(misc_2_mask)) {
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_label,
+ misc_2_mask, outer_first_mpls_over_gre_label);
+
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_exp,
+ misc_2_mask, outer_first_mpls_over_gre_exp);
+
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_s_bos,
+ misc_2_mask, outer_first_mpls_over_gre_s_bos);
+
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_ttl,
+ misc_2_mask, outer_first_mpls_over_gre_ttl);
+ } else {
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_label,
+ misc_2_mask, outer_first_mpls_over_udp_label);
+
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_exp,
+ misc_2_mask, outer_first_mpls_over_udp_exp);
+
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_s_bos,
+ misc_2_mask, outer_first_mpls_over_udp_s_bos);
+
+ DR_STE_SET_TAG(flex_parser_0, tag, parser_3_ttl,
+ misc_2_mask, outer_first_mpls_over_udp_ttl);
+ }
+ return 0;
+}
+
+void mlx5dr_ste_build_flex_parser_0(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_flex_parser_0_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_FLEX_PARSER_0;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_flex_parser_0_tag;
+}
+
+#define ICMP_TYPE_OFFSET_FIRST_DW 24
+#define ICMP_CODE_OFFSET_FIRST_DW 16
+#define ICMP_HEADER_DATA_OFFSET_SECOND_DW 0
+
+static int dr_ste_build_flex_parser_1_bit_mask(struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ u8 *bit_mask)
+{
+ struct mlx5dr_match_misc3 *misc_3_mask = &mask->misc3;
+ bool is_ipv4_mask = DR_MASK_IS_FLEX_PARSER_ICMPV4_SET(misc_3_mask);
+ u32 icmp_header_data_mask;
+ u32 icmp_type_mask;
+ u32 icmp_code_mask;
+ int dw0_location;
+ int dw1_location;
+
+ if (is_ipv4_mask) {
+ icmp_header_data_mask = misc_3_mask->icmpv4_header_data;
+ icmp_type_mask = misc_3_mask->icmpv4_type;
+ icmp_code_mask = misc_3_mask->icmpv4_code;
+ dw0_location = caps->flex_parser_id_icmp_dw0;
+ dw1_location = caps->flex_parser_id_icmp_dw1;
+ } else {
+ icmp_header_data_mask = misc_3_mask->icmpv6_header_data;
+ icmp_type_mask = misc_3_mask->icmpv6_type;
+ icmp_code_mask = misc_3_mask->icmpv6_code;
+ dw0_location = caps->flex_parser_id_icmpv6_dw0;
+ dw1_location = caps->flex_parser_id_icmpv6_dw1;
+ }
+
+ switch (dw0_location) {
+ case 4:
+ if (icmp_type_mask) {
+ MLX5_SET(ste_flex_parser_1, bit_mask, flex_parser_4,
+ (icmp_type_mask << ICMP_TYPE_OFFSET_FIRST_DW));
+ if (is_ipv4_mask)
+ misc_3_mask->icmpv4_type = 0;
+ else
+ misc_3_mask->icmpv6_type = 0;
+ }
+ if (icmp_code_mask) {
+ u32 cur_val = MLX5_GET(ste_flex_parser_1, bit_mask,
+ flex_parser_4);
+ MLX5_SET(ste_flex_parser_1, bit_mask, flex_parser_4,
+ cur_val | (icmp_code_mask << ICMP_CODE_OFFSET_FIRST_DW));
+ if (is_ipv4_mask)
+ misc_3_mask->icmpv4_code = 0;
+ else
+ misc_3_mask->icmpv6_code = 0;
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (dw1_location) {
+ case 5:
+ if (icmp_header_data_mask) {
+ MLX5_SET(ste_flex_parser_1, bit_mask, flex_parser_5,
+ (icmp_header_data_mask << ICMP_HEADER_DATA_OFFSET_SECOND_DW));
+ if (is_ipv4_mask)
+ misc_3_mask->icmpv4_header_data = 0;
+ else
+ misc_3_mask->icmpv6_header_data = 0;
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int dr_ste_build_flex_parser_1_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc3 *misc_3 = &value->misc3;
+ u8 *tag = hw_ste->tag;
+ u32 icmp_header_data;
+ int dw0_location;
+ int dw1_location;
+ u32 icmp_type;
+ u32 icmp_code;
+ bool is_ipv4;
+
+ is_ipv4 = DR_MASK_IS_FLEX_PARSER_ICMPV4_SET(misc_3);
+ if (is_ipv4) {
+ icmp_header_data = misc_3->icmpv4_header_data;
+ icmp_type = misc_3->icmpv4_type;
+ icmp_code = misc_3->icmpv4_code;
+ dw0_location = sb->caps->flex_parser_id_icmp_dw0;
+ dw1_location = sb->caps->flex_parser_id_icmp_dw1;
+ } else {
+ icmp_header_data = misc_3->icmpv6_header_data;
+ icmp_type = misc_3->icmpv6_type;
+ icmp_code = misc_3->icmpv6_code;
+ dw0_location = sb->caps->flex_parser_id_icmpv6_dw0;
+ dw1_location = sb->caps->flex_parser_id_icmpv6_dw1;
+ }
+
+ switch (dw0_location) {
+ case 4:
+ if (icmp_type) {
+ MLX5_SET(ste_flex_parser_1, tag, flex_parser_4,
+ (icmp_type << ICMP_TYPE_OFFSET_FIRST_DW));
+ if (is_ipv4)
+ misc_3->icmpv4_type = 0;
+ else
+ misc_3->icmpv6_type = 0;
+ }
+
+ if (icmp_code) {
+ u32 cur_val = MLX5_GET(ste_flex_parser_1, tag,
+ flex_parser_4);
+ MLX5_SET(ste_flex_parser_1, tag, flex_parser_4,
+ cur_val | (icmp_code << ICMP_CODE_OFFSET_FIRST_DW));
+ if (is_ipv4)
+ misc_3->icmpv4_code = 0;
+ else
+ misc_3->icmpv6_code = 0;
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (dw1_location) {
+ case 5:
+ if (icmp_header_data) {
+ MLX5_SET(ste_flex_parser_1, tag, flex_parser_5,
+ (icmp_header_data << ICMP_HEADER_DATA_OFFSET_SECOND_DW));
+ if (is_ipv4)
+ misc_3->icmpv4_header_data = 0;
+ else
+ misc_3->icmpv6_header_data = 0;
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int mlx5dr_ste_build_flex_parser_1(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ bool inner, bool rx)
+{
+ int ret;
+
+ ret = dr_ste_build_flex_parser_1_bit_mask(mask, caps, sb->bit_mask);
+ if (ret)
+ return ret;
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->caps = caps;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_FLEX_PARSER_1;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_flex_parser_1_tag;
+
+ return 0;
+}
+
+static void dr_ste_build_general_purpose_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_misc2 *misc_2_mask = &value->misc2;
+
+ DR_STE_SET_MASK_V(general_purpose, bit_mask,
+ general_purpose_lookup_field, misc_2_mask,
+ metadata_reg_a);
+}
+
+static int dr_ste_build_general_purpose_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc2 *misc_2_mask = &value->misc2;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(general_purpose, tag, general_purpose_lookup_field,
+ misc_2_mask, metadata_reg_a);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_general_purpose(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_general_purpose_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_GENERAL_PURPOSE;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_general_purpose_tag;
+}
+
+static void dr_ste_build_eth_l4_misc_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_misc3 *misc_3_mask = &value->misc3;
+
+ if (inner) {
+ DR_STE_SET_MASK_V(eth_l4_misc, bit_mask, seq_num, misc_3_mask,
+ inner_tcp_seq_num);
+ DR_STE_SET_MASK_V(eth_l4_misc, bit_mask, ack_num, misc_3_mask,
+ inner_tcp_ack_num);
+ } else {
+ DR_STE_SET_MASK_V(eth_l4_misc, bit_mask, seq_num, misc_3_mask,
+ outer_tcp_seq_num);
+ DR_STE_SET_MASK_V(eth_l4_misc, bit_mask, ack_num, misc_3_mask,
+ outer_tcp_ack_num);
+ }
+}
+
+static int dr_ste_build_eth_l4_misc_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc3 *misc3 = &value->misc3;
+ u8 *tag = hw_ste->tag;
+
+ if (sb->inner) {
+ DR_STE_SET_TAG(eth_l4_misc, tag, seq_num, misc3, inner_tcp_seq_num);
+ DR_STE_SET_TAG(eth_l4_misc, tag, ack_num, misc3, inner_tcp_ack_num);
+ } else {
+ DR_STE_SET_TAG(eth_l4_misc, tag, seq_num, misc3, outer_tcp_seq_num);
+ DR_STE_SET_TAG(eth_l4_misc, tag, ack_num, misc3, outer_tcp_ack_num);
+ }
+
+ return 0;
+}
+
+void mlx5dr_ste_build_eth_l4_misc(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_eth_l4_misc_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = DR_STE_CALC_LU_TYPE(ETHL4_MISC, rx, inner);
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_eth_l4_misc_tag;
+}
+
+static void dr_ste_build_flex_parser_tnl_bit_mask(struct mlx5dr_match_param *value,
+ bool inner, u8 *bit_mask)
+{
+ struct mlx5dr_match_misc3 *misc_3_mask = &value->misc3;
+
+ if (misc_3_mask->outer_vxlan_gpe_flags ||
+ misc_3_mask->outer_vxlan_gpe_next_protocol) {
+ MLX5_SET(ste_flex_parser_tnl, bit_mask,
+ flex_parser_tunneling_header_63_32,
+ (misc_3_mask->outer_vxlan_gpe_flags << 24) |
+ (misc_3_mask->outer_vxlan_gpe_next_protocol));
+ misc_3_mask->outer_vxlan_gpe_flags = 0;
+ misc_3_mask->outer_vxlan_gpe_next_protocol = 0;
+ }
+
+ if (misc_3_mask->outer_vxlan_gpe_vni) {
+ MLX5_SET(ste_flex_parser_tnl, bit_mask,
+ flex_parser_tunneling_header_31_0,
+ misc_3_mask->outer_vxlan_gpe_vni << 8);
+ misc_3_mask->outer_vxlan_gpe_vni = 0;
+ }
+}
+
+static int dr_ste_build_flex_parser_tnl_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc3 *misc3 = &value->misc3;
+ u8 *tag = hw_ste->tag;
+
+ if (misc3->outer_vxlan_gpe_flags ||
+ misc3->outer_vxlan_gpe_next_protocol) {
+ MLX5_SET(ste_flex_parser_tnl, tag,
+ flex_parser_tunneling_header_63_32,
+ (misc3->outer_vxlan_gpe_flags << 24) |
+ (misc3->outer_vxlan_gpe_next_protocol));
+ misc3->outer_vxlan_gpe_flags = 0;
+ misc3->outer_vxlan_gpe_next_protocol = 0;
+ }
+
+ if (misc3->outer_vxlan_gpe_vni) {
+ MLX5_SET(ste_flex_parser_tnl, tag,
+ flex_parser_tunneling_header_31_0,
+ misc3->outer_vxlan_gpe_vni << 8);
+ misc3->outer_vxlan_gpe_vni = 0;
+ }
+
+ return 0;
+}
+
+void mlx5dr_ste_build_flex_parser_tnl(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_flex_parser_tnl_bit_mask(mask, inner, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_FLEX_PARSER_TNL_HEADER;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_flex_parser_tnl_tag;
+}
+
+static void dr_ste_build_register_0_bit_mask(struct mlx5dr_match_param *value,
+ u8 *bit_mask)
+{
+ struct mlx5dr_match_misc2 *misc_2_mask = &value->misc2;
+
+ DR_STE_SET_MASK_V(register_0, bit_mask, register_0_h,
+ misc_2_mask, metadata_reg_c_0);
+ DR_STE_SET_MASK_V(register_0, bit_mask, register_0_l,
+ misc_2_mask, metadata_reg_c_1);
+ DR_STE_SET_MASK_V(register_0, bit_mask, register_1_h,
+ misc_2_mask, metadata_reg_c_2);
+ DR_STE_SET_MASK_V(register_0, bit_mask, register_1_l,
+ misc_2_mask, metadata_reg_c_3);
+}
+
+static int dr_ste_build_register_0_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc2 *misc2 = &value->misc2;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(register_0, tag, register_0_h, misc2, metadata_reg_c_0);
+ DR_STE_SET_TAG(register_0, tag, register_0_l, misc2, metadata_reg_c_1);
+ DR_STE_SET_TAG(register_0, tag, register_1_h, misc2, metadata_reg_c_2);
+ DR_STE_SET_TAG(register_0, tag, register_1_l, misc2, metadata_reg_c_3);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_register_0(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_register_0_bit_mask(mask, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_STEERING_REGISTERS_0;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_register_0_tag;
+}
+
+static void dr_ste_build_register_1_bit_mask(struct mlx5dr_match_param *value,
+ u8 *bit_mask)
+{
+ struct mlx5dr_match_misc2 *misc_2_mask = &value->misc2;
+
+ DR_STE_SET_MASK_V(register_1, bit_mask, register_2_h,
+ misc_2_mask, metadata_reg_c_4);
+ DR_STE_SET_MASK_V(register_1, bit_mask, register_2_l,
+ misc_2_mask, metadata_reg_c_5);
+ DR_STE_SET_MASK_V(register_1, bit_mask, register_3_h,
+ misc_2_mask, metadata_reg_c_6);
+ DR_STE_SET_MASK_V(register_1, bit_mask, register_3_l,
+ misc_2_mask, metadata_reg_c_7);
+}
+
+static int dr_ste_build_register_1_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc2 *misc2 = &value->misc2;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(register_1, tag, register_2_h, misc2, metadata_reg_c_4);
+ DR_STE_SET_TAG(register_1, tag, register_2_l, misc2, metadata_reg_c_5);
+ DR_STE_SET_TAG(register_1, tag, register_3_h, misc2, metadata_reg_c_6);
+ DR_STE_SET_TAG(register_1, tag, register_3_l, misc2, metadata_reg_c_7);
+
+ return 0;
+}
+
+void mlx5dr_ste_build_register_1(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx)
+{
+ dr_ste_build_register_1_bit_mask(mask, sb->bit_mask);
+
+ sb->rx = rx;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_STEERING_REGISTERS_1;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_register_1_tag;
+}
+
+static int dr_ste_build_src_gvmi_qpn_bit_mask(struct mlx5dr_match_param *value,
+ u8 *bit_mask)
+{
+ struct mlx5dr_match_misc *misc_mask = &value->misc;
+
+ if (misc_mask->source_port != 0xffff)
+ return -EINVAL;
+
+ DR_STE_SET_MASK(src_gvmi_qp, bit_mask, source_gvmi, misc_mask, source_port);
+ DR_STE_SET_MASK(src_gvmi_qp, bit_mask, source_qp, misc_mask, source_sqn);
+
+ return 0;
+}
+
+static int dr_ste_build_src_gvmi_qpn_tag(struct mlx5dr_match_param *value,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p)
+{
+ struct dr_hw_ste_format *hw_ste = (struct dr_hw_ste_format *)hw_ste_p;
+ struct mlx5dr_match_misc *misc = &value->misc;
+ struct mlx5dr_cmd_vport_cap *vport_cap;
+ u8 *tag = hw_ste->tag;
+
+ DR_STE_SET_TAG(src_gvmi_qp, tag, source_qp, misc, source_sqn);
+
+ vport_cap = mlx5dr_get_vport_cap(sb->caps, misc->source_port);
+ if (!vport_cap)
+ return -EINVAL;
+
+ if (vport_cap->vport_gvmi)
+ MLX5_SET(ste_src_gvmi_qp, tag, source_gvmi, vport_cap->vport_gvmi);
+
+ misc->source_port = 0;
+
+ return 0;
+}
+
+int mlx5dr_ste_build_src_gvmi_qpn(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ bool inner, bool rx)
+{
+ int ret;
+
+ ret = dr_ste_build_src_gvmi_qpn_bit_mask(mask, sb->bit_mask);
+ if (ret)
+ return ret;
+
+ sb->rx = rx;
+ sb->caps = caps;
+ sb->inner = inner;
+ sb->lu_type = MLX5DR_STE_LU_TYPE_SRC_GVMI_AND_QP;
+ sb->byte_mask = dr_ste_conv_bit_to_byte_mask(sb->bit_mask);
+ sb->ste_build_tag_func = &dr_ste_build_src_gvmi_qpn_tag;
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c
new file mode 100644
index 0000000..e178d8d
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c
@@ -0,0 +1,294 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies. */
+
+#include "dr_types.h"
+
+int mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
+ struct mlx5dr_action *action)
+{
+ struct mlx5dr_matcher *last_matcher = NULL;
+ struct mlx5dr_htbl_connect_info info;
+ struct mlx5dr_ste_htbl *last_htbl;
+ int ret;
+
+ if (action && action->action_type != DR_ACTION_TYP_FT)
+ return -EOPNOTSUPP;
+
+ mutex_lock(&tbl->dmn->mutex);
+
+ if (!list_empty(&tbl->matcher_list))
+ last_matcher = list_last_entry(&tbl->matcher_list,
+ struct mlx5dr_matcher,
+ matcher_list);
+
+ if (tbl->dmn->type == MLX5DR_DOMAIN_TYPE_NIC_RX ||
+ tbl->dmn->type == MLX5DR_DOMAIN_TYPE_FDB) {
+ if (last_matcher)
+ last_htbl = last_matcher->rx.e_anchor;
+ else
+ last_htbl = tbl->rx.s_anchor;
+
+ tbl->rx.default_icm_addr = action ?
+ action->dest_tbl.tbl->rx.s_anchor->chunk->icm_addr :
+ tbl->rx.nic_dmn->default_icm_addr;
+
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = tbl->rx.default_icm_addr;
+
+ ret = mlx5dr_ste_htbl_init_and_postsend(tbl->dmn,
+ tbl->rx.nic_dmn,
+ last_htbl,
+ &info, true);
+ if (ret) {
+ mlx5dr_dbg(tbl->dmn, "Failed to set RX miss action, ret %d\n", ret);
+ goto out;
+ }
+ }
+
+ if (tbl->dmn->type == MLX5DR_DOMAIN_TYPE_NIC_TX ||
+ tbl->dmn->type == MLX5DR_DOMAIN_TYPE_FDB) {
+ if (last_matcher)
+ last_htbl = last_matcher->tx.e_anchor;
+ else
+ last_htbl = tbl->tx.s_anchor;
+
+ tbl->tx.default_icm_addr = action ?
+ action->dest_tbl.tbl->tx.s_anchor->chunk->icm_addr :
+ tbl->tx.nic_dmn->default_icm_addr;
+
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = tbl->tx.default_icm_addr;
+
+ ret = mlx5dr_ste_htbl_init_and_postsend(tbl->dmn,
+ tbl->tx.nic_dmn,
+ last_htbl, &info, true);
+ if (ret) {
+ mlx5dr_dbg(tbl->dmn, "Failed to set TX miss action, ret %d\n", ret);
+ goto out;
+ }
+ }
+
+ /* Release old action */
+ if (tbl->miss_action)
+ refcount_dec(&tbl->miss_action->refcount);
+
+ /* Set new miss action */
+ tbl->miss_action = action;
+ if (tbl->miss_action)
+ refcount_inc(&action->refcount);
+
+out:
+ mutex_unlock(&tbl->dmn->mutex);
+ return ret;
+}
+
+static void dr_table_uninit_nic(struct mlx5dr_table_rx_tx *nic_tbl)
+{
+ mlx5dr_htbl_put(nic_tbl->s_anchor);
+}
+
+static void dr_table_uninit_fdb(struct mlx5dr_table *tbl)
+{
+ dr_table_uninit_nic(&tbl->rx);
+ dr_table_uninit_nic(&tbl->tx);
+}
+
+static void dr_table_uninit(struct mlx5dr_table *tbl)
+{
+ mutex_lock(&tbl->dmn->mutex);
+
+ switch (tbl->dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ dr_table_uninit_nic(&tbl->rx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ dr_table_uninit_nic(&tbl->tx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ dr_table_uninit_fdb(tbl);
+ break;
+ default:
+ WARN_ON(true);
+ break;
+ }
+
+ mutex_unlock(&tbl->dmn->mutex);
+}
+
+static int dr_table_init_nic(struct mlx5dr_domain *dmn,
+ struct mlx5dr_table_rx_tx *nic_tbl)
+{
+ struct mlx5dr_domain_rx_tx *nic_dmn = nic_tbl->nic_dmn;
+ struct mlx5dr_htbl_connect_info info;
+ int ret;
+
+ nic_tbl->default_icm_addr = nic_dmn->default_icm_addr;
+
+ nic_tbl->s_anchor = mlx5dr_ste_htbl_alloc(dmn->ste_icm_pool,
+ DR_CHUNK_SIZE_1,
+ MLX5DR_STE_LU_TYPE_DONT_CARE,
+ 0);
+ if (!nic_tbl->s_anchor)
+ return -ENOMEM;
+
+ info.type = CONNECT_MISS;
+ info.miss_icm_addr = nic_dmn->default_icm_addr;
+ ret = mlx5dr_ste_htbl_init_and_postsend(dmn, nic_dmn,
+ nic_tbl->s_anchor,
+ &info, true);
+ if (ret)
+ goto free_s_anchor;
+
+ mlx5dr_htbl_get(nic_tbl->s_anchor);
+
+ return 0;
+
+free_s_anchor:
+ mlx5dr_ste_htbl_free(nic_tbl->s_anchor);
+ return ret;
+}
+
+static int dr_table_init_fdb(struct mlx5dr_table *tbl)
+{
+ int ret;
+
+ ret = dr_table_init_nic(tbl->dmn, &tbl->rx);
+ if (ret)
+ return ret;
+
+ ret = dr_table_init_nic(tbl->dmn, &tbl->tx);
+ if (ret)
+ goto destroy_rx;
+
+ return 0;
+
+destroy_rx:
+ dr_table_uninit_nic(&tbl->rx);
+ return ret;
+}
+
+static int dr_table_init(struct mlx5dr_table *tbl)
+{
+ int ret = 0;
+
+ INIT_LIST_HEAD(&tbl->matcher_list);
+
+ mutex_lock(&tbl->dmn->mutex);
+
+ switch (tbl->dmn->type) {
+ case MLX5DR_DOMAIN_TYPE_NIC_RX:
+ tbl->table_type = MLX5_FLOW_TABLE_TYPE_NIC_RX;
+ tbl->rx.nic_dmn = &tbl->dmn->info.rx;
+ ret = dr_table_init_nic(tbl->dmn, &tbl->rx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_NIC_TX:
+ tbl->table_type = MLX5_FLOW_TABLE_TYPE_NIC_TX;
+ tbl->tx.nic_dmn = &tbl->dmn->info.tx;
+ ret = dr_table_init_nic(tbl->dmn, &tbl->tx);
+ break;
+ case MLX5DR_DOMAIN_TYPE_FDB:
+ tbl->table_type = MLX5_FLOW_TABLE_TYPE_FDB;
+ tbl->rx.nic_dmn = &tbl->dmn->info.rx;
+ tbl->tx.nic_dmn = &tbl->dmn->info.tx;
+ ret = dr_table_init_fdb(tbl);
+ break;
+ default:
+ WARN_ON(true);
+ break;
+ }
+
+ mutex_unlock(&tbl->dmn->mutex);
+
+ return ret;
+}
+
+static int dr_table_destroy_sw_owned_tbl(struct mlx5dr_table *tbl)
+{
+ return mlx5dr_cmd_destroy_flow_table(tbl->dmn->mdev,
+ tbl->table_id,
+ tbl->table_type);
+}
+
+static int dr_table_create_sw_owned_tbl(struct mlx5dr_table *tbl)
+{
+ u64 icm_addr_rx = 0;
+ u64 icm_addr_tx = 0;
+ int ret;
+
+ if (tbl->rx.s_anchor)
+ icm_addr_rx = tbl->rx.s_anchor->chunk->icm_addr;
+
+ if (tbl->tx.s_anchor)
+ icm_addr_tx = tbl->tx.s_anchor->chunk->icm_addr;
+
+ ret = mlx5dr_cmd_create_flow_table(tbl->dmn->mdev,
+ tbl->table_type,
+ icm_addr_rx,
+ icm_addr_tx,
+ tbl->dmn->info.caps.max_ft_level - 1,
+ true, false, NULL,
+ &tbl->table_id);
+
+ return ret;
+}
+
+struct mlx5dr_table *mlx5dr_table_create(struct mlx5dr_domain *dmn, u32 level)
+{
+ struct mlx5dr_table *tbl;
+ int ret;
+
+ refcount_inc(&dmn->refcount);
+
+ tbl = kzalloc(sizeof(*tbl), GFP_KERNEL);
+ if (!tbl)
+ goto dec_ref;
+
+ tbl->dmn = dmn;
+ tbl->level = level;
+ refcount_set(&tbl->refcount, 1);
+
+ ret = dr_table_init(tbl);
+ if (ret)
+ goto free_tbl;
+
+ ret = dr_table_create_sw_owned_tbl(tbl);
+ if (ret)
+ goto uninit_tbl;
+
+ return tbl;
+
+uninit_tbl:
+ dr_table_uninit(tbl);
+free_tbl:
+ kfree(tbl);
+dec_ref:
+ refcount_dec(&dmn->refcount);
+ return NULL;
+}
+
+int mlx5dr_table_destroy(struct mlx5dr_table *tbl)
+{
+ int ret;
+
+ if (refcount_read(&tbl->refcount) > 1)
+ return -EBUSY;
+
+ ret = dr_table_destroy_sw_owned_tbl(tbl);
+ if (ret)
+ return ret;
+
+ dr_table_uninit(tbl);
+
+ if (tbl->miss_action)
+ refcount_dec(&tbl->miss_action->refcount);
+
+ refcount_dec(&tbl->dmn->refcount);
+ kfree(tbl);
+
+ return ret;
+}
+
+u32 mlx5dr_table_get_id(struct mlx5dr_table *tbl)
+{
+ return tbl->table_id;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
new file mode 100644
index 0000000..a37ee63
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h
@@ -0,0 +1,1060 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2019, Mellanox Technologies */
+
+#ifndef _DR_TYPES_
+#define _DR_TYPES_
+
+#include <linux/mlx5/driver.h>
+#include <linux/refcount.h>
+#include "fs_core.h"
+#include "wq.h"
+#include "lib/mlx5.h"
+#include "mlx5_ifc_dr.h"
+#include "mlx5dr.h"
+
+#define DR_RULE_MAX_STES 17
+#define DR_ACTION_MAX_STES 5
+#define WIRE_PORT 0xFFFF
+#define DR_STE_SVLAN 0x1
+#define DR_STE_CVLAN 0x2
+
+#define mlx5dr_err(dmn, arg...) mlx5_core_err((dmn)->mdev, ##arg)
+#define mlx5dr_info(dmn, arg...) mlx5_core_info((dmn)->mdev, ##arg)
+#define mlx5dr_dbg(dmn, arg...) mlx5_core_dbg((dmn)->mdev, ##arg)
+
+enum mlx5dr_icm_chunk_size {
+ DR_CHUNK_SIZE_1,
+ DR_CHUNK_SIZE_MIN = DR_CHUNK_SIZE_1, /* keep updated when changing */
+ DR_CHUNK_SIZE_2,
+ DR_CHUNK_SIZE_4,
+ DR_CHUNK_SIZE_8,
+ DR_CHUNK_SIZE_16,
+ DR_CHUNK_SIZE_32,
+ DR_CHUNK_SIZE_64,
+ DR_CHUNK_SIZE_128,
+ DR_CHUNK_SIZE_256,
+ DR_CHUNK_SIZE_512,
+ DR_CHUNK_SIZE_1K,
+ DR_CHUNK_SIZE_2K,
+ DR_CHUNK_SIZE_4K,
+ DR_CHUNK_SIZE_8K,
+ DR_CHUNK_SIZE_16K,
+ DR_CHUNK_SIZE_32K,
+ DR_CHUNK_SIZE_64K,
+ DR_CHUNK_SIZE_128K,
+ DR_CHUNK_SIZE_256K,
+ DR_CHUNK_SIZE_512K,
+ DR_CHUNK_SIZE_1024K,
+ DR_CHUNK_SIZE_2048K,
+ DR_CHUNK_SIZE_MAX,
+};
+
+enum mlx5dr_icm_type {
+ DR_ICM_TYPE_STE,
+ DR_ICM_TYPE_MODIFY_ACTION,
+};
+
+static inline enum mlx5dr_icm_chunk_size
+mlx5dr_icm_next_higher_chunk(enum mlx5dr_icm_chunk_size chunk)
+{
+ chunk += 2;
+ if (chunk < DR_CHUNK_SIZE_MAX)
+ return chunk;
+
+ return DR_CHUNK_SIZE_MAX;
+}
+
+enum {
+ DR_STE_SIZE = 64,
+ DR_STE_SIZE_CTRL = 32,
+ DR_STE_SIZE_TAG = 16,
+ DR_STE_SIZE_MASK = 16,
+};
+
+enum {
+ DR_STE_SIZE_REDUCED = DR_STE_SIZE - DR_STE_SIZE_MASK,
+};
+
+enum {
+ DR_MODIFY_ACTION_SIZE = 8,
+};
+
+enum mlx5dr_matcher_criteria {
+ DR_MATCHER_CRITERIA_EMPTY = 0,
+ DR_MATCHER_CRITERIA_OUTER = 1 << 0,
+ DR_MATCHER_CRITERIA_MISC = 1 << 1,
+ DR_MATCHER_CRITERIA_INNER = 1 << 2,
+ DR_MATCHER_CRITERIA_MISC2 = 1 << 3,
+ DR_MATCHER_CRITERIA_MISC3 = 1 << 4,
+ DR_MATCHER_CRITERIA_MAX = 1 << 5,
+};
+
+enum mlx5dr_action_type {
+ DR_ACTION_TYP_TNL_L2_TO_L2,
+ DR_ACTION_TYP_L2_TO_TNL_L2,
+ DR_ACTION_TYP_TNL_L3_TO_L2,
+ DR_ACTION_TYP_L2_TO_TNL_L3,
+ DR_ACTION_TYP_DROP,
+ DR_ACTION_TYP_QP,
+ DR_ACTION_TYP_FT,
+ DR_ACTION_TYP_CTR,
+ DR_ACTION_TYP_TAG,
+ DR_ACTION_TYP_MODIFY_HDR,
+ DR_ACTION_TYP_VPORT,
+ DR_ACTION_TYP_POP_VLAN,
+ DR_ACTION_TYP_PUSH_VLAN,
+ DR_ACTION_TYP_MAX,
+};
+
+struct mlx5dr_icm_pool;
+struct mlx5dr_icm_chunk;
+struct mlx5dr_icm_bucket;
+struct mlx5dr_ste_htbl;
+struct mlx5dr_match_param;
+struct mlx5dr_cmd_caps;
+struct mlx5dr_matcher_rx_tx;
+
+struct mlx5dr_ste {
+ u8 *hw_ste;
+ /* refcount: indicates the num of rules that using this ste */
+ refcount_t refcount;
+
+ /* attached to the miss_list head at each htbl entry */
+ struct list_head miss_list_node;
+
+ /* each rule member that uses this ste attached here */
+ struct list_head rule_list;
+
+ /* this ste is member of htbl */
+ struct mlx5dr_ste_htbl *htbl;
+
+ struct mlx5dr_ste_htbl *next_htbl;
+
+ /* this ste is part of a rule, located in ste's chain */
+ u8 ste_chain_location;
+};
+
+struct mlx5dr_ste_htbl_ctrl {
+ /* total number of valid entries belonging to this hash table. This
+ * includes the non collision and collision entries
+ */
+ unsigned int num_of_valid_entries;
+
+ /* total number of collisions entries attached to this table */
+ unsigned int num_of_collisions;
+ unsigned int increase_threshold;
+ u8 may_grow:1;
+};
+
+struct mlx5dr_ste_htbl {
+ u8 lu_type;
+ u16 byte_mask;
+ refcount_t refcount;
+ struct mlx5dr_icm_chunk *chunk;
+ struct mlx5dr_ste *ste_arr;
+ u8 *hw_ste_arr;
+
+ struct list_head *miss_list;
+
+ enum mlx5dr_icm_chunk_size chunk_size;
+ struct mlx5dr_ste *pointing_ste;
+
+ struct mlx5dr_ste_htbl_ctrl ctrl;
+};
+
+struct mlx5dr_ste_send_info {
+ struct mlx5dr_ste *ste;
+ struct list_head send_list;
+ u16 size;
+ u16 offset;
+ u8 data_cont[DR_STE_SIZE];
+ u8 *data;
+};
+
+void mlx5dr_send_fill_and_append_ste_send_info(struct mlx5dr_ste *ste, u16 size,
+ u16 offset, u8 *data,
+ struct mlx5dr_ste_send_info *ste_info,
+ struct list_head *send_list,
+ bool copy_data);
+
+struct mlx5dr_ste_build {
+ u8 inner:1;
+ u8 rx:1;
+ struct mlx5dr_cmd_caps *caps;
+ u8 lu_type;
+ u16 byte_mask;
+ u8 bit_mask[DR_STE_SIZE_MASK];
+ int (*ste_build_tag_func)(struct mlx5dr_match_param *spec,
+ struct mlx5dr_ste_build *sb,
+ u8 *hw_ste_p);
+};
+
+struct mlx5dr_ste_htbl *
+mlx5dr_ste_htbl_alloc(struct mlx5dr_icm_pool *pool,
+ enum mlx5dr_icm_chunk_size chunk_size,
+ u8 lu_type, u16 byte_mask);
+
+int mlx5dr_ste_htbl_free(struct mlx5dr_ste_htbl *htbl);
+
+static inline void mlx5dr_htbl_put(struct mlx5dr_ste_htbl *htbl)
+{
+ if (refcount_dec_and_test(&htbl->refcount))
+ mlx5dr_ste_htbl_free(htbl);
+}
+
+static inline void mlx5dr_htbl_get(struct mlx5dr_ste_htbl *htbl)
+{
+ refcount_inc(&htbl->refcount);
+}
+
+/* STE utils */
+u32 mlx5dr_ste_calc_hash_index(u8 *hw_ste_p, struct mlx5dr_ste_htbl *htbl);
+void mlx5dr_ste_init(u8 *hw_ste_p, u8 lu_type, u8 entry_type, u16 gvmi);
+void mlx5dr_ste_always_hit_htbl(struct mlx5dr_ste *ste,
+ struct mlx5dr_ste_htbl *next_htbl);
+void mlx5dr_ste_set_miss_addr(u8 *hw_ste, u64 miss_addr);
+u64 mlx5dr_ste_get_miss_addr(u8 *hw_ste);
+void mlx5dr_ste_set_hit_gvmi(u8 *hw_ste_p, u16 gvmi);
+void mlx5dr_ste_set_hit_addr(u8 *hw_ste, u64 icm_addr, u32 ht_size);
+void mlx5dr_ste_always_miss_addr(struct mlx5dr_ste *ste, u64 miss_addr);
+void mlx5dr_ste_set_bit_mask(u8 *hw_ste_p, u8 *bit_mask);
+bool mlx5dr_ste_not_used_ste(struct mlx5dr_ste *ste);
+bool mlx5dr_ste_is_last_in_rule(struct mlx5dr_matcher_rx_tx *nic_matcher,
+ u8 ste_location);
+void mlx5dr_ste_rx_set_flow_tag(u8 *hw_ste_p, u32 flow_tag);
+void mlx5dr_ste_set_counter_id(u8 *hw_ste_p, u32 ctr_id);
+void mlx5dr_ste_set_tx_encap(void *hw_ste_p, u32 reformat_id,
+ int size, bool encap_l3);
+void mlx5dr_ste_set_rx_decap(u8 *hw_ste_p);
+void mlx5dr_ste_set_rx_decap_l3(u8 *hw_ste_p, bool vlan);
+void mlx5dr_ste_set_rx_pop_vlan(u8 *hw_ste_p);
+void mlx5dr_ste_set_tx_push_vlan(u8 *hw_ste_p, u32 vlan_tpid_pcp_dei_vid,
+ bool go_back);
+void mlx5dr_ste_set_entry_type(u8 *hw_ste_p, u8 entry_type);
+u8 mlx5dr_ste_get_entry_type(u8 *hw_ste_p);
+void mlx5dr_ste_set_rewrite_actions(u8 *hw_ste_p, u16 num_of_actions,
+ u32 re_write_index);
+void mlx5dr_ste_set_go_back_bit(u8 *hw_ste_p);
+u64 mlx5dr_ste_get_icm_addr(struct mlx5dr_ste *ste);
+u64 mlx5dr_ste_get_mr_addr(struct mlx5dr_ste *ste);
+struct list_head *mlx5dr_ste_get_miss_list(struct mlx5dr_ste *ste);
+
+void mlx5dr_ste_free(struct mlx5dr_ste *ste,
+ struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher);
+static inline void mlx5dr_ste_put(struct mlx5dr_ste *ste,
+ struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher)
+{
+ if (refcount_dec_and_test(&ste->refcount))
+ mlx5dr_ste_free(ste, matcher, nic_matcher);
+}
+
+/* initial as 0, increased only when ste appears in a new rule */
+static inline void mlx5dr_ste_get(struct mlx5dr_ste *ste)
+{
+ refcount_inc(&ste->refcount);
+}
+
+void mlx5dr_ste_set_hit_addr_by_next_htbl(u8 *hw_ste,
+ struct mlx5dr_ste_htbl *next_htbl);
+bool mlx5dr_ste_equal_tag(void *src, void *dst);
+int mlx5dr_ste_create_next_htbl(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_ste *ste,
+ u8 *cur_hw_ste,
+ enum mlx5dr_icm_chunk_size log_table_size);
+
+/* STE build functions */
+int mlx5dr_ste_build_pre_check(struct mlx5dr_domain *dmn,
+ u8 match_criteria,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_match_param *value);
+int mlx5dr_ste_build_ste_arr(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_match_param *value,
+ u8 *ste_arr);
+int mlx5dr_ste_build_eth_l2_src_des(struct mlx5dr_ste_build *builder,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l3_ipv4_5_tuple(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l3_ipv4_misc(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l3_ipv6_dst(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l3_ipv6_src(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l2_src(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l2_dst(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l2_tnl(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_ipv6_l3_l4(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_eth_l4_misc(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_gre(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_mpls(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_flex_parser_0(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+int mlx5dr_ste_build_flex_parser_1(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ bool inner, bool rx);
+void mlx5dr_ste_build_flex_parser_tnl(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_general_purpose(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_register_0(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+void mlx5dr_ste_build_register_1(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ bool inner, bool rx);
+int mlx5dr_ste_build_src_gvmi_qpn(struct mlx5dr_ste_build *sb,
+ struct mlx5dr_match_param *mask,
+ struct mlx5dr_cmd_caps *caps,
+ bool inner, bool rx);
+void mlx5dr_ste_build_empty_always_hit(struct mlx5dr_ste_build *sb, bool rx);
+
+/* Actions utils */
+int mlx5dr_actions_build_ste_arr(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ struct mlx5dr_action *actions[],
+ u32 num_actions,
+ u8 *ste_arr,
+ u32 *new_hw_ste_arr_sz);
+
+struct mlx5dr_match_spec {
+ u32 smac_47_16; /* Source MAC address of incoming packet */
+ /* Incoming packet Ethertype - this is the Ethertype
+ * following the last VLAN tag of the packet
+ */
+ u32 ethertype:16;
+ u32 smac_15_0:16; /* Source MAC address of incoming packet */
+ u32 dmac_47_16; /* Destination MAC address of incoming packet */
+ /* VLAN ID of first VLAN tag in the incoming packet.
+ * Valid only when cvlan_tag==1 or svlan_tag==1
+ */
+ u32 first_vid:12;
+ /* CFI bit of first VLAN tag in the incoming packet.
+ * Valid only when cvlan_tag==1 or svlan_tag==1
+ */
+ u32 first_cfi:1;
+ /* Priority of first VLAN tag in the incoming packet.
+ * Valid only when cvlan_tag==1 or svlan_tag==1
+ */
+ u32 first_prio:3;
+ u32 dmac_15_0:16; /* Destination MAC address of incoming packet */
+ /* TCP flags. ;Bit 0: FIN;Bit 1: SYN;Bit 2: RST;Bit 3: PSH;Bit 4: ACK;
+ * Bit 5: URG;Bit 6: ECE;Bit 7: CWR;Bit 8: NS
+ */
+ u32 tcp_flags:9;
+ u32 ip_version:4; /* IP version */
+ u32 frag:1; /* Packet is an IP fragment */
+ /* The first vlan in the packet is s-vlan (0x8a88).
+ * cvlan_tag and svlan_tag cannot be set together
+ */
+ u32 svlan_tag:1;
+ /* The first vlan in the packet is c-vlan (0x8100).
+ * cvlan_tag and svlan_tag cannot be set together
+ */
+ u32 cvlan_tag:1;
+ /* Explicit Congestion Notification derived from
+ * Traffic Class/TOS field of IPv6/v4
+ */
+ u32 ip_ecn:2;
+ /* Differentiated Services Code Point derived from
+ * Traffic Class/TOS field of IPv6/v4
+ */
+ u32 ip_dscp:6;
+ u32 ip_protocol:8; /* IP protocol */
+ /* TCP destination port.
+ * tcp and udp sport/dport are mutually exclusive
+ */
+ u32 tcp_dport:16;
+ /* TCP source port.;tcp and udp sport/dport are mutually exclusive */
+ u32 tcp_sport:16;
+ u32 ttl_hoplimit:8;
+ u32 reserved:24;
+ /* UDP destination port.;tcp and udp sport/dport are mutually exclusive */
+ u32 udp_dport:16;
+ /* UDP source port.;tcp and udp sport/dport are mutually exclusive */
+ u32 udp_sport:16;
+ /* IPv6 source address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 src_ip_127_96;
+ /* IPv6 source address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 src_ip_95_64;
+ /* IPv6 source address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 src_ip_63_32;
+ /* IPv6 source address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 src_ip_31_0;
+ /* IPv6 destination address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 dst_ip_127_96;
+ /* IPv6 destination address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 dst_ip_95_64;
+ /* IPv6 destination address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 dst_ip_63_32;
+ /* IPv6 destination address of incoming packets
+ * For IPv4 address use bits 31:0 (rest of the bits are reserved)
+ * This field should be qualified by an appropriate ethertype
+ */
+ u32 dst_ip_31_0;
+};
+
+struct mlx5dr_match_misc {
+ u32 source_sqn:24; /* Source SQN */
+ u32 source_vhca_port:4;
+ /* used with GRE, sequence number exist when gre_s_present == 1 */
+ u32 gre_s_present:1;
+ /* used with GRE, key exist when gre_k_present == 1 */
+ u32 gre_k_present:1;
+ u32 reserved_auto1:1;
+ /* used with GRE, checksum exist when gre_c_present == 1 */
+ u32 gre_c_present:1;
+ /* Source port.;0xffff determines wire port */
+ u32 source_port:16;
+ u32 reserved_auto2:16;
+ /* VLAN ID of first VLAN tag the inner header of the incoming packet.
+ * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
+ */
+ u32 inner_second_vid:12;
+ /* CFI bit of first VLAN tag in the inner header of the incoming packet.
+ * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
+ */
+ u32 inner_second_cfi:1;
+ /* Priority of second VLAN tag in the inner header of the incoming packet.
+ * Valid only when inner_second_cvlan_tag ==1 or inner_second_svlan_tag ==1
+ */
+ u32 inner_second_prio:3;
+ /* VLAN ID of first VLAN tag the outer header of the incoming packet.
+ * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
+ */
+ u32 outer_second_vid:12;
+ /* CFI bit of first VLAN tag in the outer header of the incoming packet.
+ * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
+ */
+ u32 outer_second_cfi:1;
+ /* Priority of second VLAN tag in the outer header of the incoming packet.
+ * Valid only when outer_second_cvlan_tag ==1 or outer_second_svlan_tag ==1
+ */
+ u32 outer_second_prio:3;
+ u32 gre_protocol:16; /* GRE Protocol (outer) */
+ u32 reserved_auto3:12;
+ /* The second vlan in the inner header of the packet is s-vlan (0x8a88).
+ * inner_second_cvlan_tag and inner_second_svlan_tag cannot be set together
+ */
+ u32 inner_second_svlan_tag:1;
+ /* The second vlan in the outer header of the packet is s-vlan (0x8a88).
+ * outer_second_cvlan_tag and outer_second_svlan_tag cannot be set together
+ */
+ u32 outer_second_svlan_tag:1;
+ /* The second vlan in the inner header of the packet is c-vlan (0x8100).
+ * inner_second_cvlan_tag and inner_second_svlan_tag cannot be set together
+ */
+ u32 inner_second_cvlan_tag:1;
+ /* The second vlan in the outer header of the packet is c-vlan (0x8100).
+ * outer_second_cvlan_tag and outer_second_svlan_tag cannot be set together
+ */
+ u32 outer_second_cvlan_tag:1;
+ u32 gre_key_l:8; /* GRE Key [7:0] (outer) */
+ u32 gre_key_h:24; /* GRE Key[31:8] (outer) */
+ u32 reserved_auto4:8;
+ u32 vxlan_vni:24; /* VXLAN VNI (outer) */
+ u32 geneve_oam:1; /* GENEVE OAM field (outer) */
+ u32 reserved_auto5:7;
+ u32 geneve_vni:24; /* GENEVE VNI field (outer) */
+ u32 outer_ipv6_flow_label:20; /* Flow label of incoming IPv6 packet (outer) */
+ u32 reserved_auto6:12;
+ u32 inner_ipv6_flow_label:20; /* Flow label of incoming IPv6 packet (inner) */
+ u32 reserved_auto7:12;
+ u32 geneve_protocol_type:16; /* GENEVE protocol type (outer) */
+ u32 geneve_opt_len:6; /* GENEVE OptLen (outer) */
+ u32 reserved_auto8:10;
+ u32 bth_dst_qp:24; /* Destination QP in BTH header */
+ u32 reserved_auto9:8;
+ u8 reserved_auto10[20];
+};
+
+struct mlx5dr_match_misc2 {
+ u32 outer_first_mpls_ttl:8; /* First MPLS TTL (outer) */
+ u32 outer_first_mpls_s_bos:1; /* First MPLS S_BOS (outer) */
+ u32 outer_first_mpls_exp:3; /* First MPLS EXP (outer) */
+ u32 outer_first_mpls_label:20; /* First MPLS LABEL (outer) */
+ u32 inner_first_mpls_ttl:8; /* First MPLS TTL (inner) */
+ u32 inner_first_mpls_s_bos:1; /* First MPLS S_BOS (inner) */
+ u32 inner_first_mpls_exp:3; /* First MPLS EXP (inner) */
+ u32 inner_first_mpls_label:20; /* First MPLS LABEL (inner) */
+ u32 outer_first_mpls_over_gre_ttl:8; /* last MPLS TTL (outer) */
+ u32 outer_first_mpls_over_gre_s_bos:1; /* last MPLS S_BOS (outer) */
+ u32 outer_first_mpls_over_gre_exp:3; /* last MPLS EXP (outer) */
+ u32 outer_first_mpls_over_gre_label:20; /* last MPLS LABEL (outer) */
+ u32 outer_first_mpls_over_udp_ttl:8; /* last MPLS TTL (outer) */
+ u32 outer_first_mpls_over_udp_s_bos:1; /* last MPLS S_BOS (outer) */
+ u32 outer_first_mpls_over_udp_exp:3; /* last MPLS EXP (outer) */
+ u32 outer_first_mpls_over_udp_label:20; /* last MPLS LABEL (outer) */
+ u32 metadata_reg_c_7; /* metadata_reg_c_7 */
+ u32 metadata_reg_c_6; /* metadata_reg_c_6 */
+ u32 metadata_reg_c_5; /* metadata_reg_c_5 */
+ u32 metadata_reg_c_4; /* metadata_reg_c_4 */
+ u32 metadata_reg_c_3; /* metadata_reg_c_3 */
+ u32 metadata_reg_c_2; /* metadata_reg_c_2 */
+ u32 metadata_reg_c_1; /* metadata_reg_c_1 */
+ u32 metadata_reg_c_0; /* metadata_reg_c_0 */
+ u32 metadata_reg_a; /* metadata_reg_a */
+ u32 metadata_reg_b; /* metadata_reg_b */
+ u8 reserved_auto2[8];
+};
+
+struct mlx5dr_match_misc3 {
+ u32 inner_tcp_seq_num;
+ u32 outer_tcp_seq_num;
+ u32 inner_tcp_ack_num;
+ u32 outer_tcp_ack_num;
+ u32 outer_vxlan_gpe_vni:24;
+ u32 reserved_auto1:8;
+ u32 reserved_auto2:16;
+ u32 outer_vxlan_gpe_flags:8;
+ u32 outer_vxlan_gpe_next_protocol:8;
+ u32 icmpv4_header_data;
+ u32 icmpv6_header_data;
+ u32 icmpv6_code:8;
+ u32 icmpv6_type:8;
+ u32 icmpv4_code:8;
+ u32 icmpv4_type:8;
+ u8 reserved_auto3[0x1c];
+};
+
+struct mlx5dr_match_param {
+ struct mlx5dr_match_spec outer;
+ struct mlx5dr_match_misc misc;
+ struct mlx5dr_match_spec inner;
+ struct mlx5dr_match_misc2 misc2;
+ struct mlx5dr_match_misc3 misc3;
+};
+
+#define DR_MASK_IS_FLEX_PARSER_ICMPV4_SET(_misc3) ((_misc3)->icmpv4_type || \
+ (_misc3)->icmpv4_code || \
+ (_misc3)->icmpv4_header_data)
+
+struct mlx5dr_esw_caps {
+ u64 drop_icm_address_rx;
+ u64 drop_icm_address_tx;
+ u64 uplink_icm_address_rx;
+ u64 uplink_icm_address_tx;
+ bool sw_owner;
+};
+
+struct mlx5dr_cmd_vport_cap {
+ u16 vport_gvmi;
+ u16 vhca_gvmi;
+ u64 icm_address_rx;
+ u64 icm_address_tx;
+ u32 num;
+};
+
+struct mlx5dr_cmd_caps {
+ u16 gvmi;
+ u64 nic_rx_drop_address;
+ u64 nic_tx_drop_address;
+ u64 nic_tx_allow_address;
+ u64 esw_rx_drop_address;
+ u64 esw_tx_drop_address;
+ u32 log_icm_size;
+ u64 hdr_modify_icm_addr;
+ u32 flex_protocols;
+ u8 flex_parser_id_icmp_dw0;
+ u8 flex_parser_id_icmp_dw1;
+ u8 flex_parser_id_icmpv6_dw0;
+ u8 flex_parser_id_icmpv6_dw1;
+ u8 max_ft_level;
+ u16 roce_min_src_udp;
+ u8 num_esw_ports;
+ bool eswitch_manager;
+ bool rx_sw_owner;
+ bool tx_sw_owner;
+ bool fdb_sw_owner;
+ u32 num_vports;
+ struct mlx5dr_esw_caps esw_caps;
+ struct mlx5dr_cmd_vport_cap *vports_caps;
+ bool prio_tag_required;
+};
+
+struct mlx5dr_domain_rx_tx {
+ u64 drop_icm_addr;
+ u64 default_icm_addr;
+ enum mlx5dr_ste_entry_type ste_type;
+};
+
+struct mlx5dr_domain_info {
+ bool supp_sw_steering;
+ u32 max_inline_size;
+ u32 max_send_wr;
+ u32 max_log_sw_icm_sz;
+ u32 max_log_action_icm_sz;
+ struct mlx5dr_domain_rx_tx rx;
+ struct mlx5dr_domain_rx_tx tx;
+ struct mlx5dr_cmd_caps caps;
+};
+
+struct mlx5dr_domain_cache {
+ struct mlx5dr_fw_recalc_cs_ft **recalc_cs_ft;
+};
+
+struct mlx5dr_domain {
+ struct mlx5dr_domain *peer_dmn;
+ struct mlx5_core_dev *mdev;
+ u32 pdn;
+ struct mlx5_uars_page *uar;
+ enum mlx5dr_domain_type type;
+ refcount_t refcount;
+ struct mutex mutex; /* protect domain */
+ struct mlx5dr_icm_pool *ste_icm_pool;
+ struct mlx5dr_icm_pool *action_icm_pool;
+ struct mlx5dr_send_ring *send_ring;
+ struct mlx5dr_domain_info info;
+ struct mlx5dr_domain_cache cache;
+};
+
+struct mlx5dr_table_rx_tx {
+ struct mlx5dr_ste_htbl *s_anchor;
+ struct mlx5dr_domain_rx_tx *nic_dmn;
+ u64 default_icm_addr;
+};
+
+struct mlx5dr_table {
+ struct mlx5dr_domain *dmn;
+ struct mlx5dr_table_rx_tx rx;
+ struct mlx5dr_table_rx_tx tx;
+ u32 level;
+ u32 table_type;
+ u32 table_id;
+ struct list_head matcher_list;
+ struct mlx5dr_action *miss_action;
+ refcount_t refcount;
+};
+
+struct mlx5dr_matcher_rx_tx {
+ struct mlx5dr_ste_htbl *s_htbl;
+ struct mlx5dr_ste_htbl *e_anchor;
+ struct mlx5dr_ste_build *ste_builder;
+ struct mlx5dr_ste_build ste_builder4[DR_RULE_MAX_STES];
+ struct mlx5dr_ste_build ste_builder6[DR_RULE_MAX_STES];
+ u8 num_of_builders;
+ u8 num_of_builders4;
+ u8 num_of_builders6;
+ u64 default_icm_addr;
+ struct mlx5dr_table_rx_tx *nic_tbl;
+};
+
+struct mlx5dr_matcher {
+ struct mlx5dr_table *tbl;
+ struct mlx5dr_matcher_rx_tx rx;
+ struct mlx5dr_matcher_rx_tx tx;
+ struct list_head matcher_list;
+ u16 prio;
+ struct mlx5dr_match_param mask;
+ u8 match_criteria;
+ refcount_t refcount;
+ struct mlx5dv_flow_matcher *dv_matcher;
+};
+
+struct mlx5dr_rule_member {
+ struct mlx5dr_ste *ste;
+ /* attached to mlx5dr_rule via this */
+ struct list_head list;
+ /* attached to mlx5dr_ste via this */
+ struct list_head use_ste_list;
+};
+
+struct mlx5dr_action {
+ enum mlx5dr_action_type action_type;
+ refcount_t refcount;
+ union {
+ struct {
+ struct mlx5dr_domain *dmn;
+ struct mlx5dr_icm_chunk *chunk;
+ u8 *data;
+ u32 data_size;
+ u16 num_of_actions;
+ u32 index;
+ u8 allow_rx:1;
+ u8 allow_tx:1;
+ u8 modify_ttl:1;
+ } rewrite;
+ struct {
+ struct mlx5dr_domain *dmn;
+ u32 reformat_id;
+ u32 reformat_size;
+ } reformat;
+ struct {
+ u8 is_fw_tbl:1;
+ union {
+ struct mlx5dr_table *tbl;
+ struct {
+ struct mlx5_flow_table *ft;
+ u64 rx_icm_addr;
+ u64 tx_icm_addr;
+ struct mlx5_core_dev *mdev;
+ } fw_tbl;
+ };
+ } dest_tbl;
+ struct {
+ u32 ctr_id;
+ u32 offeset;
+ } ctr;
+ struct {
+ struct mlx5dr_domain *dmn;
+ struct mlx5dr_cmd_vport_cap *caps;
+ u32 num;
+ } vport;
+ struct {
+ u32 vlan_hdr; /* tpid_pcp_dei_vid */
+ } push_vlan;
+ u32 flow_tag;
+ };
+};
+
+enum mlx5dr_connect_type {
+ CONNECT_HIT = 1,
+ CONNECT_MISS = 2,
+};
+
+struct mlx5dr_htbl_connect_info {
+ enum mlx5dr_connect_type type;
+ union {
+ struct mlx5dr_ste_htbl *hit_next_htbl;
+ u64 miss_icm_addr;
+ };
+};
+
+struct mlx5dr_rule_rx_tx {
+ struct list_head rule_members_list;
+ struct mlx5dr_matcher_rx_tx *nic_matcher;
+};
+
+struct mlx5dr_rule {
+ struct mlx5dr_matcher *matcher;
+ struct mlx5dr_rule_rx_tx rx;
+ struct mlx5dr_rule_rx_tx tx;
+ struct list_head rule_actions_list;
+};
+
+void mlx5dr_rule_update_rule_member(struct mlx5dr_ste *new_ste,
+ struct mlx5dr_ste *ste);
+
+struct mlx5dr_icm_chunk {
+ struct mlx5dr_icm_bucket *bucket;
+ struct list_head chunk_list;
+ u32 rkey;
+ u32 num_of_entries;
+ u32 byte_size;
+ u64 icm_addr;
+ u64 mr_addr;
+
+ /* Memory optimisation */
+ struct mlx5dr_ste *ste_arr;
+ u8 *hw_ste_arr;
+ struct list_head *miss_list;
+};
+
+static inline int
+mlx5dr_matcher_supp_flex_parser_icmp_v4(struct mlx5dr_cmd_caps *caps)
+{
+ return caps->flex_protocols & MLX5_FLEX_PARSER_ICMP_V4_ENABLED;
+}
+
+static inline int
+mlx5dr_matcher_supp_flex_parser_icmp_v6(struct mlx5dr_cmd_caps *caps)
+{
+ return caps->flex_protocols & MLX5_FLEX_PARSER_ICMP_V6_ENABLED;
+}
+
+int mlx5dr_matcher_select_builders(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_matcher_rx_tx *nic_matcher,
+ bool ipv6);
+
+static inline u32
+mlx5dr_icm_pool_chunk_size_to_entries(enum mlx5dr_icm_chunk_size chunk_size)
+{
+ return 1 << chunk_size;
+}
+
+static inline int
+mlx5dr_icm_pool_chunk_size_to_byte(enum mlx5dr_icm_chunk_size chunk_size,
+ enum mlx5dr_icm_type icm_type)
+{
+ int num_of_entries;
+ int entry_size;
+
+ if (icm_type == DR_ICM_TYPE_STE)
+ entry_size = DR_STE_SIZE;
+ else
+ entry_size = DR_MODIFY_ACTION_SIZE;
+
+ num_of_entries = mlx5dr_icm_pool_chunk_size_to_entries(chunk_size);
+
+ return entry_size * num_of_entries;
+}
+
+static inline struct mlx5dr_cmd_vport_cap *
+mlx5dr_get_vport_cap(struct mlx5dr_cmd_caps *caps, u32 vport)
+{
+ if (!caps->vports_caps ||
+ (vport >= caps->num_vports && vport != WIRE_PORT))
+ return NULL;
+
+ if (vport == WIRE_PORT)
+ vport = caps->num_vports;
+
+ return &caps->vports_caps[vport];
+}
+
+struct mlx5dr_cmd_query_flow_table_details {
+ u8 status;
+ u8 level;
+ u64 sw_owner_icm_root_1;
+ u64 sw_owner_icm_root_0;
+};
+
+/* internal API functions */
+int mlx5dr_cmd_query_device(struct mlx5_core_dev *mdev,
+ struct mlx5dr_cmd_caps *caps);
+int mlx5dr_cmd_query_esw_vport_context(struct mlx5_core_dev *mdev,
+ bool other_vport, u16 vport_number,
+ u64 *icm_address_rx,
+ u64 *icm_address_tx);
+int mlx5dr_cmd_query_gvmi(struct mlx5_core_dev *mdev,
+ bool other_vport, u16 vport_number, u16 *gvmi);
+int mlx5dr_cmd_query_esw_caps(struct mlx5_core_dev *mdev,
+ struct mlx5dr_esw_caps *caps);
+int mlx5dr_cmd_sync_steering(struct mlx5_core_dev *mdev);
+int mlx5dr_cmd_set_fte_modify_and_vport(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id,
+ u32 group_id,
+ u32 modify_header_id,
+ u32 vport_id);
+int mlx5dr_cmd_del_flow_table_entry(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id);
+int mlx5dr_cmd_alloc_modify_header(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u8 num_of_actions,
+ u64 *actions,
+ u32 *modify_header_id);
+int mlx5dr_cmd_dealloc_modify_header(struct mlx5_core_dev *mdev,
+ u32 modify_header_id);
+int mlx5dr_cmd_create_empty_flow_group(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id,
+ u32 *group_id);
+int mlx5dr_cmd_destroy_flow_group(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u32 table_id,
+ u32 group_id);
+int mlx5dr_cmd_create_flow_table(struct mlx5_core_dev *mdev,
+ u32 table_type,
+ u64 icm_addr_rx,
+ u64 icm_addr_tx,
+ u8 level,
+ bool sw_owner,
+ bool term_tbl,
+ u64 *fdb_rx_icm_addr,
+ u32 *table_id);
+int mlx5dr_cmd_destroy_flow_table(struct mlx5_core_dev *mdev,
+ u32 table_id,
+ u32 table_type);
+int mlx5dr_cmd_query_flow_table(struct mlx5_core_dev *dev,
+ enum fs_flow_table_type type,
+ u32 table_id,
+ struct mlx5dr_cmd_query_flow_table_details *output);
+int mlx5dr_cmd_create_reformat_ctx(struct mlx5_core_dev *mdev,
+ enum mlx5_reformat_ctx_type rt,
+ size_t reformat_size,
+ void *reformat_data,
+ u32 *reformat_id);
+void mlx5dr_cmd_destroy_reformat_ctx(struct mlx5_core_dev *mdev,
+ u32 reformat_id);
+
+struct mlx5dr_cmd_gid_attr {
+ u8 gid[16];
+ u8 mac[6];
+ u32 roce_ver;
+};
+
+struct mlx5dr_cmd_qp_create_attr {
+ u32 page_id;
+ u32 pdn;
+ u32 cqn;
+ u32 pm_state;
+ u32 service_type;
+ u32 buff_umem_id;
+ u32 db_umem_id;
+ u32 sq_wqe_cnt;
+ u32 rq_wqe_cnt;
+ u32 rq_wqe_shift;
+};
+
+int mlx5dr_cmd_query_gid(struct mlx5_core_dev *mdev, u8 vhca_port_num,
+ u16 index, struct mlx5dr_cmd_gid_attr *attr);
+
+struct mlx5dr_icm_pool *mlx5dr_icm_pool_create(struct mlx5dr_domain *dmn,
+ enum mlx5dr_icm_type icm_type);
+void mlx5dr_icm_pool_destroy(struct mlx5dr_icm_pool *pool);
+
+struct mlx5dr_icm_chunk *
+mlx5dr_icm_alloc_chunk(struct mlx5dr_icm_pool *pool,
+ enum mlx5dr_icm_chunk_size chunk_size);
+void mlx5dr_icm_free_chunk(struct mlx5dr_icm_chunk *chunk);
+bool mlx5dr_ste_is_not_valid_entry(u8 *p_hw_ste);
+int mlx5dr_ste_htbl_init_and_postsend(struct mlx5dr_domain *dmn,
+ struct mlx5dr_domain_rx_tx *nic_dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ struct mlx5dr_htbl_connect_info *connect_info,
+ bool update_hw_ste);
+void mlx5dr_ste_set_formatted_ste(u16 gvmi,
+ struct mlx5dr_domain_rx_tx *nic_dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 *formatted_ste,
+ struct mlx5dr_htbl_connect_info *connect_info);
+void mlx5dr_ste_copy_param(u8 match_criteria,
+ struct mlx5dr_match_param *set_param,
+ struct mlx5dr_match_parameters *mask);
+
+void mlx5dr_crc32_init_table(void);
+u32 mlx5dr_crc32_slice8_calc(const void *input_data, size_t length);
+
+struct mlx5dr_qp {
+ struct mlx5_core_dev *mdev;
+ struct mlx5_wq_qp wq;
+ struct mlx5_uars_page *uar;
+ struct mlx5_wq_ctrl wq_ctrl;
+ struct mlx5_core_qp mqp;
+ struct {
+ unsigned int pc;
+ unsigned int cc;
+ unsigned int size;
+ unsigned int *wqe_head;
+ unsigned int wqe_cnt;
+ } sq;
+ struct {
+ unsigned int pc;
+ unsigned int cc;
+ unsigned int size;
+ unsigned int wqe_cnt;
+ } rq;
+ int max_inline_data;
+};
+
+struct mlx5dr_cq {
+ struct mlx5_core_dev *mdev;
+ struct mlx5_cqwq wq;
+ struct mlx5_wq_ctrl wq_ctrl;
+ struct mlx5_core_cq mcq;
+ struct mlx5dr_qp *qp;
+};
+
+struct mlx5dr_mr {
+ struct mlx5_core_dev *mdev;
+ struct mlx5_core_mkey mkey;
+ dma_addr_t dma_addr;
+ void *addr;
+ size_t size;
+};
+
+#define MAX_SEND_CQE 64
+#define MIN_READ_SYNC 64
+
+struct mlx5dr_send_ring {
+ struct mlx5dr_cq *cq;
+ struct mlx5dr_qp *qp;
+ struct mlx5dr_mr *mr;
+ /* How much wqes are waiting for completion */
+ u32 pending_wqe;
+ /* Signal request per this trash hold value */
+ u16 signal_th;
+ /* Each post_send_size less than max_post_send_size */
+ u32 max_post_send_size;
+ /* manage the send queue */
+ u32 tx_head;
+ void *buf;
+ u32 buf_size;
+ struct ib_wc wc[MAX_SEND_CQE];
+ u8 sync_buff[MIN_READ_SYNC];
+ struct mlx5dr_mr *sync_mr;
+};
+
+int mlx5dr_send_ring_alloc(struct mlx5dr_domain *dmn);
+void mlx5dr_send_ring_free(struct mlx5dr_domain *dmn,
+ struct mlx5dr_send_ring *send_ring);
+int mlx5dr_send_ring_force_drain(struct mlx5dr_domain *dmn);
+int mlx5dr_send_postsend_ste(struct mlx5dr_domain *dmn,
+ struct mlx5dr_ste *ste,
+ u8 *data,
+ u16 size,
+ u16 offset);
+int mlx5dr_send_postsend_htbl(struct mlx5dr_domain *dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 *formatted_ste, u8 *mask);
+int mlx5dr_send_postsend_formatted_htbl(struct mlx5dr_domain *dmn,
+ struct mlx5dr_ste_htbl *htbl,
+ u8 *ste_init_data,
+ bool update_hw_ste);
+int mlx5dr_send_postsend_action(struct mlx5dr_domain *dmn,
+ struct mlx5dr_action *action);
+
+struct mlx5dr_fw_recalc_cs_ft {
+ u64 rx_icm_addr;
+ u32 table_id;
+ u32 group_id;
+ u32 modify_hdr_id;
+};
+
+struct mlx5dr_fw_recalc_cs_ft *
+mlx5dr_fw_create_recalc_cs_ft(struct mlx5dr_domain *dmn, u32 vport_num);
+void mlx5dr_fw_destroy_recalc_cs_ft(struct mlx5dr_domain *dmn,
+ struct mlx5dr_fw_recalc_cs_ft *recalc_cs_ft);
+int mlx5dr_domain_cache_get_recalc_cs_ft_addr(struct mlx5dr_domain *dmn,
+ u32 vport_num,
+ u64 *rx_icm_addr);
+#endif /* _DR_TYPES_H_ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c
new file mode 100644
index 0000000..3d587d0
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.c
@@ -0,0 +1,600 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019 Mellanox Technologies */
+
+#include "mlx5_core.h"
+#include "fs_core.h"
+#include "fs_cmd.h"
+#include "mlx5dr.h"
+#include "fs_dr.h"
+
+static bool mlx5_dr_is_fw_table(u32 flags)
+{
+ if (flags & MLX5_FLOW_TABLE_TERMINATION)
+ return true;
+
+ return false;
+}
+
+static int mlx5_cmd_dr_update_root_ft(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ u32 underlay_qpn,
+ bool disconnect)
+{
+ return mlx5_fs_cmd_get_fw_cmds()->update_root_ft(ns, ft, underlay_qpn,
+ disconnect);
+}
+
+static int set_miss_action(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ struct mlx5_flow_table *next_ft)
+{
+ struct mlx5dr_action *old_miss_action;
+ struct mlx5dr_action *action = NULL;
+ struct mlx5dr_table *next_tbl;
+ int err;
+
+ next_tbl = next_ft ? next_ft->fs_dr_table.dr_table : NULL;
+ if (next_tbl) {
+ action = mlx5dr_action_create_dest_table(next_tbl);
+ if (!action)
+ return -EINVAL;
+ }
+ old_miss_action = ft->fs_dr_table.miss_action;
+ err = mlx5dr_table_set_miss_action(ft->fs_dr_table.dr_table, action);
+ if (err && action) {
+ err = mlx5dr_action_destroy(action);
+ if (err) {
+ action = NULL;
+ mlx5_core_err(ns->dev, "Failed to destroy action (%d)\n",
+ err);
+ }
+ }
+ ft->fs_dr_table.miss_action = action;
+ if (old_miss_action) {
+ err = mlx5dr_action_destroy(old_miss_action);
+ if (err)
+ mlx5_core_err(ns->dev, "Failed to destroy action (%d)\n",
+ err);
+ }
+
+ return err;
+}
+
+static int mlx5_cmd_dr_create_flow_table(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ unsigned int log_size,
+ struct mlx5_flow_table *next_ft)
+{
+ struct mlx5dr_table *tbl;
+ int err;
+
+ if (mlx5_dr_is_fw_table(ft->flags))
+ return mlx5_fs_cmd_get_fw_cmds()->create_flow_table(ns, ft,
+ log_size,
+ next_ft);
+
+ tbl = mlx5dr_table_create(ns->fs_dr_domain.dr_domain,
+ ft->level);
+ if (!tbl) {
+ mlx5_core_err(ns->dev, "Failed creating dr flow_table\n");
+ return -EINVAL;
+ }
+
+ ft->fs_dr_table.dr_table = tbl;
+ ft->id = mlx5dr_table_get_id(tbl);
+
+ if (next_ft) {
+ err = set_miss_action(ns, ft, next_ft);
+ if (err) {
+ mlx5dr_table_destroy(tbl);
+ ft->fs_dr_table.dr_table = NULL;
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static int mlx5_cmd_dr_destroy_flow_table(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft)
+{
+ struct mlx5dr_action *action = ft->fs_dr_table.miss_action;
+ int err;
+
+ if (mlx5_dr_is_fw_table(ft->flags))
+ return mlx5_fs_cmd_get_fw_cmds()->destroy_flow_table(ns, ft);
+
+ err = mlx5dr_table_destroy(ft->fs_dr_table.dr_table);
+ if (err) {
+ mlx5_core_err(ns->dev, "Failed to destroy flow_table (%d)\n",
+ err);
+ return err;
+ }
+ if (action) {
+ err = mlx5dr_action_destroy(action);
+ if (err) {
+ mlx5_core_err(ns->dev, "Failed to destroy action(%d)\n",
+ err);
+ return err;
+ }
+ }
+
+ return err;
+}
+
+static int mlx5_cmd_dr_modify_flow_table(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ struct mlx5_flow_table *next_ft)
+{
+ return set_miss_action(ns, ft, next_ft);
+}
+
+static int mlx5_cmd_dr_create_flow_group(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ u32 *in,
+ struct mlx5_flow_group *fg)
+{
+ struct mlx5dr_matcher *matcher;
+ u16 priority = MLX5_GET(create_flow_group_in, in,
+ start_flow_index);
+ u8 match_criteria_enable = MLX5_GET(create_flow_group_in,
+ in,
+ match_criteria_enable);
+ struct mlx5dr_match_parameters mask;
+
+ if (mlx5_dr_is_fw_table(ft->flags))
+ return mlx5_fs_cmd_get_fw_cmds()->create_flow_group(ns, ft, in,
+ fg);
+
+ mask.match_buf = MLX5_ADDR_OF(create_flow_group_in,
+ in, match_criteria);
+ mask.match_sz = sizeof(fg->mask.match_criteria);
+
+ matcher = mlx5dr_matcher_create(ft->fs_dr_table.dr_table,
+ priority,
+ match_criteria_enable,
+ &mask);
+ if (!matcher) {
+ mlx5_core_err(ns->dev, "Failed creating matcher\n");
+ return -EINVAL;
+ }
+
+ fg->fs_dr_matcher.dr_matcher = matcher;
+ return 0;
+}
+
+static int mlx5_cmd_dr_destroy_flow_group(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ struct mlx5_flow_group *fg)
+{
+ if (mlx5_dr_is_fw_table(ft->flags))
+ return mlx5_fs_cmd_get_fw_cmds()->destroy_flow_group(ns, ft, fg);
+
+ return mlx5dr_matcher_destroy(fg->fs_dr_matcher.dr_matcher);
+}
+
+static struct mlx5dr_action *create_vport_action(struct mlx5dr_domain *domain,
+ struct mlx5_flow_rule *dst)
+{
+ struct mlx5_flow_destination *dest_attr = &dst->dest_attr;
+
+ return mlx5dr_action_create_dest_vport(domain, dest_attr->vport.num,
+ dest_attr->vport.flags &
+ MLX5_FLOW_DEST_VPORT_VHCA_ID,
+ dest_attr->vport.vhca_id);
+}
+
+static struct mlx5dr_action *create_ft_action(struct mlx5_core_dev *dev,
+ struct mlx5_flow_rule *dst)
+{
+ struct mlx5_flow_table *dest_ft = dst->dest_attr.ft;
+
+ if (mlx5_dr_is_fw_table(dest_ft->flags))
+ return mlx5dr_create_action_dest_flow_fw_table(dest_ft, dev);
+ return mlx5dr_action_create_dest_table(dest_ft->fs_dr_table.dr_table);
+}
+
+static struct mlx5dr_action *create_action_push_vlan(struct mlx5dr_domain *domain,
+ struct mlx5_fs_vlan *vlan)
+{
+ u16 n_ethtype = vlan->ethtype;
+ u8 prio = vlan->prio;
+ u16 vid = vlan->vid;
+ u32 vlan_hdr;
+
+ vlan_hdr = (u32)n_ethtype << 16 | (u32)(prio) << 12 | (u32)vid;
+ return mlx5dr_action_create_push_vlan(domain, htonl(vlan_hdr));
+}
+
+#define MLX5_FLOW_CONTEXT_ACTION_MAX 20
+static int mlx5_cmd_dr_create_fte(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ struct mlx5_flow_group *group,
+ struct fs_fte *fte)
+{
+ struct mlx5dr_domain *domain = ns->fs_dr_domain.dr_domain;
+ struct mlx5dr_action *term_action = NULL;
+ struct mlx5dr_match_parameters params;
+ struct mlx5_core_dev *dev = ns->dev;
+ struct mlx5dr_action **fs_dr_actions;
+ struct mlx5dr_action *tmp_action;
+ struct mlx5dr_action **actions;
+ bool delay_encap_set = false;
+ struct mlx5dr_rule *rule;
+ struct mlx5_flow_rule *dst;
+ int fs_dr_num_actions = 0;
+ int num_actions = 0;
+ size_t match_sz;
+ int err = 0;
+ int i;
+
+ if (mlx5_dr_is_fw_table(ft->flags))
+ return mlx5_fs_cmd_get_fw_cmds()->create_fte(ns, ft, group, fte);
+
+ actions = kcalloc(MLX5_FLOW_CONTEXT_ACTION_MAX, sizeof(*actions),
+ GFP_KERNEL);
+ if (!actions)
+ return -ENOMEM;
+
+ fs_dr_actions = kcalloc(MLX5_FLOW_CONTEXT_ACTION_MAX,
+ sizeof(*fs_dr_actions), GFP_KERNEL);
+ if (!fs_dr_actions) {
+ kfree(actions);
+ return -ENOMEM;
+ }
+
+ match_sz = sizeof(fte->val);
+
+ /* The order of the actions are must to be keep, only the following
+ * order is supported by SW steering:
+ * TX: push vlan -> modify header -> encap
+ * RX: decap -> pop vlan -> modify header
+ */
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH) {
+ tmp_action = create_action_push_vlan(domain, &fte->action.vlan[0]);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH_2) {
+ tmp_action = create_action_push_vlan(domain, &fte->action.vlan[1]);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_DECAP) {
+ enum mlx5dr_action_reformat_type decap_type =
+ DR_ACTION_REFORMAT_TYP_TNL_L2_TO_L2;
+
+ tmp_action = mlx5dr_action_create_packet_reformat(domain,
+ decap_type, 0,
+ NULL);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_PACKET_REFORMAT) {
+ bool is_decap = fte->action.pkt_reformat->reformat_type ==
+ MLX5_REFORMAT_TYPE_L3_TUNNEL_TO_L2;
+
+ if (is_decap)
+ actions[num_actions++] =
+ fte->action.pkt_reformat->action.dr_action;
+ else
+ delay_encap_set = true;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP) {
+ tmp_action =
+ mlx5dr_action_create_pop_vlan();
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP_2) {
+ tmp_action =
+ mlx5dr_action_create_pop_vlan();
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)
+ actions[num_actions++] =
+ fte->action.modify_hdr->action.dr_action;
+
+ if (delay_encap_set)
+ actions[num_actions++] =
+ fte->action.pkt_reformat->action.dr_action;
+
+ /* The order of the actions below is not important */
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_DROP) {
+ tmp_action = mlx5dr_action_create_drop();
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ term_action = tmp_action;
+ }
+
+ if (fte->flow_context.flow_tag) {
+ tmp_action =
+ mlx5dr_action_create_tag(fte->flow_context.flow_tag);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ }
+
+ if (fte->action.action & MLX5_FLOW_CONTEXT_ACTION_FWD_DEST) {
+ list_for_each_entry(dst, &fte->node.children, node.list) {
+ enum mlx5_flow_destination_type type = dst->dest_attr.type;
+ u32 id;
+
+ if (num_actions == MLX5_FLOW_CONTEXT_ACTION_MAX) {
+ err = -ENOSPC;
+ goto free_actions;
+ }
+
+ switch (type) {
+ case MLX5_FLOW_DESTINATION_TYPE_COUNTER:
+ id = dst->dest_attr.counter_id;
+
+ tmp_action =
+ mlx5dr_action_create_flow_counter(id);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ actions[num_actions++] = tmp_action;
+ break;
+ case MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE:
+ tmp_action = create_ft_action(dev, dst);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ term_action = tmp_action;
+ break;
+ case MLX5_FLOW_DESTINATION_TYPE_VPORT:
+ tmp_action = create_vport_action(domain, dst);
+ if (!tmp_action) {
+ err = -ENOMEM;
+ goto free_actions;
+ }
+ fs_dr_actions[fs_dr_num_actions++] = tmp_action;
+ term_action = tmp_action;
+ break;
+ default:
+ err = -EOPNOTSUPP;
+ goto free_actions;
+ }
+ }
+ }
+
+ params.match_sz = match_sz;
+ params.match_buf = (u64 *)fte->val;
+
+ if (term_action)
+ actions[num_actions++] = term_action;
+
+ rule = mlx5dr_rule_create(group->fs_dr_matcher.dr_matcher,
+ ¶ms,
+ num_actions,
+ actions);
+ if (!rule) {
+ err = -EINVAL;
+ goto free_actions;
+ }
+
+ kfree(actions);
+ fte->fs_dr_rule.dr_rule = rule;
+ fte->fs_dr_rule.num_actions = fs_dr_num_actions;
+ fte->fs_dr_rule.dr_actions = fs_dr_actions;
+
+ return 0;
+
+free_actions:
+ for (i = 0; i < fs_dr_num_actions; i++)
+ if (!IS_ERR_OR_NULL(fs_dr_actions[i]))
+ mlx5dr_action_destroy(fs_dr_actions[i]);
+
+ mlx5_core_err(dev, "Failed to create dr rule err(%d)\n", err);
+ kfree(actions);
+ kfree(fs_dr_actions);
+ return err;
+}
+
+static int mlx5_cmd_dr_packet_reformat_alloc(struct mlx5_flow_root_namespace *ns,
+ int reformat_type,
+ size_t size,
+ void *reformat_data,
+ enum mlx5_flow_namespace_type namespace,
+ struct mlx5_pkt_reformat *pkt_reformat)
+{
+ struct mlx5dr_domain *dr_domain = ns->fs_dr_domain.dr_domain;
+ struct mlx5dr_action *action;
+ int dr_reformat;
+
+ switch (reformat_type) {
+ case MLX5_REFORMAT_TYPE_L2_TO_VXLAN:
+ case MLX5_REFORMAT_TYPE_L2_TO_NVGRE:
+ case MLX5_REFORMAT_TYPE_L2_TO_L2_TUNNEL:
+ dr_reformat = DR_ACTION_REFORMAT_TYP_L2_TO_TNL_L2;
+ break;
+ case MLX5_REFORMAT_TYPE_L3_TUNNEL_TO_L2:
+ dr_reformat = DR_ACTION_REFORMAT_TYP_TNL_L3_TO_L2;
+ break;
+ case MLX5_REFORMAT_TYPE_L2_TO_L3_TUNNEL:
+ dr_reformat = DR_ACTION_REFORMAT_TYP_L2_TO_TNL_L3;
+ break;
+ default:
+ mlx5_core_err(ns->dev, "Packet-reformat not supported(%d)\n",
+ reformat_type);
+ return -EOPNOTSUPP;
+ }
+
+ action = mlx5dr_action_create_packet_reformat(dr_domain,
+ dr_reformat,
+ size,
+ reformat_data);
+ if (!action) {
+ mlx5_core_err(ns->dev, "Failed allocating packet-reformat action\n");
+ return -EINVAL;
+ }
+
+ pkt_reformat->action.dr_action = action;
+
+ return 0;
+}
+
+static void mlx5_cmd_dr_packet_reformat_dealloc(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_pkt_reformat *pkt_reformat)
+{
+ mlx5dr_action_destroy(pkt_reformat->action.dr_action);
+}
+
+static int mlx5_cmd_dr_modify_header_alloc(struct mlx5_flow_root_namespace *ns,
+ u8 namespace, u8 num_actions,
+ void *modify_actions,
+ struct mlx5_modify_hdr *modify_hdr)
+{
+ struct mlx5dr_domain *dr_domain = ns->fs_dr_domain.dr_domain;
+ struct mlx5dr_action *action;
+ size_t actions_sz;
+
+ actions_sz = MLX5_UN_SZ_BYTES(set_action_in_add_action_in_auto) *
+ num_actions;
+ action = mlx5dr_action_create_modify_header(dr_domain, 0,
+ actions_sz,
+ modify_actions);
+ if (!action) {
+ mlx5_core_err(ns->dev, "Failed allocating modify-header action\n");
+ return -EINVAL;
+ }
+
+ modify_hdr->action.dr_action = action;
+
+ return 0;
+}
+
+static void mlx5_cmd_dr_modify_header_dealloc(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_modify_hdr *modify_hdr)
+{
+ mlx5dr_action_destroy(modify_hdr->action.dr_action);
+}
+
+static int mlx5_cmd_dr_update_fte(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ struct mlx5_flow_group *group,
+ int modify_mask,
+ struct fs_fte *fte)
+{
+ return -EOPNOTSUPP;
+}
+
+static int mlx5_cmd_dr_delete_fte(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_table *ft,
+ struct fs_fte *fte)
+{
+ struct mlx5_fs_dr_rule *rule = &fte->fs_dr_rule;
+ int err;
+ int i;
+
+ if (mlx5_dr_is_fw_table(ft->flags))
+ return mlx5_fs_cmd_get_fw_cmds()->delete_fte(ns, ft, fte);
+
+ err = mlx5dr_rule_destroy(rule->dr_rule);
+ if (err)
+ return err;
+
+ for (i = 0; i < rule->num_actions; i++)
+ if (!IS_ERR_OR_NULL(rule->dr_actions[i]))
+ mlx5dr_action_destroy(rule->dr_actions[i]);
+
+ kfree(rule->dr_actions);
+ return 0;
+}
+
+static int mlx5_cmd_dr_set_peer(struct mlx5_flow_root_namespace *ns,
+ struct mlx5_flow_root_namespace *peer_ns)
+{
+ struct mlx5dr_domain *peer_domain = NULL;
+
+ if (peer_ns)
+ peer_domain = peer_ns->fs_dr_domain.dr_domain;
+ mlx5dr_domain_set_peer(ns->fs_dr_domain.dr_domain,
+ peer_domain);
+ return 0;
+}
+
+static int mlx5_cmd_dr_create_ns(struct mlx5_flow_root_namespace *ns)
+{
+ ns->fs_dr_domain.dr_domain =
+ mlx5dr_domain_create(ns->dev,
+ MLX5DR_DOMAIN_TYPE_FDB);
+ if (!ns->fs_dr_domain.dr_domain) {
+ mlx5_core_err(ns->dev, "Failed to create dr flow namespace\n");
+ return -EOPNOTSUPP;
+ }
+ return 0;
+}
+
+static int mlx5_cmd_dr_destroy_ns(struct mlx5_flow_root_namespace *ns)
+{
+ return mlx5dr_domain_destroy(ns->fs_dr_domain.dr_domain);
+}
+
+bool mlx5_fs_dr_is_supported(struct mlx5_core_dev *dev)
+{
+ return mlx5dr_is_supported(dev);
+}
+
+static const struct mlx5_flow_cmds mlx5_flow_cmds_dr = {
+ .create_flow_table = mlx5_cmd_dr_create_flow_table,
+ .destroy_flow_table = mlx5_cmd_dr_destroy_flow_table,
+ .modify_flow_table = mlx5_cmd_dr_modify_flow_table,
+ .create_flow_group = mlx5_cmd_dr_create_flow_group,
+ .destroy_flow_group = mlx5_cmd_dr_destroy_flow_group,
+ .create_fte = mlx5_cmd_dr_create_fte,
+ .update_fte = mlx5_cmd_dr_update_fte,
+ .delete_fte = mlx5_cmd_dr_delete_fte,
+ .update_root_ft = mlx5_cmd_dr_update_root_ft,
+ .packet_reformat_alloc = mlx5_cmd_dr_packet_reformat_alloc,
+ .packet_reformat_dealloc = mlx5_cmd_dr_packet_reformat_dealloc,
+ .modify_header_alloc = mlx5_cmd_dr_modify_header_alloc,
+ .modify_header_dealloc = mlx5_cmd_dr_modify_header_dealloc,
+ .set_peer = mlx5_cmd_dr_set_peer,
+ .create_ns = mlx5_cmd_dr_create_ns,
+ .destroy_ns = mlx5_cmd_dr_destroy_ns,
+};
+
+const struct mlx5_flow_cmds *mlx5_fs_cmd_get_dr_cmds(void)
+{
+ return &mlx5_flow_cmds_dr;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h
new file mode 100644
index 0000000..1fb185d
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/fs_dr.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+ * Copyright (c) 2019 Mellanox Technologies
+ */
+
+#ifndef _MLX5_FS_DR_
+#define _MLX5_FS_DR_
+
+#include "mlx5dr.h"
+
+struct mlx5_flow_root_namespace;
+struct fs_fte;
+
+struct mlx5_fs_dr_action {
+ struct mlx5dr_action *dr_action;
+};
+
+struct mlx5_fs_dr_ns {
+ struct mlx5_dr_ns *dr_ns;
+};
+
+struct mlx5_fs_dr_rule {
+ struct mlx5dr_rule *dr_rule;
+ /* Only actions created by fs_dr */
+ struct mlx5dr_action **dr_actions;
+ int num_actions;
+};
+
+struct mlx5_fs_dr_domain {
+ struct mlx5dr_domain *dr_domain;
+};
+
+struct mlx5_fs_dr_matcher {
+ struct mlx5dr_matcher *dr_matcher;
+};
+
+struct mlx5_fs_dr_table {
+ struct mlx5dr_table *dr_table;
+ struct mlx5dr_action *miss_action;
+};
+
+#ifdef CONFIG_MLX5_SW_STEERING
+
+bool mlx5_fs_dr_is_supported(struct mlx5_core_dev *dev);
+
+const struct mlx5_flow_cmds *mlx5_fs_cmd_get_dr_cmds(void);
+
+#else
+
+static inline const struct mlx5_flow_cmds *mlx5_fs_cmd_get_dr_cmds(void)
+{
+ return NULL;
+}
+
+static inline bool mlx5_fs_dr_is_supported(struct mlx5_core_dev *dev)
+{
+ return false;
+}
+
+#endif /* CONFIG_MLX5_SW_STEERING */
+#endif
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h
new file mode 100644
index 0000000..596c927
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h
@@ -0,0 +1,604 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2019, Mellanox Technologies */
+
+#ifndef MLX5_IFC_DR_H
+#define MLX5_IFC_DR_H
+
+enum {
+ MLX5DR_ACTION_MDFY_HW_FLD_L2_0 = 0,
+ MLX5DR_ACTION_MDFY_HW_FLD_L2_1 = 1,
+ MLX5DR_ACTION_MDFY_HW_FLD_L2_2 = 2,
+ MLX5DR_ACTION_MDFY_HW_FLD_L3_0 = 3,
+ MLX5DR_ACTION_MDFY_HW_FLD_L3_1 = 4,
+ MLX5DR_ACTION_MDFY_HW_FLD_L3_2 = 5,
+ MLX5DR_ACTION_MDFY_HW_FLD_L3_3 = 6,
+ MLX5DR_ACTION_MDFY_HW_FLD_L3_4 = 7,
+ MLX5DR_ACTION_MDFY_HW_FLD_L4_0 = 8,
+ MLX5DR_ACTION_MDFY_HW_FLD_L4_1 = 9,
+ MLX5DR_ACTION_MDFY_HW_FLD_MPLS = 10,
+ MLX5DR_ACTION_MDFY_HW_FLD_L2_TNL_0 = 11,
+ MLX5DR_ACTION_MDFY_HW_FLD_REG_0 = 12,
+ MLX5DR_ACTION_MDFY_HW_FLD_REG_1 = 13,
+ MLX5DR_ACTION_MDFY_HW_FLD_REG_2 = 14,
+ MLX5DR_ACTION_MDFY_HW_FLD_REG_3 = 15,
+ MLX5DR_ACTION_MDFY_HW_FLD_L4_2 = 16,
+ MLX5DR_ACTION_MDFY_HW_FLD_FLEX_0 = 17,
+ MLX5DR_ACTION_MDFY_HW_FLD_FLEX_1 = 18,
+ MLX5DR_ACTION_MDFY_HW_FLD_FLEX_2 = 19,
+ MLX5DR_ACTION_MDFY_HW_FLD_FLEX_3 = 20,
+ MLX5DR_ACTION_MDFY_HW_FLD_L2_TNL_1 = 21,
+ MLX5DR_ACTION_MDFY_HW_FLD_METADATA = 22,
+ MLX5DR_ACTION_MDFY_HW_FLD_RESERVED = 23,
+};
+
+enum {
+ MLX5DR_ACTION_MDFY_HW_OP_SET = 0x2,
+ MLX5DR_ACTION_MDFY_HW_OP_ADD = 0x3,
+};
+
+enum {
+ MLX5DR_ACTION_MDFY_HW_HDR_L3_NONE = 0x0,
+ MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV4 = 0x1,
+ MLX5DR_ACTION_MDFY_HW_HDR_L3_IPV6 = 0x2,
+};
+
+enum {
+ MLX5DR_ACTION_MDFY_HW_HDR_L4_NONE = 0x0,
+ MLX5DR_ACTION_MDFY_HW_HDR_L4_TCP = 0x1,
+ MLX5DR_ACTION_MDFY_HW_HDR_L4_UDP = 0x2,
+};
+
+enum {
+ MLX5DR_STE_LU_TYPE_NOP = 0x00,
+ MLX5DR_STE_LU_TYPE_SRC_GVMI_AND_QP = 0x05,
+ MLX5DR_STE_LU_TYPE_ETHL2_TUNNELING_I = 0x0a,
+ MLX5DR_STE_LU_TYPE_ETHL2_DST_O = 0x06,
+ MLX5DR_STE_LU_TYPE_ETHL2_DST_I = 0x07,
+ MLX5DR_STE_LU_TYPE_ETHL2_DST_D = 0x1b,
+ MLX5DR_STE_LU_TYPE_ETHL2_SRC_O = 0x08,
+ MLX5DR_STE_LU_TYPE_ETHL2_SRC_I = 0x09,
+ MLX5DR_STE_LU_TYPE_ETHL2_SRC_D = 0x1c,
+ MLX5DR_STE_LU_TYPE_ETHL2_SRC_DST_O = 0x36,
+ MLX5DR_STE_LU_TYPE_ETHL2_SRC_DST_I = 0x37,
+ MLX5DR_STE_LU_TYPE_ETHL2_SRC_DST_D = 0x38,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV6_DST_O = 0x0d,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV6_DST_I = 0x0e,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV6_DST_D = 0x1e,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV6_SRC_O = 0x0f,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV6_SRC_I = 0x10,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV6_SRC_D = 0x1f,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV4_5_TUPLE_O = 0x11,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV4_5_TUPLE_I = 0x12,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV4_5_TUPLE_D = 0x20,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV4_MISC_O = 0x29,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV4_MISC_I = 0x2a,
+ MLX5DR_STE_LU_TYPE_ETHL3_IPV4_MISC_D = 0x2b,
+ MLX5DR_STE_LU_TYPE_ETHL4_O = 0x13,
+ MLX5DR_STE_LU_TYPE_ETHL4_I = 0x14,
+ MLX5DR_STE_LU_TYPE_ETHL4_D = 0x21,
+ MLX5DR_STE_LU_TYPE_ETHL4_MISC_O = 0x2c,
+ MLX5DR_STE_LU_TYPE_ETHL4_MISC_I = 0x2d,
+ MLX5DR_STE_LU_TYPE_ETHL4_MISC_D = 0x2e,
+ MLX5DR_STE_LU_TYPE_MPLS_FIRST_O = 0x15,
+ MLX5DR_STE_LU_TYPE_MPLS_FIRST_I = 0x24,
+ MLX5DR_STE_LU_TYPE_MPLS_FIRST_D = 0x25,
+ MLX5DR_STE_LU_TYPE_GRE = 0x16,
+ MLX5DR_STE_LU_TYPE_FLEX_PARSER_0 = 0x22,
+ MLX5DR_STE_LU_TYPE_FLEX_PARSER_1 = 0x23,
+ MLX5DR_STE_LU_TYPE_FLEX_PARSER_TNL_HEADER = 0x19,
+ MLX5DR_STE_LU_TYPE_GENERAL_PURPOSE = 0x18,
+ MLX5DR_STE_LU_TYPE_STEERING_REGISTERS_0 = 0x2f,
+ MLX5DR_STE_LU_TYPE_STEERING_REGISTERS_1 = 0x30,
+ MLX5DR_STE_LU_TYPE_DONT_CARE = 0x0f,
+};
+
+enum mlx5dr_ste_entry_type {
+ MLX5DR_STE_TYPE_TX = 1,
+ MLX5DR_STE_TYPE_RX = 2,
+ MLX5DR_STE_TYPE_MODIFY_PKT = 6,
+};
+
+struct mlx5_ifc_ste_general_bits {
+ u8 entry_type[0x4];
+ u8 reserved_at_4[0x4];
+ u8 entry_sub_type[0x8];
+ u8 byte_mask[0x10];
+
+ u8 next_table_base_63_48[0x10];
+ u8 next_lu_type[0x8];
+ u8 next_table_base_39_32_size[0x8];
+
+ u8 next_table_base_31_5_size[0x1b];
+ u8 linear_hash_enable[0x1];
+ u8 reserved_at_5c[0x2];
+ u8 next_table_rank[0x2];
+
+ u8 reserved_at_60[0xa0];
+ u8 tag_value[0x60];
+ u8 bit_mask[0x60];
+};
+
+struct mlx5_ifc_ste_sx_transmit_bits {
+ u8 entry_type[0x4];
+ u8 reserved_at_4[0x4];
+ u8 entry_sub_type[0x8];
+ u8 byte_mask[0x10];
+
+ u8 next_table_base_63_48[0x10];
+ u8 next_lu_type[0x8];
+ u8 next_table_base_39_32_size[0x8];
+
+ u8 next_table_base_31_5_size[0x1b];
+ u8 linear_hash_enable[0x1];
+ u8 reserved_at_5c[0x2];
+ u8 next_table_rank[0x2];
+
+ u8 sx_wire[0x1];
+ u8 sx_func_lb[0x1];
+ u8 sx_sniffer[0x1];
+ u8 sx_wire_enable[0x1];
+ u8 sx_func_lb_enable[0x1];
+ u8 sx_sniffer_enable[0x1];
+ u8 action_type[0x3];
+ u8 reserved_at_69[0x1];
+ u8 action_description[0x6];
+ u8 gvmi[0x10];
+
+ u8 encap_pointer_vlan_data[0x20];
+
+ u8 loopback_syndome_en[0x8];
+ u8 loopback_syndome[0x8];
+ u8 counter_trigger[0x10];
+
+ u8 miss_address_63_48[0x10];
+ u8 counter_trigger_23_16[0x8];
+ u8 miss_address_39_32[0x8];
+
+ u8 miss_address_31_6[0x1a];
+ u8 learning_point[0x1];
+ u8 go_back[0x1];
+ u8 match_polarity[0x1];
+ u8 mask_mode[0x1];
+ u8 miss_rank[0x2];
+};
+
+struct mlx5_ifc_ste_rx_steering_mult_bits {
+ u8 entry_type[0x4];
+ u8 reserved_at_4[0x4];
+ u8 entry_sub_type[0x8];
+ u8 byte_mask[0x10];
+
+ u8 next_table_base_63_48[0x10];
+ u8 next_lu_type[0x8];
+ u8 next_table_base_39_32_size[0x8];
+
+ u8 next_table_base_31_5_size[0x1b];
+ u8 linear_hash_enable[0x1];
+ u8 reserved_at_[0x2];
+ u8 next_table_rank[0x2];
+
+ u8 member_count[0x10];
+ u8 gvmi[0x10];
+
+ u8 qp_list_pointer[0x20];
+
+ u8 reserved_at_a0[0x1];
+ u8 tunneling_action[0x3];
+ u8 action_description[0x4];
+ u8 reserved_at_a8[0x8];
+ u8 counter_trigger_15_0[0x10];
+
+ u8 miss_address_63_48[0x10];
+ u8 counter_trigger_23_16[0x08];
+ u8 miss_address_39_32[0x8];
+
+ u8 miss_address_31_6[0x1a];
+ u8 learning_point[0x1];
+ u8 fail_on_error[0x1];
+ u8 match_polarity[0x1];
+ u8 mask_mode[0x1];
+ u8 miss_rank[0x2];
+};
+
+struct mlx5_ifc_ste_modify_packet_bits {
+ u8 entry_type[0x4];
+ u8 reserved_at_4[0x4];
+ u8 entry_sub_type[0x8];
+ u8 byte_mask[0x10];
+
+ u8 next_table_base_63_48[0x10];
+ u8 next_lu_type[0x8];
+ u8 next_table_base_39_32_size[0x8];
+
+ u8 next_table_base_31_5_size[0x1b];
+ u8 linear_hash_enable[0x1];
+ u8 reserved_at_[0x2];
+ u8 next_table_rank[0x2];
+
+ u8 number_of_re_write_actions[0x10];
+ u8 gvmi[0x10];
+
+ u8 header_re_write_actions_pointer[0x20];
+
+ u8 reserved_at_a0[0x1];
+ u8 tunneling_action[0x3];
+ u8 action_description[0x4];
+ u8 reserved_at_a8[0x8];
+ u8 counter_trigger_15_0[0x10];
+
+ u8 miss_address_63_48[0x10];
+ u8 counter_trigger_23_16[0x08];
+ u8 miss_address_39_32[0x8];
+
+ u8 miss_address_31_6[0x1a];
+ u8 learning_point[0x1];
+ u8 fail_on_error[0x1];
+ u8 match_polarity[0x1];
+ u8 mask_mode[0x1];
+ u8 miss_rank[0x2];
+};
+
+struct mlx5_ifc_ste_eth_l2_src_bits {
+ u8 smac_47_16[0x20];
+
+ u8 smac_15_0[0x10];
+ u8 l3_ethertype[0x10];
+
+ u8 qp_type[0x2];
+ u8 ethertype_filter[0x1];
+ u8 reserved_at_43[0x1];
+ u8 sx_sniffer[0x1];
+ u8 force_lb[0x1];
+ u8 functional_lb[0x1];
+ u8 port[0x1];
+ u8 reserved_at_48[0x4];
+ u8 first_priority[0x3];
+ u8 first_cfi[0x1];
+ u8 first_vlan_qualifier[0x2];
+ u8 reserved_at_52[0x2];
+ u8 first_vlan_id[0xc];
+
+ u8 ip_fragmented[0x1];
+ u8 tcp_syn[0x1];
+ u8 encp_type[0x2];
+ u8 l3_type[0x2];
+ u8 l4_type[0x2];
+ u8 reserved_at_68[0x4];
+ u8 second_priority[0x3];
+ u8 second_cfi[0x1];
+ u8 second_vlan_qualifier[0x2];
+ u8 reserved_at_72[0x2];
+ u8 second_vlan_id[0xc];
+};
+
+struct mlx5_ifc_ste_eth_l2_dst_bits {
+ u8 dmac_47_16[0x20];
+
+ u8 dmac_15_0[0x10];
+ u8 l3_ethertype[0x10];
+
+ u8 qp_type[0x2];
+ u8 ethertype_filter[0x1];
+ u8 reserved_at_43[0x1];
+ u8 sx_sniffer[0x1];
+ u8 force_lb[0x1];
+ u8 functional_lb[0x1];
+ u8 port[0x1];
+ u8 reserved_at_48[0x4];
+ u8 first_priority[0x3];
+ u8 first_cfi[0x1];
+ u8 first_vlan_qualifier[0x2];
+ u8 reserved_at_52[0x2];
+ u8 first_vlan_id[0xc];
+
+ u8 ip_fragmented[0x1];
+ u8 tcp_syn[0x1];
+ u8 encp_type[0x2];
+ u8 l3_type[0x2];
+ u8 l4_type[0x2];
+ u8 reserved_at_68[0x4];
+ u8 second_priority[0x3];
+ u8 second_cfi[0x1];
+ u8 second_vlan_qualifier[0x2];
+ u8 reserved_at_72[0x2];
+ u8 second_vlan_id[0xc];
+};
+
+struct mlx5_ifc_ste_eth_l2_src_dst_bits {
+ u8 dmac_47_16[0x20];
+
+ u8 dmac_15_0[0x10];
+ u8 smac_47_32[0x10];
+
+ u8 smac_31_0[0x20];
+
+ u8 sx_sniffer[0x1];
+ u8 force_lb[0x1];
+ u8 functional_lb[0x1];
+ u8 port[0x1];
+ u8 l3_type[0x2];
+ u8 reserved_at_66[0x6];
+ u8 first_priority[0x3];
+ u8 first_cfi[0x1];
+ u8 first_vlan_qualifier[0x2];
+ u8 reserved_at_72[0x2];
+ u8 first_vlan_id[0xc];
+};
+
+struct mlx5_ifc_ste_eth_l3_ipv4_5_tuple_bits {
+ u8 destination_address[0x20];
+
+ u8 source_address[0x20];
+
+ u8 source_port[0x10];
+ u8 destination_port[0x10];
+
+ u8 fragmented[0x1];
+ u8 first_fragment[0x1];
+ u8 reserved_at_62[0x2];
+ u8 reserved_at_64[0x1];
+ u8 ecn[0x2];
+ u8 tcp_ns[0x1];
+ u8 tcp_cwr[0x1];
+ u8 tcp_ece[0x1];
+ u8 tcp_urg[0x1];
+ u8 tcp_ack[0x1];
+ u8 tcp_psh[0x1];
+ u8 tcp_rst[0x1];
+ u8 tcp_syn[0x1];
+ u8 tcp_fin[0x1];
+ u8 dscp[0x6];
+ u8 reserved_at_76[0x2];
+ u8 protocol[0x8];
+};
+
+struct mlx5_ifc_ste_eth_l3_ipv6_dst_bits {
+ u8 dst_ip_127_96[0x20];
+
+ u8 dst_ip_95_64[0x20];
+
+ u8 dst_ip_63_32[0x20];
+
+ u8 dst_ip_31_0[0x20];
+};
+
+struct mlx5_ifc_ste_eth_l2_tnl_bits {
+ u8 dmac_47_16[0x20];
+
+ u8 dmac_15_0[0x10];
+ u8 l3_ethertype[0x10];
+
+ u8 l2_tunneling_network_id[0x20];
+
+ u8 ip_fragmented[0x1];
+ u8 tcp_syn[0x1];
+ u8 encp_type[0x2];
+ u8 l3_type[0x2];
+ u8 l4_type[0x2];
+ u8 first_priority[0x3];
+ u8 first_cfi[0x1];
+ u8 reserved_at_6c[0x3];
+ u8 gre_key_flag[0x1];
+ u8 first_vlan_qualifier[0x2];
+ u8 reserved_at_72[0x2];
+ u8 first_vlan_id[0xc];
+};
+
+struct mlx5_ifc_ste_eth_l3_ipv6_src_bits {
+ u8 src_ip_127_96[0x20];
+
+ u8 src_ip_95_64[0x20];
+
+ u8 src_ip_63_32[0x20];
+
+ u8 src_ip_31_0[0x20];
+};
+
+struct mlx5_ifc_ste_eth_l3_ipv4_misc_bits {
+ u8 version[0x4];
+ u8 ihl[0x4];
+ u8 reserved_at_8[0x8];
+ u8 total_length[0x10];
+
+ u8 identification[0x10];
+ u8 flags[0x3];
+ u8 fragment_offset[0xd];
+
+ u8 time_to_live[0x8];
+ u8 reserved_at_48[0x8];
+ u8 checksum[0x10];
+
+ u8 reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_ste_eth_l4_bits {
+ u8 fragmented[0x1];
+ u8 first_fragment[0x1];
+ u8 reserved_at_2[0x6];
+ u8 protocol[0x8];
+ u8 dst_port[0x10];
+
+ u8 ipv6_version[0x4];
+ u8 reserved_at_24[0x1];
+ u8 ecn[0x2];
+ u8 tcp_ns[0x1];
+ u8 tcp_cwr[0x1];
+ u8 tcp_ece[0x1];
+ u8 tcp_urg[0x1];
+ u8 tcp_ack[0x1];
+ u8 tcp_psh[0x1];
+ u8 tcp_rst[0x1];
+ u8 tcp_syn[0x1];
+ u8 tcp_fin[0x1];
+ u8 src_port[0x10];
+
+ u8 ipv6_payload_length[0x10];
+ u8 ipv6_hop_limit[0x8];
+ u8 dscp[0x6];
+ u8 reserved_at_5e[0x2];
+
+ u8 tcp_data_offset[0x4];
+ u8 reserved_at_64[0x8];
+ u8 flow_label[0x14];
+};
+
+struct mlx5_ifc_ste_eth_l4_misc_bits {
+ u8 checksum[0x10];
+ u8 length[0x10];
+
+ u8 seq_num[0x20];
+
+ u8 ack_num[0x20];
+
+ u8 urgent_pointer[0x10];
+ u8 window_size[0x10];
+};
+
+struct mlx5_ifc_ste_mpls_bits {
+ u8 mpls0_label[0x14];
+ u8 mpls0_exp[0x3];
+ u8 mpls0_s_bos[0x1];
+ u8 mpls0_ttl[0x8];
+
+ u8 mpls1_label[0x20];
+
+ u8 mpls2_label[0x20];
+
+ u8 reserved_at_60[0x16];
+ u8 mpls4_s_bit[0x1];
+ u8 mpls4_qualifier[0x1];
+ u8 mpls3_s_bit[0x1];
+ u8 mpls3_qualifier[0x1];
+ u8 mpls2_s_bit[0x1];
+ u8 mpls2_qualifier[0x1];
+ u8 mpls1_s_bit[0x1];
+ u8 mpls1_qualifier[0x1];
+ u8 mpls0_s_bit[0x1];
+ u8 mpls0_qualifier[0x1];
+};
+
+struct mlx5_ifc_ste_register_0_bits {
+ u8 register_0_h[0x20];
+
+ u8 register_0_l[0x20];
+
+ u8 register_1_h[0x20];
+
+ u8 register_1_l[0x20];
+};
+
+struct mlx5_ifc_ste_register_1_bits {
+ u8 register_2_h[0x20];
+
+ u8 register_2_l[0x20];
+
+ u8 register_3_h[0x20];
+
+ u8 register_3_l[0x20];
+};
+
+struct mlx5_ifc_ste_gre_bits {
+ u8 gre_c_present[0x1];
+ u8 reserved_at_30[0x1];
+ u8 gre_k_present[0x1];
+ u8 gre_s_present[0x1];
+ u8 strict_src_route[0x1];
+ u8 recur[0x3];
+ u8 flags[0x5];
+ u8 version[0x3];
+ u8 gre_protocol[0x10];
+
+ u8 checksum[0x10];
+ u8 offset[0x10];
+
+ u8 gre_key_h[0x18];
+ u8 gre_key_l[0x8];
+
+ u8 seq_num[0x20];
+};
+
+struct mlx5_ifc_ste_flex_parser_0_bits {
+ u8 parser_3_label[0x14];
+ u8 parser_3_exp[0x3];
+ u8 parser_3_s_bos[0x1];
+ u8 parser_3_ttl[0x8];
+
+ u8 flex_parser_2[0x20];
+
+ u8 flex_parser_1[0x20];
+
+ u8 flex_parser_0[0x20];
+};
+
+struct mlx5_ifc_ste_flex_parser_1_bits {
+ u8 flex_parser_7[0x20];
+
+ u8 flex_parser_6[0x20];
+
+ u8 flex_parser_5[0x20];
+
+ u8 flex_parser_4[0x20];
+};
+
+struct mlx5_ifc_ste_flex_parser_tnl_bits {
+ u8 flex_parser_tunneling_header_63_32[0x20];
+
+ u8 flex_parser_tunneling_header_31_0[0x20];
+
+ u8 reserved_at_40[0x40];
+};
+
+struct mlx5_ifc_ste_general_purpose_bits {
+ u8 general_purpose_lookup_field[0x20];
+
+ u8 reserved_at_20[0x20];
+
+ u8 reserved_at_40[0x20];
+
+ u8 reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_ste_src_gvmi_qp_bits {
+ u8 loopback_syndrome[0x8];
+ u8 reserved_at_8[0x8];
+ u8 source_gvmi[0x10];
+
+ u8 reserved_at_20[0x5];
+ u8 force_lb[0x1];
+ u8 functional_lb[0x1];
+ u8 source_is_requestor[0x1];
+ u8 source_qp[0x18];
+
+ u8 reserved_at_40[0x20];
+
+ u8 reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_l2_hdr_bits {
+ u8 dmac_47_16[0x20];
+
+ u8 dmac_15_0[0x10];
+ u8 smac_47_32[0x10];
+
+ u8 smac_31_0[0x20];
+
+ u8 ethertype[0x10];
+ u8 vlan_type[0x10];
+
+ u8 vlan[0x10];
+ u8 reserved_at_90[0x10];
+};
+
+/* Both HW set and HW add share the same HW format with different opcodes */
+struct mlx5_ifc_dr_action_hw_set_bits {
+ u8 opcode[0x8];
+ u8 destination_field_code[0x8];
+ u8 reserved_at_10[0x2];
+ u8 destination_left_shifter[0x6];
+ u8 reserved_at_18[0x3];
+ u8 destination_length[0x5];
+
+ u8 inline_data[0x20];
+};
+
+#endif /* MLX5_IFC_DR_H */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h
new file mode 100644
index 0000000..adda9cb
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5dr.h
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2019, Mellanox Technologies */
+
+#ifndef _MLX5DR_H_
+#define _MLX5DR_H_
+
+struct mlx5dr_domain;
+struct mlx5dr_table;
+struct mlx5dr_matcher;
+struct mlx5dr_rule;
+struct mlx5dr_action;
+
+enum mlx5dr_domain_type {
+ MLX5DR_DOMAIN_TYPE_NIC_RX,
+ MLX5DR_DOMAIN_TYPE_NIC_TX,
+ MLX5DR_DOMAIN_TYPE_FDB,
+};
+
+enum mlx5dr_domain_sync_flags {
+ MLX5DR_DOMAIN_SYNC_FLAGS_SW = 1 << 0,
+ MLX5DR_DOMAIN_SYNC_FLAGS_HW = 1 << 1,
+};
+
+enum mlx5dr_action_reformat_type {
+ DR_ACTION_REFORMAT_TYP_TNL_L2_TO_L2,
+ DR_ACTION_REFORMAT_TYP_L2_TO_TNL_L2,
+ DR_ACTION_REFORMAT_TYP_TNL_L3_TO_L2,
+ DR_ACTION_REFORMAT_TYP_L2_TO_TNL_L3,
+};
+
+struct mlx5dr_match_parameters {
+ size_t match_sz;
+ u64 *match_buf; /* Device spec format */
+};
+
+#ifdef CONFIG_MLX5_SW_STEERING
+
+struct mlx5dr_domain *
+mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type);
+
+int mlx5dr_domain_destroy(struct mlx5dr_domain *domain);
+
+int mlx5dr_domain_sync(struct mlx5dr_domain *domain, u32 flags);
+
+void mlx5dr_domain_set_peer(struct mlx5dr_domain *dmn,
+ struct mlx5dr_domain *peer_dmn);
+
+struct mlx5dr_table *
+mlx5dr_table_create(struct mlx5dr_domain *domain, u32 level);
+
+int mlx5dr_table_destroy(struct mlx5dr_table *table);
+
+u32 mlx5dr_table_get_id(struct mlx5dr_table *table);
+
+struct mlx5dr_matcher *
+mlx5dr_matcher_create(struct mlx5dr_table *table,
+ u16 priority,
+ u8 match_criteria_enable,
+ struct mlx5dr_match_parameters *mask);
+
+int mlx5dr_matcher_destroy(struct mlx5dr_matcher *matcher);
+
+struct mlx5dr_rule *
+mlx5dr_rule_create(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *value,
+ size_t num_actions,
+ struct mlx5dr_action *actions[]);
+
+int mlx5dr_rule_destroy(struct mlx5dr_rule *rule);
+
+int mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
+ struct mlx5dr_action *action);
+
+struct mlx5dr_action *
+mlx5dr_action_create_dest_table(struct mlx5dr_table *table);
+
+struct mlx5dr_action *
+mlx5dr_create_action_dest_flow_fw_table(struct mlx5_flow_table *ft,
+ struct mlx5_core_dev *mdev);
+
+struct mlx5dr_action *
+mlx5dr_action_create_dest_vport(struct mlx5dr_domain *domain,
+ u32 vport, u8 vhca_id_valid,
+ u16 vhca_id);
+
+struct mlx5dr_action *mlx5dr_action_create_drop(void);
+
+struct mlx5dr_action *mlx5dr_action_create_tag(u32 tag_value);
+
+struct mlx5dr_action *
+mlx5dr_action_create_flow_counter(u32 counter_id);
+
+struct mlx5dr_action *
+mlx5dr_action_create_packet_reformat(struct mlx5dr_domain *dmn,
+ enum mlx5dr_action_reformat_type reformat_type,
+ size_t data_sz,
+ void *data);
+
+struct mlx5dr_action *
+mlx5dr_action_create_modify_header(struct mlx5dr_domain *domain,
+ u32 flags,
+ size_t actions_sz,
+ __be64 actions[]);
+
+struct mlx5dr_action *mlx5dr_action_create_pop_vlan(void);
+
+struct mlx5dr_action *
+mlx5dr_action_create_push_vlan(struct mlx5dr_domain *domain, __be32 vlan_hdr);
+
+int mlx5dr_action_destroy(struct mlx5dr_action *action);
+
+static inline bool
+mlx5dr_is_supported(struct mlx5_core_dev *dev)
+{
+ return MLX5_CAP_ESW_FLOWTABLE_FDB(dev, sw_owner);
+}
+
+#else /* CONFIG_MLX5_SW_STEERING */
+
+static inline struct mlx5dr_domain *
+mlx5dr_domain_create(struct mlx5_core_dev *mdev, enum mlx5dr_domain_type type) { return NULL; }
+
+static inline int
+mlx5dr_domain_destroy(struct mlx5dr_domain *domain) { return 0; }
+
+static inline int
+mlx5dr_domain_sync(struct mlx5dr_domain *domain, u32 flags) { return 0; }
+
+static inline void
+mlx5dr_domain_set_peer(struct mlx5dr_domain *dmn,
+ struct mlx5dr_domain *peer_dmn) { }
+
+static inline struct mlx5dr_table *
+mlx5dr_table_create(struct mlx5dr_domain *domain, u32 level) { return NULL; }
+
+static inline int
+mlx5dr_table_destroy(struct mlx5dr_table *table) { return 0; }
+
+static inline u32
+mlx5dr_table_get_id(struct mlx5dr_table *table) { return 0; }
+
+static inline struct mlx5dr_matcher *
+mlx5dr_matcher_create(struct mlx5dr_table *table,
+ u16 priority,
+ u8 match_criteria_enable,
+ struct mlx5dr_match_parameters *mask) { return NULL; }
+
+static inline int
+mlx5dr_matcher_destroy(struct mlx5dr_matcher *matcher) { return 0; }
+
+static inline struct mlx5dr_rule *
+mlx5dr_rule_create(struct mlx5dr_matcher *matcher,
+ struct mlx5dr_match_parameters *value,
+ size_t num_actions,
+ struct mlx5dr_action *actions[]) { return NULL; }
+
+static inline int
+mlx5dr_rule_destroy(struct mlx5dr_rule *rule) { return 0; }
+
+static inline int
+mlx5dr_table_set_miss_action(struct mlx5dr_table *tbl,
+ struct mlx5dr_action *action) { return 0; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_dest_table(struct mlx5dr_table *table) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_create_action_dest_flow_fw_table(struct mlx5_flow_table *ft,
+ struct mlx5_core_dev *mdev) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_dest_vport(struct mlx5dr_domain *domain,
+ u32 vport, u8 vhca_id_valid,
+ u16 vhca_id) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_drop(void) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_tag(u32 tag_value) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_flow_counter(u32 counter_id) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_packet_reformat(struct mlx5dr_domain *dmn,
+ enum mlx5dr_action_reformat_type reformat_type,
+ size_t data_sz,
+ void *data) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_modify_header(struct mlx5dr_domain *domain,
+ u32 flags,
+ size_t actions_sz,
+ __be64 actions[]) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_pop_vlan(void) { return NULL; }
+
+static inline struct mlx5dr_action *
+mlx5dr_action_create_push_vlan(struct mlx5dr_domain *domain,
+ __be32 vlan_hdr) { return NULL; }
+
+static inline int
+mlx5dr_action_destroy(struct mlx5dr_action *action) { return 0; }
+
+static inline bool
+mlx5dr_is_supported(struct mlx5_core_dev *dev) { return false; }
+
+#endif /* CONFIG_MLX5_SW_STEERING */
+
+#endif /* _MLX5DR_H_ */
diff --git a/drivers/net/ethernet/mellanox/mlxsw/reg.h b/drivers/net/ethernet/mellanox/mlxsw/reg.h
index baa20cd..5494cf9 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/reg.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/reg.h
@@ -4126,7 +4126,6 @@ MLXSW_ITEM32(reg, ptys, ext_eth_proto_cap, 0x08, 0, 32);
#define MLXSW_REG_PTYS_ETH_SPEED_20GBASE_KR2 BIT(5)
#define MLXSW_REG_PTYS_ETH_SPEED_40GBASE_CR4 BIT(6)
#define MLXSW_REG_PTYS_ETH_SPEED_40GBASE_KR4 BIT(7)
-#define MLXSW_REG_PTYS_ETH_SPEED_56GBASE_R4 BIT(8)
#define MLXSW_REG_PTYS_ETH_SPEED_10GBASE_CR BIT(12)
#define MLXSW_REG_PTYS_ETH_SPEED_10GBASE_SR BIT(13)
#define MLXSW_REG_PTYS_ETH_SPEED_10GBASE_ER_LR BIT(14)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
index 7de9833..91e4792 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
@@ -48,7 +48,7 @@
#define MLXSW_SP1_FWREV_MAJOR 13
#define MLXSW_SP1_FWREV_MINOR 2000
-#define MLXSW_SP1_FWREV_SUBMINOR 1122
+#define MLXSW_SP1_FWREV_SUBMINOR 1886
#define MLXSW_SP1_FWREV_CAN_RESET_MINOR 1702
static const struct mlxsw_fw_rev mlxsw_sp1_fw_rev = {
@@ -175,6 +175,10 @@ struct mlxsw_sp_ptp_ops {
void (*shaper_work)(struct work_struct *work);
int (*get_ts_info)(struct mlxsw_sp *mlxsw_sp,
struct ethtool_ts_info *info);
+ int (*get_stats_count)(void);
+ void (*get_stats_strings)(u8 **p);
+ void (*get_stats)(struct mlxsw_sp_port *mlxsw_sp_port,
+ u64 *data, int data_index);
};
static int mlxsw_sp_component_query(struct mlxfw_dev *mlxfw_dev,
@@ -2329,6 +2333,7 @@ static void mlxsw_sp_port_get_tc_strings(u8 **p, int tc)
static void mlxsw_sp_port_get_strings(struct net_device *dev,
u32 stringset, u8 *data)
{
+ struct mlxsw_sp_port *mlxsw_sp_port = netdev_priv(dev);
u8 *p = data;
int i;
@@ -2370,6 +2375,7 @@ static void mlxsw_sp_port_get_strings(struct net_device *dev,
for (i = 0; i < TC_MAX_QUEUE; i++)
mlxsw_sp_port_get_tc_strings(&p, i);
+ mlxsw_sp_port->mlxsw_sp->ptp_ops->get_stats_strings(&p);
break;
}
}
@@ -2464,6 +2470,7 @@ static void __mlxsw_sp_port_get_stats(struct net_device *dev,
static void mlxsw_sp_port_get_stats(struct net_device *dev,
struct ethtool_stats *stats, u64 *data)
{
+ struct mlxsw_sp_port *mlxsw_sp_port = netdev_priv(dev);
int i, data_index = 0;
/* IEEE 802.3 Counters */
@@ -2504,13 +2511,21 @@ static void mlxsw_sp_port_get_stats(struct net_device *dev,
data, data_index);
data_index += MLXSW_SP_PORT_HW_TC_STATS_LEN;
}
+
+ /* PTP counters */
+ mlxsw_sp_port->mlxsw_sp->ptp_ops->get_stats(mlxsw_sp_port,
+ data, data_index);
+ data_index += mlxsw_sp_port->mlxsw_sp->ptp_ops->get_stats_count();
}
static int mlxsw_sp_port_get_sset_count(struct net_device *dev, int sset)
{
+ struct mlxsw_sp_port *mlxsw_sp_port = netdev_priv(dev);
+
switch (sset) {
case ETH_SS_STATS:
- return MLXSW_SP_PORT_ETHTOOL_STATS_LEN;
+ return MLXSW_SP_PORT_ETHTOOL_STATS_LEN +
+ mlxsw_sp_port->mlxsw_sp->ptp_ops->get_stats_count();
default:
return -EOPNOTSUPP;
}
@@ -2609,26 +2624,6 @@ static const struct mlxsw_sp1_port_link_mode mlxsw_sp1_port_link_mode[] = {
.speed = SPEED_50000,
},
{
- .mask = MLXSW_REG_PTYS_ETH_SPEED_56GBASE_R4,
- .mask_ethtool = ETHTOOL_LINK_MODE_56000baseKR4_Full_BIT,
- .speed = SPEED_56000,
- },
- {
- .mask = MLXSW_REG_PTYS_ETH_SPEED_56GBASE_R4,
- .mask_ethtool = ETHTOOL_LINK_MODE_56000baseCR4_Full_BIT,
- .speed = SPEED_56000,
- },
- {
- .mask = MLXSW_REG_PTYS_ETH_SPEED_56GBASE_R4,
- .mask_ethtool = ETHTOOL_LINK_MODE_56000baseSR4_Full_BIT,
- .speed = SPEED_56000,
- },
- {
- .mask = MLXSW_REG_PTYS_ETH_SPEED_56GBASE_R4,
- .mask_ethtool = ETHTOOL_LINK_MODE_56000baseLR4_Full_BIT,
- .speed = SPEED_56000,
- },
- {
.mask = MLXSW_REG_PTYS_ETH_SPEED_100GBASE_CR4,
.mask_ethtool = ETHTOOL_LINK_MODE_100000baseCR4_Full_BIT,
.speed = SPEED_100000,
@@ -2675,7 +2670,7 @@ mlxsw_sp1_from_ptys_supported_port(struct mlxsw_sp *mlxsw_sp,
static void
mlxsw_sp1_from_ptys_link(struct mlxsw_sp *mlxsw_sp, u32 ptys_eth_proto,
- unsigned long *mode)
+ u8 width, unsigned long *mode)
{
int i;
@@ -2716,7 +2711,7 @@ mlxsw_sp1_from_ptys_speed_duplex(struct mlxsw_sp *mlxsw_sp, bool carrier_ok,
}
static u32
-mlxsw_sp1_to_ptys_advert_link(struct mlxsw_sp *mlxsw_sp,
+mlxsw_sp1_to_ptys_advert_link(struct mlxsw_sp *mlxsw_sp, u8 width,
const struct ethtool_link_ksettings *cmd)
{
u32 ptys_proto = 0;
@@ -2730,7 +2725,8 @@ mlxsw_sp1_to_ptys_advert_link(struct mlxsw_sp *mlxsw_sp,
return ptys_proto;
}
-static u32 mlxsw_sp1_to_ptys_speed(struct mlxsw_sp *mlxsw_sp, u32 speed)
+static u32 mlxsw_sp1_to_ptys_speed(struct mlxsw_sp *mlxsw_sp, u8 width,
+ u32 speed)
{
u32 ptys_proto = 0;
int i;
@@ -2918,11 +2914,31 @@ mlxsw_sp2_mask_ethtool_200gaui_4_200gbase_cr4_kr4[] = {
#define MLXSW_SP2_MASK_ETHTOOL_200GAUI_4_200GBASE_CR4_KR4_LEN \
ARRAY_SIZE(mlxsw_sp2_mask_ethtool_200gaui_4_200gbase_cr4_kr4)
+#define MLXSW_SP_PORT_MASK_WIDTH_1X BIT(0)
+#define MLXSW_SP_PORT_MASK_WIDTH_2X BIT(1)
+#define MLXSW_SP_PORT_MASK_WIDTH_4X BIT(2)
+
+static u8 mlxsw_sp_port_mask_width_get(u8 width)
+{
+ switch (width) {
+ case 1:
+ return MLXSW_SP_PORT_MASK_WIDTH_1X;
+ case 2:
+ return MLXSW_SP_PORT_MASK_WIDTH_2X;
+ case 4:
+ return MLXSW_SP_PORT_MASK_WIDTH_4X;
+ default:
+ WARN_ON_ONCE(1);
+ return 0;
+ }
+}
+
struct mlxsw_sp2_port_link_mode {
const enum ethtool_link_mode_bit_indices *mask_ethtool;
int m_ethtool_len;
u32 mask;
u32 speed;
+ u8 mask_width;
};
static const struct mlxsw_sp2_port_link_mode mlxsw_sp2_port_link_mode[] = {
@@ -2930,72 +2946,97 @@ static const struct mlxsw_sp2_port_link_mode mlxsw_sp2_port_link_mode[] = {
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_SGMII_100M,
.mask_ethtool = mlxsw_sp2_mask_ethtool_sgmii_100m,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_SGMII_100M_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X |
+ MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_100,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_1000BASE_X_SGMII,
.mask_ethtool = mlxsw_sp2_mask_ethtool_1000base_x_sgmii,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_1000BASE_X_SGMII_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X |
+ MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_1000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_2_5GBASE_X_2_5GMII,
.mask_ethtool = mlxsw_sp2_mask_ethtool_2_5gbase_x_2_5gmii,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_2_5GBASE_X_2_5GMII_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X |
+ MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_2500,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_5GBASE_R,
.mask_ethtool = mlxsw_sp2_mask_ethtool_5gbase_r,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_5GBASE_R_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X |
+ MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_5000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_XFI_XAUI_1_10G,
.mask_ethtool = mlxsw_sp2_mask_ethtool_xfi_xaui_1_10g,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_XFI_XAUI_1_10G_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X |
+ MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_10000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_XLAUI_4_XLPPI_4_40G,
.mask_ethtool = mlxsw_sp2_mask_ethtool_xlaui_4_xlppi_4_40g,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_XLAUI_4_XLPPI_4_40G_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_40000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_25GAUI_1_25GBASE_CR_KR,
.mask_ethtool = mlxsw_sp2_mask_ethtool_25gaui_1_25gbase_cr_kr,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_25GAUI_1_25GBASE_CR_KR_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X |
+ MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_25000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_50GAUI_2_LAUI_2_50GBASE_CR2_KR2,
.mask_ethtool = mlxsw_sp2_mask_ethtool_50gaui_2_laui_2_50gbase_cr2_kr2,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_50GAUI_2_LAUI_2_50GBASE_CR2_KR2_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_2X |
+ MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_50000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_50GAUI_1_LAUI_1_50GBASE_CR_KR,
.mask_ethtool = mlxsw_sp2_mask_ethtool_50gaui_1_laui_1_50gbase_cr_kr,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_50GAUI_1_LAUI_1_50GBASE_CR_KR_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_1X,
.speed = SPEED_50000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_CAUI_4_100GBASE_CR4_KR4,
.mask_ethtool = mlxsw_sp2_mask_ethtool_caui_4_100gbase_cr4_kr4,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_CAUI_4_100GBASE_CR4_KR4_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_100000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_100GAUI_2_100GBASE_CR2_KR2,
.mask_ethtool = mlxsw_sp2_mask_ethtool_100gaui_2_100gbase_cr2_kr2,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_100GAUI_2_100GBASE_CR2_KR2_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_2X,
.speed = SPEED_100000,
},
{
.mask = MLXSW_REG_PTYS_EXT_ETH_SPEED_200GAUI_4_200GBASE_CR4_KR4,
.mask_ethtool = mlxsw_sp2_mask_ethtool_200gaui_4_200gbase_cr4_kr4,
.m_ethtool_len = MLXSW_SP2_MASK_ETHTOOL_200GAUI_4_200GBASE_CR4_KR4_LEN,
+ .mask_width = MLXSW_SP_PORT_MASK_WIDTH_4X,
.speed = SPEED_200000,
},
};
@@ -3023,12 +3064,14 @@ mlxsw_sp2_set_bit_ethtool(const struct mlxsw_sp2_port_link_mode *link_mode,
static void
mlxsw_sp2_from_ptys_link(struct mlxsw_sp *mlxsw_sp, u32 ptys_eth_proto,
- unsigned long *mode)
+ u8 width, unsigned long *mode)
{
+ u8 mask_width = mlxsw_sp_port_mask_width_get(width);
int i;
for (i = 0; i < MLXSW_SP2_PORT_LINK_MODE_LEN; i++) {
- if (ptys_eth_proto & mlxsw_sp2_port_link_mode[i].mask)
+ if ((ptys_eth_proto & mlxsw_sp2_port_link_mode[i].mask) &&
+ (mask_width & mlxsw_sp2_port_link_mode[i].mask_width))
mlxsw_sp2_set_bit_ethtool(&mlxsw_sp2_port_link_mode[i],
mode);
}
@@ -3079,27 +3122,32 @@ mlxsw_sp2_test_bit_ethtool(const struct mlxsw_sp2_port_link_mode *link_mode,
}
static u32
-mlxsw_sp2_to_ptys_advert_link(struct mlxsw_sp *mlxsw_sp,
+mlxsw_sp2_to_ptys_advert_link(struct mlxsw_sp *mlxsw_sp, u8 width,
const struct ethtool_link_ksettings *cmd)
{
+ u8 mask_width = mlxsw_sp_port_mask_width_get(width);
u32 ptys_proto = 0;
int i;
for (i = 0; i < MLXSW_SP2_PORT_LINK_MODE_LEN; i++) {
- if (mlxsw_sp2_test_bit_ethtool(&mlxsw_sp2_port_link_mode[i],
+ if ((mask_width & mlxsw_sp2_port_link_mode[i].mask_width) &&
+ mlxsw_sp2_test_bit_ethtool(&mlxsw_sp2_port_link_mode[i],
cmd->link_modes.advertising))
ptys_proto |= mlxsw_sp2_port_link_mode[i].mask;
}
return ptys_proto;
}
-static u32 mlxsw_sp2_to_ptys_speed(struct mlxsw_sp *mlxsw_sp, u32 speed)
+static u32 mlxsw_sp2_to_ptys_speed(struct mlxsw_sp *mlxsw_sp,
+ u8 width, u32 speed)
{
+ u8 mask_width = mlxsw_sp_port_mask_width_get(width);
u32 ptys_proto = 0;
int i;
for (i = 0; i < MLXSW_SP2_PORT_LINK_MODE_LEN; i++) {
- if (speed == mlxsw_sp2_port_link_mode[i].speed)
+ if ((speed == mlxsw_sp2_port_link_mode[i].speed) &&
+ (mask_width & mlxsw_sp2_port_link_mode[i].mask_width))
ptys_proto |= mlxsw_sp2_port_link_mode[i].mask;
}
return ptys_proto;
@@ -3183,7 +3231,7 @@ mlxsw_sp2_port_type_speed_ops = {
static void
mlxsw_sp_port_get_link_supported(struct mlxsw_sp *mlxsw_sp, u32 eth_proto_cap,
- struct ethtool_link_ksettings *cmd)
+ u8 width, struct ethtool_link_ksettings *cmd)
{
const struct mlxsw_sp_port_type_speed_ops *ops;
@@ -3194,12 +3242,13 @@ mlxsw_sp_port_get_link_supported(struct mlxsw_sp *mlxsw_sp, u32 eth_proto_cap,
ethtool_link_ksettings_add_link_mode(cmd, supported, Pause);
ops->from_ptys_supported_port(mlxsw_sp, eth_proto_cap, cmd);
- ops->from_ptys_link(mlxsw_sp, eth_proto_cap, cmd->link_modes.supported);
+ ops->from_ptys_link(mlxsw_sp, eth_proto_cap, width,
+ cmd->link_modes.supported);
}
static void
mlxsw_sp_port_get_link_advertise(struct mlxsw_sp *mlxsw_sp,
- u32 eth_proto_admin, bool autoneg,
+ u32 eth_proto_admin, bool autoneg, u8 width,
struct ethtool_link_ksettings *cmd)
{
const struct mlxsw_sp_port_type_speed_ops *ops;
@@ -3210,7 +3259,7 @@ mlxsw_sp_port_get_link_advertise(struct mlxsw_sp *mlxsw_sp,
return;
ethtool_link_ksettings_add_link_mode(cmd, advertising, Autoneg);
- ops->from_ptys_link(mlxsw_sp, eth_proto_admin,
+ ops->from_ptys_link(mlxsw_sp, eth_proto_admin, width,
cmd->link_modes.advertising);
}
@@ -3265,10 +3314,11 @@ static int mlxsw_sp_port_get_link_ksettings(struct net_device *dev,
ops->reg_ptys_eth_unpack(mlxsw_sp, ptys_pl, ð_proto_cap,
ð_proto_admin, ð_proto_oper);
- mlxsw_sp_port_get_link_supported(mlxsw_sp, eth_proto_cap, cmd);
+ mlxsw_sp_port_get_link_supported(mlxsw_sp, eth_proto_cap,
+ mlxsw_sp_port->mapping.width, cmd);
mlxsw_sp_port_get_link_advertise(mlxsw_sp, eth_proto_admin, autoneg,
- cmd);
+ mlxsw_sp_port->mapping.width, cmd);
cmd->base.autoneg = autoneg ? AUTONEG_ENABLE : AUTONEG_DISABLE;
connector_type = mlxsw_reg_ptys_connector_type_get(ptys_pl);
@@ -3301,13 +3351,11 @@ mlxsw_sp_port_set_link_ksettings(struct net_device *dev,
ops->reg_ptys_eth_unpack(mlxsw_sp, ptys_pl, ð_proto_cap, NULL, NULL);
autoneg = cmd->base.autoneg == AUTONEG_ENABLE;
- if (!autoneg && cmd->base.speed == SPEED_56000) {
- netdev_err(dev, "56G not supported with autoneg off\n");
- return -EINVAL;
- }
eth_proto_new = autoneg ?
- ops->to_ptys_advert_link(mlxsw_sp, cmd) :
- ops->to_ptys_speed(mlxsw_sp, cmd->base.speed);
+ ops->to_ptys_advert_link(mlxsw_sp, mlxsw_sp_port->mapping.width,
+ cmd) :
+ ops->to_ptys_speed(mlxsw_sp, mlxsw_sp_port->mapping.width,
+ cmd->base.speed);
eth_proto_new = eth_proto_new & eth_proto_cap;
if (!eth_proto_new) {
@@ -4610,6 +4658,9 @@ static const struct mlxsw_sp_ptp_ops mlxsw_sp1_ptp_ops = {
.hwtstamp_set = mlxsw_sp1_ptp_hwtstamp_set,
.shaper_work = mlxsw_sp1_ptp_shaper_work,
.get_ts_info = mlxsw_sp1_ptp_get_ts_info,
+ .get_stats_count = mlxsw_sp1_get_stats_count,
+ .get_stats_strings = mlxsw_sp1_get_stats_strings,
+ .get_stats = mlxsw_sp1_get_stats,
};
static const struct mlxsw_sp_ptp_ops mlxsw_sp2_ptp_ops = {
@@ -4623,6 +4674,9 @@ static const struct mlxsw_sp_ptp_ops mlxsw_sp2_ptp_ops = {
.hwtstamp_set = mlxsw_sp2_ptp_hwtstamp_set,
.shaper_work = mlxsw_sp2_ptp_shaper_work,
.get_ts_info = mlxsw_sp2_ptp_get_ts_info,
+ .get_stats_count = mlxsw_sp2_get_stats_count,
+ .get_stats_strings = mlxsw_sp2_get_stats_strings,
+ .get_stats = mlxsw_sp2_get_stats,
};
static int mlxsw_sp_netdevice_event(struct notifier_block *unused,
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
index 20c14bb..b2a0028 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
@@ -225,6 +225,16 @@ struct mlxsw_sp_port_xstats {
u64 tx_packets[IEEE_8021QAZ_MAX_TCS];
};
+struct mlxsw_sp_ptp_port_dir_stats {
+ u64 packets;
+ u64 timestamps;
+};
+
+struct mlxsw_sp_ptp_port_stats {
+ struct mlxsw_sp_ptp_port_dir_stats rx_gcd;
+ struct mlxsw_sp_ptp_port_dir_stats tx_gcd;
+};
+
struct mlxsw_sp_port {
struct net_device *dev;
struct mlxsw_sp_port_pcpu_stats __percpu *pcpu_stats;
@@ -271,6 +281,7 @@ struct mlxsw_sp_port {
struct hwtstamp_config hwtstamp_config;
u16 ing_types;
u16 egr_types;
+ struct mlxsw_sp_ptp_port_stats stats;
} ptp;
};
@@ -279,14 +290,14 @@ struct mlxsw_sp_port_type_speed_ops {
u32 ptys_eth_proto,
struct ethtool_link_ksettings *cmd);
void (*from_ptys_link)(struct mlxsw_sp *mlxsw_sp, u32 ptys_eth_proto,
- unsigned long *mode);
+ u8 width, unsigned long *mode);
u32 (*from_ptys_speed)(struct mlxsw_sp *mlxsw_sp, u32 ptys_eth_proto);
void (*from_ptys_speed_duplex)(struct mlxsw_sp *mlxsw_sp,
bool carrier_ok, u32 ptys_eth_proto,
struct ethtool_link_ksettings *cmd);
- u32 (*to_ptys_advert_link)(struct mlxsw_sp *mlxsw_sp,
+ u32 (*to_ptys_advert_link)(struct mlxsw_sp *mlxsw_sp, u8 width,
const struct ethtool_link_ksettings *cmd);
- u32 (*to_ptys_speed)(struct mlxsw_sp *mlxsw_sp, u32 speed);
+ u32 (*to_ptys_speed)(struct mlxsw_sp *mlxsw_sp, u8 width, u32 speed);
u32 (*to_ptys_upper_speed)(struct mlxsw_sp *mlxsw_sp, u32 upper_speed);
int (*port_speed_base)(struct mlxsw_sp *mlxsw_sp, u8 local_port,
u32 *base_speed);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
index 38bb1cf..ec2ff3d 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
@@ -630,6 +630,8 @@ static void
mlxsw_sp1_ptp_ht_gc_collect(struct mlxsw_sp_ptp_state *ptp_state,
struct mlxsw_sp1_ptp_unmatched *unmatched)
{
+ struct mlxsw_sp_ptp_port_dir_stats *stats;
+ struct mlxsw_sp_port *mlxsw_sp_port;
int err;
/* If an unmatched entry has an SKB, it has to be handed over to the
@@ -650,6 +652,17 @@ mlxsw_sp1_ptp_ht_gc_collect(struct mlxsw_sp_ptp_state *ptp_state,
/* The packet was matched with timestamp during the walk. */
goto out;
+ mlxsw_sp_port = ptp_state->mlxsw_sp->ports[unmatched->key.local_port];
+ if (mlxsw_sp_port) {
+ stats = unmatched->key.ingress ?
+ &mlxsw_sp_port->ptp.stats.rx_gcd :
+ &mlxsw_sp_port->ptp.stats.tx_gcd;
+ if (unmatched->skb)
+ stats->packets++;
+ else
+ stats->timestamps++;
+ }
+
/* mlxsw_sp1_ptp_unmatched_finish() invokes netif_receive_skb(). While
* the comment at that function states that it can only be called in
* soft IRQ context, this pattern of local_bh_disable() +
@@ -1098,3 +1111,57 @@ int mlxsw_sp1_ptp_get_ts_info(struct mlxsw_sp *mlxsw_sp,
return 0;
}
+
+struct mlxsw_sp_ptp_port_stat {
+ char str[ETH_GSTRING_LEN];
+ ptrdiff_t offset;
+};
+
+#define MLXSW_SP_PTP_PORT_STAT(NAME, FIELD) \
+ { \
+ .str = NAME, \
+ .offset = offsetof(struct mlxsw_sp_ptp_port_stats, \
+ FIELD), \
+ }
+
+static const struct mlxsw_sp_ptp_port_stat mlxsw_sp_ptp_port_stats[] = {
+ MLXSW_SP_PTP_PORT_STAT("ptp_rx_gcd_packets", rx_gcd.packets),
+ MLXSW_SP_PTP_PORT_STAT("ptp_rx_gcd_timestamps", rx_gcd.timestamps),
+ MLXSW_SP_PTP_PORT_STAT("ptp_tx_gcd_packets", tx_gcd.packets),
+ MLXSW_SP_PTP_PORT_STAT("ptp_tx_gcd_timestamps", tx_gcd.timestamps),
+};
+
+#undef MLXSW_SP_PTP_PORT_STAT
+
+#define MLXSW_SP_PTP_PORT_STATS_LEN \
+ ARRAY_SIZE(mlxsw_sp_ptp_port_stats)
+
+int mlxsw_sp1_get_stats_count(void)
+{
+ return MLXSW_SP_PTP_PORT_STATS_LEN;
+}
+
+void mlxsw_sp1_get_stats_strings(u8 **p)
+{
+ int i;
+
+ for (i = 0; i < MLXSW_SP_PTP_PORT_STATS_LEN; i++) {
+ memcpy(*p, mlxsw_sp_ptp_port_stats[i].str,
+ ETH_GSTRING_LEN);
+ *p += ETH_GSTRING_LEN;
+ }
+}
+
+void mlxsw_sp1_get_stats(struct mlxsw_sp_port *mlxsw_sp_port,
+ u64 *data, int data_index)
+{
+ void *stats = &mlxsw_sp_port->ptp.stats;
+ ptrdiff_t offset;
+ int i;
+
+ data += data_index;
+ for (i = 0; i < MLXSW_SP_PTP_PORT_STATS_LEN; i++) {
+ offset = mlxsw_sp_ptp_port_stats[i].offset;
+ *data++ = *(u64 *)(stats + offset);
+ }
+}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h
index 72e55f6..8c38657 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.h
@@ -59,6 +59,11 @@ void mlxsw_sp1_ptp_shaper_work(struct work_struct *work);
int mlxsw_sp1_ptp_get_ts_info(struct mlxsw_sp *mlxsw_sp,
struct ethtool_ts_info *info);
+int mlxsw_sp1_get_stats_count(void);
+void mlxsw_sp1_get_stats_strings(u8 **p);
+void mlxsw_sp1_get_stats(struct mlxsw_sp_port *mlxsw_sp_port,
+ u64 *data, int data_index);
+
#else
static inline struct mlxsw_sp_ptp_clock *
@@ -125,6 +130,19 @@ static inline int mlxsw_sp1_ptp_get_ts_info(struct mlxsw_sp *mlxsw_sp,
return mlxsw_sp_ptp_get_ts_info_noptp(info);
}
+static inline int mlxsw_sp1_get_stats_count(void)
+{
+ return 0;
+}
+
+static inline void mlxsw_sp1_get_stats_strings(u8 **p)
+{
+}
+
+static inline void mlxsw_sp1_get_stats(struct mlxsw_sp_port *mlxsw_sp_port,
+ u64 *data, int data_index)
+{
+}
#endif
static inline struct mlxsw_sp_ptp_clock *
@@ -183,4 +201,18 @@ static inline int mlxsw_sp2_ptp_get_ts_info(struct mlxsw_sp *mlxsw_sp,
return mlxsw_sp_ptp_get_ts_info_noptp(info);
}
+static inline int mlxsw_sp2_get_stats_count(void)
+{
+ return 0;
+}
+
+static inline void mlxsw_sp2_get_stats_strings(u8 **p)
+{
+}
+
+static inline void mlxsw_sp2_get_stats(struct mlxsw_sp_port *mlxsw_sp_port,
+ u64 *data, int data_index)
+{
+}
+
#endif
diff --git a/drivers/net/ethernet/mellanox/mlxsw/switchx2.c b/drivers/net/ethernet/mellanox/mlxsw/switchx2.c
index bdab96f5..1c14c05 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/switchx2.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/switchx2.c
@@ -637,12 +637,6 @@ static const struct mlxsw_sx_port_link_mode mlxsw_sx_port_link_mode[] = {
.speed = 50000,
},
{
- .mask = MLXSW_REG_PTYS_ETH_SPEED_56GBASE_R4,
- .supported = SUPPORTED_56000baseKR4_Full,
- .advertised = ADVERTISED_56000baseKR4_Full,
- .speed = 56000,
- },
- {
.mask = MLXSW_REG_PTYS_ETH_SPEED_100GBASE_CR4 |
MLXSW_REG_PTYS_ETH_SPEED_100GBASE_SR4 |
MLXSW_REG_PTYS_ETH_SPEED_100GBASE_KR4 |
diff --git a/drivers/net/ethernet/micrel/ks8842.c b/drivers/net/ethernet/micrel/ks8842.c
index ccd0670..da329ca 100644
--- a/drivers/net/ethernet/micrel/ks8842.c
+++ b/drivers/net/ethernet/micrel/ks8842.c
@@ -580,9 +580,7 @@ static int __ks8842_start_new_rx_dma(struct net_device *netdev)
dma_unmap_single(adapter->dev, sg_dma_address(sg),
DMA_BUFFER_SIZE, DMA_FROM_DEVICE);
sg_dma_address(sg) = 0;
- if (ctl->skb)
- dev_kfree_skb(ctl->skb);
-
+ dev_kfree_skb(ctl->skb);
ctl->skb = NULL;
printk(KERN_ERR DRV_NAME": Failed to start RX DMA: %d\n", err);
diff --git a/drivers/net/ethernet/micrel/ksz884x.c b/drivers/net/ethernet/micrel/ksz884x.c
index 3103446..e102e15 100644
--- a/drivers/net/ethernet/micrel/ksz884x.c
+++ b/drivers/net/ethernet/micrel/ksz884x.c
@@ -2166,7 +2166,7 @@ static void sw_get_broad_storm(struct ksz_hw *hw, u8 *percent)
num = (data & BROADCAST_STORM_RATE_HI);
num <<= 8;
num |= (data & BROADCAST_STORM_RATE_LO) >> 8;
- num = (num * 100 + BROADCAST_STORM_VALUE / 2) / BROADCAST_STORM_VALUE;
+ num = DIV_ROUND_CLOSEST(num * 100, BROADCAST_STORM_VALUE);
*percent = (u8) num;
}
diff --git a/drivers/net/ethernet/microchip/lan743x_main.c b/drivers/net/ethernet/microchip/lan743x_main.c
index 15a8be6..a43140f 100644
--- a/drivers/net/ethernet/microchip/lan743x_main.c
+++ b/drivers/net/ethernet/microchip/lan743x_main.c
@@ -2172,9 +2172,8 @@ static int lan743x_rx_napi_poll(struct napi_struct *napi, int weight)
}
count = 0;
while (count < weight) {
- int rx_process_result = -1;
+ int rx_process_result = lan743x_rx_process_packet(rx);
- rx_process_result = lan743x_rx_process_packet(rx);
if (rx_process_result == RX_PROCESS_RESULT_PACKET_RECEIVED) {
count++;
} else if (rx_process_result ==
diff --git a/drivers/net/ethernet/microchip/lan743x_ptp.c b/drivers/net/ethernet/microchip/lan743x_ptp.c
index b2109ec..57b26c2 100644
--- a/drivers/net/ethernet/microchip/lan743x_ptp.c
+++ b/drivers/net/ethernet/microchip/lan743x_ptp.c
@@ -963,8 +963,7 @@ void lan743x_ptp_close(struct lan743x_adapter *adapter)
index++) {
struct sk_buff *skb = ptp->tx_ts_skb_queue[index];
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
ptp->tx_ts_skb_queue[index] = NULL;
ptp->tx_ts_seconds_queue[index] = 0;
ptp->tx_ts_nseconds_queue[index] = 0;
diff --git a/drivers/net/ethernet/mscc/ocelot_ace.c b/drivers/net/ethernet/mscc/ocelot_ace.c
index 39aca1a..86fc6e6 100644
--- a/drivers/net/ethernet/mscc/ocelot_ace.c
+++ b/drivers/net/ethernet/mscc/ocelot_ace.c
@@ -317,7 +317,7 @@ static void is2_action_set(struct vcap_data *data,
break;
case OCELOT_ACL_ACTION_TRAP:
VCAP_ACT_SET(PORT_MASK, 0x0);
- VCAP_ACT_SET(MASK_MODE, 0x0);
+ VCAP_ACT_SET(MASK_MODE, 0x1);
VCAP_ACT_SET(POLICE_ENA, 0x0);
VCAP_ACT_SET(POLICE_IDX, 0x0);
VCAP_ACT_SET(CPU_QU_NUM, 0x0);
diff --git a/drivers/net/ethernet/netronome/nfp/Makefile b/drivers/net/ethernet/netronome/nfp/Makefile
index 2805641..d31772a 100644
--- a/drivers/net/ethernet/netronome/nfp/Makefile
+++ b/drivers/net/ethernet/netronome/nfp/Makefile
@@ -17,6 +17,7 @@
nfpcore/nfp_target.o \
ccm.o \
ccm_mbox.o \
+ devlink_param.o \
nfp_asm.o \
nfp_app.o \
nfp_app_nic.o \
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c b/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c
index bc9850e..0e2db6e 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c
+++ b/drivers/net/ethernet/netronome/nfp/bpf/cmsg.c
@@ -6,6 +6,7 @@
#include <linux/bug.h>
#include <linux/jiffies.h>
#include <linux/skbuff.h>
+#include <linux/timekeeping.h>
#include "../ccm.h"
#include "../nfp_app.h"
@@ -175,29 +176,151 @@ nfp_bpf_ctrl_reply_val(struct nfp_app_bpf *bpf, struct cmsg_reply_map_op *reply,
return &reply->data[bpf->cmsg_key_sz * (n + 1) + bpf->cmsg_val_sz * n];
}
+static bool nfp_bpf_ctrl_op_cache_invalidate(enum nfp_ccm_type op)
+{
+ return op == NFP_CCM_TYPE_BPF_MAP_UPDATE ||
+ op == NFP_CCM_TYPE_BPF_MAP_DELETE;
+}
+
+static bool nfp_bpf_ctrl_op_cache_capable(enum nfp_ccm_type op)
+{
+ return op == NFP_CCM_TYPE_BPF_MAP_LOOKUP ||
+ op == NFP_CCM_TYPE_BPF_MAP_GETNEXT;
+}
+
+static bool nfp_bpf_ctrl_op_cache_fill(enum nfp_ccm_type op)
+{
+ return op == NFP_CCM_TYPE_BPF_MAP_GETFIRST ||
+ op == NFP_CCM_TYPE_BPF_MAP_GETNEXT;
+}
+
+static unsigned int
+nfp_bpf_ctrl_op_cache_get(struct nfp_bpf_map *nfp_map, enum nfp_ccm_type op,
+ const u8 *key, u8 *out_key, u8 *out_value,
+ u32 *cache_gen)
+{
+ struct bpf_map *map = &nfp_map->offmap->map;
+ struct nfp_app_bpf *bpf = nfp_map->bpf;
+ unsigned int i, count, n_entries;
+ struct cmsg_reply_map_op *reply;
+
+ n_entries = nfp_bpf_ctrl_op_cache_fill(op) ? bpf->cmsg_cache_cnt : 1;
+
+ spin_lock(&nfp_map->cache_lock);
+ *cache_gen = nfp_map->cache_gen;
+ if (nfp_map->cache_blockers)
+ n_entries = 1;
+
+ if (nfp_bpf_ctrl_op_cache_invalidate(op))
+ goto exit_block;
+ if (!nfp_bpf_ctrl_op_cache_capable(op))
+ goto exit_unlock;
+
+ if (!nfp_map->cache)
+ goto exit_unlock;
+ if (nfp_map->cache_to < ktime_get_ns())
+ goto exit_invalidate;
+
+ reply = (void *)nfp_map->cache->data;
+ count = be32_to_cpu(reply->count);
+
+ for (i = 0; i < count; i++) {
+ void *cached_key;
+
+ cached_key = nfp_bpf_ctrl_reply_key(bpf, reply, i);
+ if (memcmp(cached_key, key, map->key_size))
+ continue;
+
+ if (op == NFP_CCM_TYPE_BPF_MAP_LOOKUP)
+ memcpy(out_value, nfp_bpf_ctrl_reply_val(bpf, reply, i),
+ map->value_size);
+ if (op == NFP_CCM_TYPE_BPF_MAP_GETNEXT) {
+ if (i + 1 == count)
+ break;
+
+ memcpy(out_key,
+ nfp_bpf_ctrl_reply_key(bpf, reply, i + 1),
+ map->key_size);
+ }
+
+ n_entries = 0;
+ goto exit_unlock;
+ }
+ goto exit_unlock;
+
+exit_block:
+ nfp_map->cache_blockers++;
+exit_invalidate:
+ dev_consume_skb_any(nfp_map->cache);
+ nfp_map->cache = NULL;
+exit_unlock:
+ spin_unlock(&nfp_map->cache_lock);
+ return n_entries;
+}
+
+static void
+nfp_bpf_ctrl_op_cache_put(struct nfp_bpf_map *nfp_map, enum nfp_ccm_type op,
+ struct sk_buff *skb, u32 cache_gen)
+{
+ bool blocker, filler;
+
+ blocker = nfp_bpf_ctrl_op_cache_invalidate(op);
+ filler = nfp_bpf_ctrl_op_cache_fill(op);
+ if (blocker || filler) {
+ u64 to = 0;
+
+ if (filler)
+ to = ktime_get_ns() + NFP_BPF_MAP_CACHE_TIME_NS;
+
+ spin_lock(&nfp_map->cache_lock);
+ if (blocker) {
+ nfp_map->cache_blockers--;
+ nfp_map->cache_gen++;
+ }
+ if (filler && !nfp_map->cache_blockers &&
+ nfp_map->cache_gen == cache_gen) {
+ nfp_map->cache_to = to;
+ swap(nfp_map->cache, skb);
+ }
+ spin_unlock(&nfp_map->cache_lock);
+ }
+
+ dev_consume_skb_any(skb);
+}
+
static int
nfp_bpf_ctrl_entry_op(struct bpf_offloaded_map *offmap, enum nfp_ccm_type op,
u8 *key, u8 *value, u64 flags, u8 *out_key, u8 *out_value)
{
struct nfp_bpf_map *nfp_map = offmap->dev_priv;
+ unsigned int n_entries, reply_entries, count;
struct nfp_app_bpf *bpf = nfp_map->bpf;
struct bpf_map *map = &offmap->map;
struct cmsg_reply_map_op *reply;
struct cmsg_req_map_op *req;
struct sk_buff *skb;
+ u32 cache_gen;
int err;
/* FW messages have no space for more than 32 bits of flags */
if (flags >> 32)
return -EOPNOTSUPP;
+ /* Handle op cache */
+ n_entries = nfp_bpf_ctrl_op_cache_get(nfp_map, op, key, out_key,
+ out_value, &cache_gen);
+ if (!n_entries)
+ return 0;
+
skb = nfp_bpf_cmsg_map_req_alloc(bpf, 1);
- if (!skb)
- return -ENOMEM;
+ if (!skb) {
+ err = -ENOMEM;
+ goto err_cache_put;
+ }
req = (void *)skb->data;
req->tid = cpu_to_be32(nfp_map->tid);
- req->count = cpu_to_be32(1);
+ req->count = cpu_to_be32(n_entries);
req->flags = cpu_to_be32(flags);
/* Copy inputs */
@@ -207,16 +330,38 @@ nfp_bpf_ctrl_entry_op(struct bpf_offloaded_map *offmap, enum nfp_ccm_type op,
memcpy(nfp_bpf_ctrl_req_val(bpf, req, 0), value,
map->value_size);
- skb = nfp_ccm_communicate(&bpf->ccm, skb, op,
- nfp_bpf_cmsg_map_reply_size(bpf, 1));
- if (IS_ERR(skb))
- return PTR_ERR(skb);
+ skb = nfp_ccm_communicate(&bpf->ccm, skb, op, 0);
+ if (IS_ERR(skb)) {
+ err = PTR_ERR(skb);
+ goto err_cache_put;
+ }
+
+ if (skb->len < sizeof(*reply)) {
+ cmsg_warn(bpf, "cmsg drop - type 0x%02x too short %d!\n",
+ op, skb->len);
+ err = -EIO;
+ goto err_free;
+ }
reply = (void *)skb->data;
+ count = be32_to_cpu(reply->count);
err = nfp_bpf_ctrl_rc_to_errno(bpf, &reply->reply_hdr);
+ /* FW responds with message sized to hold the good entries,
+ * plus one extra entry if there was an error.
+ */
+ reply_entries = count + !!err;
+ if (n_entries > 1 && count)
+ err = 0;
if (err)
goto err_free;
+ if (skb->len != nfp_bpf_cmsg_map_reply_size(bpf, reply_entries)) {
+ cmsg_warn(bpf, "cmsg drop - type 0x%02x too short %d for %d entries!\n",
+ op, skb->len, reply_entries);
+ err = -EIO;
+ goto err_free;
+ }
+
/* Copy outputs */
if (out_key)
memcpy(out_key, nfp_bpf_ctrl_reply_key(bpf, reply, 0),
@@ -225,11 +370,13 @@ nfp_bpf_ctrl_entry_op(struct bpf_offloaded_map *offmap, enum nfp_ccm_type op,
memcpy(out_value, nfp_bpf_ctrl_reply_val(bpf, reply, 0),
map->value_size);
- dev_consume_skb_any(skb);
+ nfp_bpf_ctrl_op_cache_put(nfp_map, op, skb, cache_gen);
return 0;
err_free:
dev_kfree_skb_any(skb);
+err_cache_put:
+ nfp_bpf_ctrl_op_cache_put(nfp_map, op, NULL, cache_gen);
return err;
}
@@ -267,11 +414,29 @@ int nfp_bpf_ctrl_getnext_entry(struct bpf_offloaded_map *offmap,
key, NULL, 0, next_key, NULL);
}
+unsigned int nfp_bpf_ctrl_cmsg_min_mtu(struct nfp_app_bpf *bpf)
+{
+ return max(nfp_bpf_cmsg_map_req_size(bpf, 1),
+ nfp_bpf_cmsg_map_reply_size(bpf, 1));
+}
+
unsigned int nfp_bpf_ctrl_cmsg_mtu(struct nfp_app_bpf *bpf)
{
- return max3((unsigned int)NFP_NET_DEFAULT_MTU,
- nfp_bpf_cmsg_map_req_size(bpf, 1),
- nfp_bpf_cmsg_map_reply_size(bpf, 1));
+ return max3(NFP_NET_DEFAULT_MTU,
+ nfp_bpf_cmsg_map_req_size(bpf, NFP_BPF_MAP_CACHE_CNT),
+ nfp_bpf_cmsg_map_reply_size(bpf, NFP_BPF_MAP_CACHE_CNT));
+}
+
+unsigned int nfp_bpf_ctrl_cmsg_cache_cnt(struct nfp_app_bpf *bpf)
+{
+ unsigned int mtu, req_max, reply_max, entry_sz;
+
+ mtu = bpf->app->ctrl->dp.mtu;
+ entry_sz = bpf->cmsg_key_sz + bpf->cmsg_val_sz;
+ req_max = (mtu - sizeof(struct cmsg_req_map_op)) / entry_sz;
+ reply_max = (mtu - sizeof(struct cmsg_reply_map_op)) / entry_sz;
+
+ return min3(req_max, reply_max, NFP_BPF_MAP_CACHE_CNT);
}
void nfp_bpf_ctrl_msg_rx(struct nfp_app *app, struct sk_buff *skb)
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/fw.h b/drivers/net/ethernet/netronome/nfp/bpf/fw.h
index 06c4286b..a83a0ad 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/fw.h
+++ b/drivers/net/ethernet/netronome/nfp/bpf/fw.h
@@ -24,6 +24,7 @@ enum bpf_cap_tlv_type {
NFP_BPF_CAP_TYPE_QUEUE_SELECT = 5,
NFP_BPF_CAP_TYPE_ADJUST_TAIL = 6,
NFP_BPF_CAP_TYPE_ABI_VERSION = 7,
+ NFP_BPF_CAP_TYPE_CMSG_MULTI_ENT = 8,
};
struct nfp_bpf_cap_tlv_func {
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c
index 4054b70..5afcb3c 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c
+++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c
@@ -1163,7 +1163,7 @@ mem_op_stack(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta,
bool clr_gpr, lmem_step step)
{
s32 off = nfp_prog->stack_frame_depth + meta->insn.off + ptr_off;
- bool first = true, last;
+ bool first = true, narrow_ld, last;
bool needs_inc = false;
swreg stack_off_reg;
u8 prev_gpr = 255;
@@ -1209,13 +1209,22 @@ mem_op_stack(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta,
needs_inc = true;
}
+
+ narrow_ld = clr_gpr && size < 8;
+
if (lm3) {
+ unsigned int nop_cnt;
+
emit_csr_wr(nfp_prog, imm_b(nfp_prog), NFP_CSR_ACT_LM_ADDR3);
- /* For size < 4 one slot will be filled by zeroing of upper. */
- wrp_nops(nfp_prog, clr_gpr && size < 8 ? 2 : 3);
+ /* For size < 4 one slot will be filled by zeroing of upper,
+ * but be careful, that zeroing could be eliminated by zext
+ * optimization.
+ */
+ nop_cnt = narrow_ld && meta->flags & FLAG_INSN_DO_ZEXT ? 2 : 3;
+ wrp_nops(nfp_prog, nop_cnt);
}
- if (clr_gpr && size < 8)
+ if (narrow_ld)
wrp_zext(nfp_prog, meta, gpr);
while (size) {
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.c b/drivers/net/ethernet/netronome/nfp/bpf/main.c
index 1c9fb11..8f73277 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/main.c
+++ b/drivers/net/ethernet/netronome/nfp/bpf/main.c
@@ -300,6 +300,14 @@ nfp_bpf_parse_cap_adjust_tail(struct nfp_app_bpf *bpf, void __iomem *value,
}
static int
+nfp_bpf_parse_cap_cmsg_multi_ent(struct nfp_app_bpf *bpf, void __iomem *value,
+ u32 length)
+{
+ bpf->cmsg_multi_ent = true;
+ return 0;
+}
+
+static int
nfp_bpf_parse_cap_abi_version(struct nfp_app_bpf *bpf, void __iomem *value,
u32 length)
{
@@ -375,6 +383,11 @@ static int nfp_bpf_parse_capabilities(struct nfp_app *app)
length))
goto err_release_free;
break;
+ case NFP_BPF_CAP_TYPE_CMSG_MULTI_ENT:
+ if (nfp_bpf_parse_cap_cmsg_multi_ent(app->priv, value,
+ length))
+ goto err_release_free;
+ break;
default:
nfp_dbg(cpp, "unknown BPF capability: %d\n", type);
break;
@@ -415,6 +428,25 @@ static void nfp_bpf_ndo_uninit(struct nfp_app *app, struct net_device *netdev)
bpf_offload_dev_netdev_unregister(bpf->bpf_dev, netdev);
}
+static int nfp_bpf_start(struct nfp_app *app)
+{
+ struct nfp_app_bpf *bpf = app->priv;
+
+ if (app->ctrl->dp.mtu < nfp_bpf_ctrl_cmsg_min_mtu(bpf)) {
+ nfp_err(bpf->app->cpp,
+ "ctrl channel MTU below min required %u < %u\n",
+ app->ctrl->dp.mtu, nfp_bpf_ctrl_cmsg_min_mtu(bpf));
+ return -EINVAL;
+ }
+
+ if (bpf->cmsg_multi_ent)
+ bpf->cmsg_cache_cnt = nfp_bpf_ctrl_cmsg_cache_cnt(bpf);
+ else
+ bpf->cmsg_cache_cnt = 1;
+
+ return 0;
+}
+
static int nfp_bpf_init(struct nfp_app *app)
{
struct nfp_app_bpf *bpf;
@@ -488,6 +520,7 @@ const struct nfp_app_type app_bpf = {
.init = nfp_bpf_init,
.clean = nfp_bpf_clean,
+ .start = nfp_bpf_start,
.check_mtu = nfp_bpf_check_mtu,
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/main.h b/drivers/net/ethernet/netronome/nfp/bpf/main.h
index 57d6ff5..fac9c6f 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/main.h
+++ b/drivers/net/ethernet/netronome/nfp/bpf/main.h
@@ -99,6 +99,7 @@ enum pkt_vec {
* @maps_neutral: hash table of offload-neutral maps (on pointer)
*
* @abi_version: global BPF ABI version
+ * @cmsg_cache_cnt: number of entries to read for caching
*
* @adjust_head: adjust head capability
* @adjust_head.flags: extra flags for adjust head
@@ -124,6 +125,7 @@ enum pkt_vec {
* @pseudo_random: FW initialized the pseudo-random machinery (CSRs)
* @queue_select: BPF can set the RX queue ID in packet vector
* @adjust_tail: BPF can simply trunc packet size for adjust tail
+ * @cmsg_multi_ent: FW can pack multiple map entries in a single cmsg
*/
struct nfp_app_bpf {
struct nfp_app *app;
@@ -134,6 +136,8 @@ struct nfp_app_bpf {
unsigned int cmsg_key_sz;
unsigned int cmsg_val_sz;
+ unsigned int cmsg_cache_cnt;
+
struct list_head map_list;
unsigned int maps_in_use;
unsigned int map_elems_in_use;
@@ -169,6 +173,7 @@ struct nfp_app_bpf {
bool pseudo_random;
bool queue_select;
bool adjust_tail;
+ bool cmsg_multi_ent;
};
enum nfp_bpf_map_use {
@@ -183,11 +188,21 @@ struct nfp_bpf_map_word {
unsigned char non_zero_update :1;
};
+#define NFP_BPF_MAP_CACHE_CNT 4U
+#define NFP_BPF_MAP_CACHE_TIME_NS (250 * 1000)
+
/**
* struct nfp_bpf_map - private per-map data attached to BPF maps for offload
* @offmap: pointer to the offloaded BPF map
* @bpf: back pointer to bpf app private structure
* @tid: table id identifying map on datapath
+ *
+ * @cache_lock: protects @cache_blockers, @cache_to, @cache
+ * @cache_blockers: number of ops in flight which block caching
+ * @cache_gen: counter incremented by every blocker on exit
+ * @cache_to: time when cache will no longer be valid (ns)
+ * @cache: skb with cached response
+ *
* @l: link on the nfp_app_bpf->map_list list
* @use_map: map of how the value is used (in 4B chunks)
*/
@@ -195,6 +210,13 @@ struct nfp_bpf_map {
struct bpf_offloaded_map *offmap;
struct nfp_app_bpf *bpf;
u32 tid;
+
+ spinlock_t cache_lock;
+ u32 cache_blockers;
+ u32 cache_gen;
+ u64 cache_to;
+ struct sk_buff *cache;
+
struct list_head l;
struct nfp_bpf_map_word use_map[];
};
@@ -564,7 +586,9 @@ nfp_bpf_goto_meta(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta,
void *nfp_bpf_relo_for_vnic(struct nfp_prog *nfp_prog, struct nfp_bpf_vnic *bv);
+unsigned int nfp_bpf_ctrl_cmsg_min_mtu(struct nfp_app_bpf *bpf);
unsigned int nfp_bpf_ctrl_cmsg_mtu(struct nfp_app_bpf *bpf);
+unsigned int nfp_bpf_ctrl_cmsg_cache_cnt(struct nfp_app_bpf *bpf);
long long int
nfp_bpf_ctrl_alloc_map(struct nfp_app_bpf *bpf, struct bpf_map *map);
void
diff --git a/drivers/net/ethernet/netronome/nfp/bpf/offload.c b/drivers/net/ethernet/netronome/nfp/bpf/offload.c
index 39c9fec222..88fab6a 100644
--- a/drivers/net/ethernet/netronome/nfp/bpf/offload.c
+++ b/drivers/net/ethernet/netronome/nfp/bpf/offload.c
@@ -385,6 +385,7 @@ nfp_bpf_map_alloc(struct nfp_app_bpf *bpf, struct bpf_offloaded_map *offmap)
offmap->dev_priv = nfp_map;
nfp_map->offmap = offmap;
nfp_map->bpf = bpf;
+ spin_lock_init(&nfp_map->cache_lock);
res = nfp_bpf_ctrl_alloc_map(bpf, &offmap->map);
if (res < 0) {
@@ -407,6 +408,8 @@ nfp_bpf_map_free(struct nfp_app_bpf *bpf, struct bpf_offloaded_map *offmap)
struct nfp_bpf_map *nfp_map = offmap->dev_priv;
nfp_bpf_ctrl_free_map(bpf, nfp_map);
+ dev_consume_skb_any(nfp_map->cache);
+ WARN_ON_ONCE(nfp_map->cache_blockers);
list_del_init(&nfp_map->l);
bpf->map_elems_in_use -= offmap->map.max_entries;
bpf->maps_in_use--;
diff --git a/drivers/net/ethernet/netronome/nfp/devlink_param.c b/drivers/net/ethernet/netronome/nfp/devlink_param.c
new file mode 100644
index 0000000..4a8141b
--- /dev/null
+++ b/drivers/net/ethernet/netronome/nfp/devlink_param.c
@@ -0,0 +1,254 @@
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+/* Copyright (C) 2019 Netronome Systems, Inc. */
+
+#include <net/devlink.h>
+
+#include "nfpcore/nfp.h"
+#include "nfpcore/nfp_nsp.h"
+#include "nfp_main.h"
+
+/**
+ * struct nfp_devlink_param_u8_arg - Devlink u8 parameter get/set arguments
+ * @hwinfo_name: HWinfo key name
+ * @default_hi_val: Default HWinfo value if HWinfo doesn't exist
+ * @invalid_dl_val: Devlink value to use if HWinfo is unknown/invalid.
+ * -errno if there is no unknown/invalid value available
+ * @hi_to_dl: HWinfo to devlink value mapping
+ * @dl_to_hi: Devlink to hwinfo value mapping
+ * @max_dl_val: Maximum devlink value supported, for validation only
+ * @max_hi_val: Maximum HWinfo value supported, for validation only
+ */
+struct nfp_devlink_param_u8_arg {
+ const char *hwinfo_name;
+ const char *default_hi_val;
+ int invalid_dl_val;
+ u8 hi_to_dl[4];
+ u8 dl_to_hi[4];
+ u8 max_dl_val;
+ u8 max_hi_val;
+};
+
+static const struct nfp_devlink_param_u8_arg nfp_devlink_u8_args[] = {
+ [DEVLINK_PARAM_GENERIC_ID_FW_LOAD_POLICY] = {
+ .hwinfo_name = "app_fw_from_flash",
+ .default_hi_val = NFP_NSP_APP_FW_LOAD_DEFAULT,
+ .invalid_dl_val = -EINVAL,
+ .hi_to_dl = {
+ [NFP_NSP_APP_FW_LOAD_DISK] =
+ DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK,
+ [NFP_NSP_APP_FW_LOAD_FLASH] =
+ DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH,
+ [NFP_NSP_APP_FW_LOAD_PREF] =
+ DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER,
+ },
+ .dl_to_hi = {
+ [DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER] =
+ NFP_NSP_APP_FW_LOAD_PREF,
+ [DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH] =
+ NFP_NSP_APP_FW_LOAD_FLASH,
+ [DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK] =
+ NFP_NSP_APP_FW_LOAD_DISK,
+ },
+ .max_dl_val = DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK,
+ .max_hi_val = NFP_NSP_APP_FW_LOAD_PREF,
+ },
+ [DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE] = {
+ .hwinfo_name = "abi_drv_reset",
+ .default_hi_val = NFP_NSP_DRV_RESET_DEFAULT,
+ .invalid_dl_val =
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN,
+ .hi_to_dl = {
+ [NFP_NSP_DRV_RESET_ALWAYS] =
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS,
+ [NFP_NSP_DRV_RESET_NEVER] =
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER,
+ [NFP_NSP_DRV_RESET_DISK] =
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK,
+ },
+ .dl_to_hi = {
+ [DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS] =
+ NFP_NSP_DRV_RESET_ALWAYS,
+ [DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER] =
+ NFP_NSP_DRV_RESET_NEVER,
+ [DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK] =
+ NFP_NSP_DRV_RESET_DISK,
+ },
+ .max_dl_val = DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK,
+ .max_hi_val = NFP_NSP_DRV_RESET_NEVER,
+ }
+};
+
+static int
+nfp_devlink_param_u8_get(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ const struct nfp_devlink_param_u8_arg *arg;
+ struct nfp_pf *pf = devlink_priv(devlink);
+ struct nfp_nsp *nsp;
+ char hwinfo[32];
+ long value;
+ int err;
+
+ if (id >= ARRAY_SIZE(nfp_devlink_u8_args))
+ return -EOPNOTSUPP;
+
+ arg = &nfp_devlink_u8_args[id];
+
+ nsp = nfp_nsp_open(pf->cpp);
+ if (IS_ERR(nsp)) {
+ err = PTR_ERR(nsp);
+ nfp_warn(pf->cpp, "can't access NSP: %d\n", err);
+ return err;
+ }
+
+ snprintf(hwinfo, sizeof(hwinfo), arg->hwinfo_name);
+ err = nfp_nsp_hwinfo_lookup_optional(nsp, hwinfo, sizeof(hwinfo),
+ arg->default_hi_val);
+ if (err) {
+ nfp_warn(pf->cpp, "HWinfo lookup failed: %d\n", err);
+ goto exit_close_nsp;
+ }
+
+ err = kstrtol(hwinfo, 0, &value);
+ if (err || value < 0 || value > arg->max_hi_val) {
+ nfp_warn(pf->cpp, "HWinfo '%s' value %li invalid\n",
+ arg->hwinfo_name, value);
+
+ if (arg->invalid_dl_val >= 0)
+ ctx->val.vu8 = arg->invalid_dl_val;
+ else
+ err = arg->invalid_dl_val;
+
+ goto exit_close_nsp;
+ }
+
+ ctx->val.vu8 = arg->hi_to_dl[value];
+
+exit_close_nsp:
+ nfp_nsp_close(nsp);
+ return err;
+}
+
+static int
+nfp_devlink_param_u8_set(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ const struct nfp_devlink_param_u8_arg *arg;
+ struct nfp_pf *pf = devlink_priv(devlink);
+ struct nfp_nsp *nsp;
+ char hwinfo[32];
+ int err;
+
+ if (id >= ARRAY_SIZE(nfp_devlink_u8_args))
+ return -EOPNOTSUPP;
+
+ arg = &nfp_devlink_u8_args[id];
+
+ nsp = nfp_nsp_open(pf->cpp);
+ if (IS_ERR(nsp)) {
+ err = PTR_ERR(nsp);
+ nfp_warn(pf->cpp, "can't access NSP: %d\n", err);
+ return err;
+ }
+
+ /* Note the value has already been validated. */
+ snprintf(hwinfo, sizeof(hwinfo), "%s=%u",
+ arg->hwinfo_name, arg->dl_to_hi[ctx->val.vu8]);
+ err = nfp_nsp_hwinfo_set(nsp, hwinfo, sizeof(hwinfo));
+ if (err) {
+ nfp_warn(pf->cpp, "HWinfo set failed: %d\n", err);
+ goto exit_close_nsp;
+ }
+
+exit_close_nsp:
+ nfp_nsp_close(nsp);
+ return err;
+}
+
+static int
+nfp_devlink_param_u8_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ const struct nfp_devlink_param_u8_arg *arg;
+
+ if (id >= ARRAY_SIZE(nfp_devlink_u8_args))
+ return -EOPNOTSUPP;
+
+ arg = &nfp_devlink_u8_args[id];
+
+ if (val.vu8 > arg->max_dl_val) {
+ NL_SET_ERR_MSG_MOD(extack, "parameter out of range");
+ return -EINVAL;
+ }
+
+ if (val.vu8 == arg->invalid_dl_val) {
+ NL_SET_ERR_MSG_MOD(extack, "unknown/invalid value specified");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct devlink_param nfp_devlink_params[] = {
+ DEVLINK_PARAM_GENERIC(FW_LOAD_POLICY,
+ BIT(DEVLINK_PARAM_CMODE_PERMANENT),
+ nfp_devlink_param_u8_get,
+ nfp_devlink_param_u8_set,
+ nfp_devlink_param_u8_validate),
+ DEVLINK_PARAM_GENERIC(RESET_DEV_ON_DRV_PROBE,
+ BIT(DEVLINK_PARAM_CMODE_PERMANENT),
+ nfp_devlink_param_u8_get,
+ nfp_devlink_param_u8_set,
+ nfp_devlink_param_u8_validate),
+};
+
+static int nfp_devlink_supports_params(struct nfp_pf *pf)
+{
+ struct nfp_nsp *nsp;
+ bool supported;
+ int err;
+
+ nsp = nfp_nsp_open(pf->cpp);
+ if (IS_ERR(nsp)) {
+ err = PTR_ERR(nsp);
+ dev_err(&pf->pdev->dev, "Failed to access the NSP: %d\n", err);
+ return err;
+ }
+
+ supported = nfp_nsp_has_hwinfo_lookup(nsp) &&
+ nfp_nsp_has_hwinfo_set(nsp);
+
+ nfp_nsp_close(nsp);
+ return supported;
+}
+
+int nfp_devlink_params_register(struct nfp_pf *pf)
+{
+ struct devlink *devlink = priv_to_devlink(pf);
+ int err;
+
+ err = nfp_devlink_supports_params(pf);
+ if (err <= 0)
+ return err;
+
+ err = devlink_params_register(devlink, nfp_devlink_params,
+ ARRAY_SIZE(nfp_devlink_params));
+ if (err)
+ return err;
+
+ devlink_params_publish(devlink);
+ return 0;
+}
+
+void nfp_devlink_params_unregister(struct nfp_pf *pf)
+{
+ int err;
+
+ err = nfp_devlink_supports_params(pf);
+ if (err <= 0)
+ return;
+
+ devlink_params_unregister(priv_to_devlink(pf), nfp_devlink_params,
+ ARRAY_SIZE(nfp_devlink_params));
+}
diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c
index 3a4f4f0..987ae22 100644
--- a/drivers/net/ethernet/netronome/nfp/flower/offload.c
+++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c
@@ -1579,13 +1579,21 @@ nfp_flower_setup_indr_tc_block(struct net_device *netdev, struct nfp_app *app,
struct nfp_flower_priv *priv = app->priv;
struct flow_block_cb *block_cb;
- if (f->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS &&
- !(f->binder_type == FLOW_BLOCK_BINDER_TYPE_CLSACT_EGRESS &&
- nfp_flower_internal_port_can_offload(app, netdev)))
+ if ((f->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS &&
+ !nfp_flower_internal_port_can_offload(app, netdev)) ||
+ (f->binder_type != FLOW_BLOCK_BINDER_TYPE_CLSACT_EGRESS &&
+ nfp_flower_internal_port_can_offload(app, netdev)))
return -EOPNOTSUPP;
switch (f->command) {
case FLOW_BLOCK_BIND:
+ cb_priv = nfp_flower_indr_block_cb_priv_lookup(app, netdev);
+ if (cb_priv &&
+ flow_block_cb_is_busy(nfp_flower_setup_indr_block_cb,
+ cb_priv,
+ &nfp_block_cb_list))
+ return -EBUSY;
+
cb_priv = kmalloc(sizeof(*cb_priv), GFP_KERNEL);
if (!cb_priv)
return -ENOMEM;
diff --git a/drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c b/drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c
index def8c19..2600ce4 100644
--- a/drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c
+++ b/drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c
@@ -348,13 +348,13 @@ nfp_tun_neigh_event_handler(struct notifier_block *nb, unsigned long event,
flow.daddr = *(__be32 *)n->primary_key;
- /* Only concerned with route changes for representors. */
- if (!nfp_netdev_is_nfp_repr(n->dev))
- return NOTIFY_DONE;
-
app_priv = container_of(nb, struct nfp_flower_priv, tun.neigh_nb);
app = app_priv->app;
+ if (!nfp_netdev_is_nfp_repr(n->dev) &&
+ !nfp_flower_internal_port_can_offload(app, n->dev))
+ return NOTIFY_DONE;
+
/* Only concerned with changes to routes already added to NFP. */
if (!nfp_tun_has_route(app, flow.daddr))
return NOTIFY_DONE;
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.c b/drivers/net/ethernet/netronome/nfp/nfp_main.c
index 60e57f0..4d282fc5 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_main.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_main.c
@@ -352,7 +352,7 @@ nfp_net_fw_request(struct pci_dev *pdev, struct nfp_pf *pf, const char *name)
err = request_firmware_direct(&fw, name, &pdev->dev);
nfp_info(pf->cpp, " %s: %s\n",
- name, err ? "not found" : "found, loading...");
+ name, err ? "not found" : "found");
if (err)
return NULL;
@@ -430,8 +430,35 @@ nfp_net_fw_find(struct pci_dev *pdev, struct nfp_pf *pf)
return nfp_net_fw_request(pdev, pf, fw_name);
}
+static int
+nfp_get_fw_policy_value(struct pci_dev *pdev, struct nfp_nsp *nsp,
+ const char *key, const char *default_val, int max_val,
+ int *value)
+{
+ char hwinfo[64];
+ long hi_val;
+ int err;
+
+ snprintf(hwinfo, sizeof(hwinfo), key);
+ err = nfp_nsp_hwinfo_lookup_optional(nsp, hwinfo, sizeof(hwinfo),
+ default_val);
+ if (err)
+ return err;
+
+ err = kstrtol(hwinfo, 0, &hi_val);
+ if (err || hi_val < 0 || hi_val > max_val) {
+ dev_warn(&pdev->dev,
+ "Invalid value '%s' from '%s', ignoring\n",
+ hwinfo, key);
+ err = kstrtol(default_val, 0, &hi_val);
+ }
+
+ *value = hi_val;
+ return err;
+}
+
/**
- * nfp_net_fw_load() - Load the firmware image
+ * nfp_fw_load() - Load the firmware image
* @pdev: PCI Device structure
* @pf: NFP PF Device structure
* @nsp: NFP SP handle
@@ -441,44 +468,107 @@ nfp_net_fw_find(struct pci_dev *pdev, struct nfp_pf *pf)
static int
nfp_fw_load(struct pci_dev *pdev, struct nfp_pf *pf, struct nfp_nsp *nsp)
{
- const struct firmware *fw;
+ bool do_reset, fw_loaded = false;
+ const struct firmware *fw = NULL;
+ int err, reset, policy, ifcs = 0;
+ char *token, *ptr;
+ char hwinfo[64];
u16 interface;
- int err;
+
+ snprintf(hwinfo, sizeof(hwinfo), "abi_drv_load_ifc");
+ err = nfp_nsp_hwinfo_lookup_optional(nsp, hwinfo, sizeof(hwinfo),
+ NFP_NSP_DRV_LOAD_IFC_DEFAULT);
+ if (err)
+ return err;
interface = nfp_cpp_interface(pf->cpp);
- if (NFP_CPP_INTERFACE_UNIT_of(interface) != 0) {
- /* Only Unit 0 should reset or load firmware */
+ ptr = hwinfo;
+ while ((token = strsep(&ptr, ","))) {
+ unsigned long interface_hi;
+
+ err = kstrtoul(token, 0, &interface_hi);
+ if (err) {
+ dev_err(&pdev->dev,
+ "Failed to parse interface '%s': %d\n",
+ token, err);
+ return err;
+ }
+
+ ifcs++;
+ if (interface == interface_hi)
+ break;
+ }
+
+ if (!token) {
dev_info(&pdev->dev, "Firmware will be loaded by partner\n");
return 0;
}
+ err = nfp_get_fw_policy_value(pdev, nsp, "abi_drv_reset",
+ NFP_NSP_DRV_RESET_DEFAULT,
+ NFP_NSP_DRV_RESET_NEVER, &reset);
+ if (err)
+ return err;
+
+ err = nfp_get_fw_policy_value(pdev, nsp, "app_fw_from_flash",
+ NFP_NSP_APP_FW_LOAD_DEFAULT,
+ NFP_NSP_APP_FW_LOAD_PREF, &policy);
+ if (err)
+ return err;
+
fw = nfp_net_fw_find(pdev, pf);
- if (!fw) {
- if (nfp_nsp_has_stored_fw_load(nsp))
- nfp_nsp_load_stored_fw(nsp);
- return 0;
+ do_reset = reset == NFP_NSP_DRV_RESET_ALWAYS ||
+ (fw && reset == NFP_NSP_DRV_RESET_DISK);
+
+ if (do_reset) {
+ dev_info(&pdev->dev, "Soft-resetting the NFP\n");
+ err = nfp_nsp_device_soft_reset(nsp);
+ if (err < 0) {
+ dev_err(&pdev->dev,
+ "Failed to soft reset the NFP: %d\n", err);
+ goto exit_release_fw;
+ }
}
- dev_info(&pdev->dev, "Soft-reset, loading FW image\n");
- err = nfp_nsp_device_soft_reset(nsp);
- if (err < 0) {
- dev_err(&pdev->dev, "Failed to soft reset the NFP: %d\n",
- err);
- goto exit_release_fw;
- }
+ if (fw && policy != NFP_NSP_APP_FW_LOAD_FLASH) {
+ if (nfp_nsp_has_fw_loaded(nsp) && nfp_nsp_fw_loaded(nsp))
+ goto exit_release_fw;
- err = nfp_nsp_load_fw(nsp, fw);
- if (err < 0) {
- dev_err(&pdev->dev, "FW loading failed: %d\n", err);
- goto exit_release_fw;
- }
+ err = nfp_nsp_load_fw(nsp, fw);
+ if (err < 0) {
+ dev_err(&pdev->dev, "FW loading failed: %d\n",
+ err);
+ goto exit_release_fw;
+ }
+ dev_info(&pdev->dev, "Finished loading FW image\n");
+ fw_loaded = true;
+ } else if (policy != NFP_NSP_APP_FW_LOAD_DISK &&
+ nfp_nsp_has_stored_fw_load(nsp)) {
- dev_info(&pdev->dev, "Finished loading FW image\n");
+ /* Don't propagate this error to stick with legacy driver
+ * behavior, failure will be detected later during init.
+ */
+ if (!nfp_nsp_load_stored_fw(nsp))
+ dev_info(&pdev->dev, "Finished loading stored FW image\n");
+
+ /* Don't flag the fw_loaded in this case since other devices
+ * may reuse the firmware when configured this way
+ */
+ } else {
+ dev_warn(&pdev->dev, "Didn't load firmware, please update flash or reconfigure card\n");
+ }
exit_release_fw:
release_firmware(fw);
- return err < 0 ? err : 1;
+ /* We don't want to unload firmware when other devices may still be
+ * dependent on it, which could be the case if there are multiple
+ * devices that could load firmware.
+ */
+ if (fw_loaded && ifcs == 1)
+ pf->unload_fw_on_remove = true;
+
+ return err < 0 ? err : fw_loaded;
}
static void
@@ -704,7 +794,7 @@ static int nfp_pci_probe(struct pci_dev *pdev,
err_fw_unload:
kfree(pf->rtbl);
nfp_mip_close(pf->mip);
- if (pf->fw_loaded)
+ if (pf->unload_fw_on_remove)
nfp_fw_unload(pf);
kfree(pf->eth_tbl);
kfree(pf->nspi);
@@ -744,7 +834,7 @@ static void __nfp_pci_shutdown(struct pci_dev *pdev, bool unload_fw)
vfree(pf->dumpspec);
kfree(pf->rtbl);
nfp_mip_close(pf->mip);
- if (unload_fw && pf->fw_loaded)
+ if (unload_fw && pf->unload_fw_on_remove)
nfp_fw_unload(pf);
destroy_workqueue(pf->wq);
@@ -815,6 +905,8 @@ static void __exit nfp_main_exit(void)
module_init(nfp_main_init);
module_exit(nfp_main_exit);
+MODULE_FIRMWARE("netronome/nic_AMDA0058-0011_2x40.nffw");
+MODULE_FIRMWARE("netronome/nic_AMDA0058-0012_2x40.nffw");
MODULE_FIRMWARE("netronome/nic_AMDA0081-0001_1x40.nffw");
MODULE_FIRMWARE("netronome/nic_AMDA0081-0001_4x10.nffw");
MODULE_FIRMWARE("netronome/nic_AMDA0096-0001_2x10.nffw");
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.h b/drivers/net/ethernet/netronome/nfp/nfp_main.h
index b7211f2..5d5812f 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_main.h
+++ b/drivers/net/ethernet/netronome/nfp/nfp_main.h
@@ -64,6 +64,7 @@ struct nfp_dumpspec {
* @limit_vfs: Number of VFs supported by firmware (~0 for PCI limit)
* @num_vfs: Number of SR-IOV VFs enabled
* @fw_loaded: Is the firmware loaded?
+ * @unload_fw_on_remove:Do we need to unload firmware on driver removal?
* @ctrl_vnic: Pointer to the control vNIC if available
* @mip: MIP handle
* @rtbl: RTsym table
@@ -110,6 +111,7 @@ struct nfp_pf {
unsigned int num_vfs;
bool fw_loaded;
+ bool unload_fw_on_remove;
struct nfp_net *ctrl_vnic;
@@ -185,4 +187,7 @@ int nfp_shared_buf_pool_get(struct nfp_pf *pf, unsigned int sb, u16 pool_index,
int nfp_shared_buf_pool_set(struct nfp_pf *pf, unsigned int sb,
u16 pool_index, u32 size,
enum devlink_sb_threshold_type threshold_type);
+
+int nfp_devlink_params_register(struct nfp_pf *pf);
+void nfp_devlink_params_unregister(struct nfp_pf *pf);
#endif /* NFP_MAIN_H */
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h
index 5d6c373..250f510 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net.h
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h
@@ -66,7 +66,7 @@
#define NFP_NET_MAX_DMA_BITS 40
/* Default size for MTU and freelist buffer sizes */
-#define NFP_NET_DEFAULT_MTU 1500
+#define NFP_NET_DEFAULT_MTU 1500U
/* Maximum number of bytes prepended to a packet */
#define NFP_NET_MAX_PREPEND 64
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
index 6f97b55..61aabff 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
@@ -4116,14 +4116,7 @@ int nfp_net_init(struct nfp_net *nn)
/* Set default MTU and Freelist buffer size */
if (!nfp_net_is_data_vnic(nn) && nn->app->ctrl_mtu) {
- if (nn->app->ctrl_mtu <= nn->max_mtu) {
- nn->dp.mtu = nn->app->ctrl_mtu;
- } else {
- if (nn->app->ctrl_mtu != NFP_APP_CTRL_MTU_MAX)
- nn_warn(nn, "app requested MTU above max supported %u > %u\n",
- nn->app->ctrl_mtu, nn->max_mtu);
- nn->dp.mtu = nn->max_mtu;
- }
+ nn->dp.mtu = min(nn->app->ctrl_mtu, nn->max_mtu);
} else if (nn->max_mtu < NFP_NET_DEFAULT_MTU) {
nn->dp.mtu = nn->max_mtu;
} else {
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_main.c b/drivers/net/ethernet/netronome/nfp/nfp_net_main.c
index 986464d..921db40 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_main.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_main.c
@@ -205,10 +205,8 @@ nfp_net_pf_alloc_vnics(struct nfp_pf *pf, void __iomem *ctrl_bar,
ctrl_bar += NFP_PF_CSR_SLICE_SIZE;
/* Kill the vNIC if app init marked it as invalid */
- if (nn->port && nn->port->type == NFP_PORT_INVALID) {
+ if (nn->port && nn->port->type == NFP_PORT_INVALID)
nfp_net_pf_free_vnic(pf, nn);
- continue;
- }
}
if (list_empty(&pf->vnics))
@@ -711,6 +709,10 @@ int nfp_net_pci_probe(struct nfp_pf *pf)
if (err)
goto err_devlink_unreg;
+ err = nfp_devlink_params_register(pf);
+ if (err)
+ goto err_shared_buf_unreg;
+
mutex_lock(&pf->lock);
pf->ddir = nfp_net_debugfs_device_add(pf->pdev);
@@ -744,6 +746,8 @@ int nfp_net_pci_probe(struct nfp_pf *pf)
err_clean_ddir:
nfp_net_debugfs_dir_clean(&pf->ddir);
mutex_unlock(&pf->lock);
+ nfp_devlink_params_unregister(pf);
+err_shared_buf_unreg:
nfp_shared_buf_unregister(pf);
err_devlink_unreg:
cancel_work_sync(&pf->port_refresh_work);
@@ -773,6 +777,7 @@ void nfp_net_pci_remove(struct nfp_pf *pf)
mutex_unlock(&pf->lock);
+ nfp_devlink_params_unregister(pf);
nfp_shared_buf_unregister(pf);
devlink_unregister(priv_to_devlink(pf));
diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.c
index 9a08623..f18e787 100644
--- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.c
+++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.c
@@ -96,6 +96,8 @@ enum nfp_nsp_cmd {
SPCODE_NSP_IDENTIFY = 13, /* Read NSP version */
SPCODE_FW_STORED = 16, /* If no FW loaded, load flash app FW */
SPCODE_HWINFO_LOOKUP = 17, /* Lookup HWinfo with overwrites etc. */
+ SPCODE_HWINFO_SET = 18, /* Set HWinfo entry */
+ SPCODE_FW_LOADED = 19, /* Is application firmware loaded */
SPCODE_VERSIONS = 21, /* Report FW versions */
SPCODE_READ_SFF_EEPROM = 22, /* Read module EEPROM */
};
@@ -143,6 +145,8 @@ struct nfp_nsp {
* @option: NFP SP Command Argument
* @buf: NFP SP Buffer Address
* @error_cb: Callback for interpreting option if error occurred
+ * @error_quiet:Don't print command error/warning. Protocol errors are still
+ * logged.
*/
struct nfp_nsp_command_arg {
u16 code;
@@ -151,6 +155,7 @@ struct nfp_nsp_command_arg {
u32 option;
u64 buf;
void (*error_cb)(struct nfp_nsp *state, u32 ret_val);
+ bool error_quiet;
};
/**
@@ -405,8 +410,10 @@ __nfp_nsp_command(struct nfp_nsp *state, const struct nfp_nsp_command_arg *arg)
err = FIELD_GET(NSP_STATUS_RESULT, reg);
if (err) {
- nfp_warn(cpp, "Result (error) code set: %d (%d) command: %d\n",
- -err, (int)ret_val, arg->code);
+ if (!arg->error_quiet)
+ nfp_warn(cpp, "Result (error) code set: %d (%d) command: %d\n",
+ -err, (int)ret_val, arg->code);
+
if (arg->error_cb)
arg->error_cb(state, ret_val);
else
@@ -891,12 +898,14 @@ int nfp_nsp_load_stored_fw(struct nfp_nsp *state)
}
static int
-__nfp_nsp_hwinfo_lookup(struct nfp_nsp *state, void *buf, unsigned int size)
+__nfp_nsp_hwinfo_lookup(struct nfp_nsp *state, void *buf, unsigned int size,
+ bool optional)
{
struct nfp_nsp_command_buf_arg hwinfo_lookup = {
{
.code = SPCODE_HWINFO_LOOKUP,
.option = size,
+ .error_quiet = optional,
},
.in_buf = buf,
.in_size = size,
@@ -913,7 +922,7 @@ int nfp_nsp_hwinfo_lookup(struct nfp_nsp *state, void *buf, unsigned int size)
size = min_t(u32, size, NFP_HWINFO_LOOKUP_SIZE);
- err = __nfp_nsp_hwinfo_lookup(state, buf, size);
+ err = __nfp_nsp_hwinfo_lookup(state, buf, size, false);
if (err)
return err;
@@ -925,6 +934,66 @@ int nfp_nsp_hwinfo_lookup(struct nfp_nsp *state, void *buf, unsigned int size)
return 0;
}
+int nfp_nsp_hwinfo_lookup_optional(struct nfp_nsp *state, void *buf,
+ unsigned int size, const char *default_val)
+{
+ int err;
+
+ /* Ensure that the default value is usable irrespective of whether
+ * it is actually going to be used.
+ */
+ if (strnlen(default_val, size) == size)
+ return -EINVAL;
+
+ if (!nfp_nsp_has_hwinfo_lookup(state)) {
+ strcpy(buf, default_val);
+ return 0;
+ }
+
+ size = min_t(u32, size, NFP_HWINFO_LOOKUP_SIZE);
+
+ err = __nfp_nsp_hwinfo_lookup(state, buf, size, true);
+ if (err) {
+ if (err == -ENOENT) {
+ strcpy(buf, default_val);
+ return 0;
+ }
+
+ nfp_err(state->cpp, "NSP HWinfo lookup failed: %d\n", err);
+ return err;
+ }
+
+ if (strnlen(buf, size) == size) {
+ nfp_err(state->cpp, "NSP HWinfo value not NULL-terminated\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int nfp_nsp_hwinfo_set(struct nfp_nsp *state, void *buf, unsigned int size)
+{
+ struct nfp_nsp_command_buf_arg hwinfo_set = {
+ {
+ .code = SPCODE_HWINFO_SET,
+ .option = size,
+ },
+ .in_buf = buf,
+ .in_size = size,
+ };
+
+ return nfp_nsp_command_buf(state, &hwinfo_set);
+}
+
+int nfp_nsp_fw_loaded(struct nfp_nsp *state)
+{
+ const struct nfp_nsp_command_arg arg = {
+ .code = SPCODE_FW_LOADED,
+ };
+
+ return __nfp_nsp_command(state, &arg);
+}
+
int nfp_nsp_versions(struct nfp_nsp *state, void *buf, unsigned int size)
{
struct nfp_nsp_command_buf_arg versions = {
diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h
index 22ee6985e..1531c18 100644
--- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h
+++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_nsp.h
@@ -22,6 +22,10 @@ int nfp_nsp_write_flash(struct nfp_nsp *state, const struct firmware *fw);
int nfp_nsp_mac_reinit(struct nfp_nsp *state);
int nfp_nsp_load_stored_fw(struct nfp_nsp *state);
int nfp_nsp_hwinfo_lookup(struct nfp_nsp *state, void *buf, unsigned int size);
+int nfp_nsp_hwinfo_lookup_optional(struct nfp_nsp *state, void *buf,
+ unsigned int size, const char *default_val);
+int nfp_nsp_hwinfo_set(struct nfp_nsp *state, void *buf, unsigned int size);
+int nfp_nsp_fw_loaded(struct nfp_nsp *state);
int nfp_nsp_read_module_eeprom(struct nfp_nsp *state, int eth_index,
unsigned int offset, void *data,
unsigned int len, unsigned int *read_len);
@@ -41,6 +45,16 @@ static inline bool nfp_nsp_has_hwinfo_lookup(struct nfp_nsp *state)
return nfp_nsp_get_abi_ver_minor(state) > 24;
}
+static inline bool nfp_nsp_has_hwinfo_set(struct nfp_nsp *state)
+{
+ return nfp_nsp_get_abi_ver_minor(state) > 25;
+}
+
+static inline bool nfp_nsp_has_fw_loaded(struct nfp_nsp *state)
+{
+ return nfp_nsp_get_abi_ver_minor(state) > 25;
+}
+
static inline bool nfp_nsp_has_versions(struct nfp_nsp *state)
{
return nfp_nsp_get_abi_ver_minor(state) > 27;
@@ -88,6 +102,21 @@ enum nfp_eth_fec {
#define NFP_FEC_REED_SOLOMON BIT(NFP_FEC_REED_SOLOMON_BIT)
#define NFP_FEC_DISABLED BIT(NFP_FEC_DISABLED_BIT)
+/* Defines the valid values of the 'abi_drv_reset' hwinfo key */
+#define NFP_NSP_DRV_RESET_DISK 0
+#define NFP_NSP_DRV_RESET_ALWAYS 1
+#define NFP_NSP_DRV_RESET_NEVER 2
+#define NFP_NSP_DRV_RESET_DEFAULT "0"
+
+/* Defines the valid values of the 'app_fw_from_flash' hwinfo key */
+#define NFP_NSP_APP_FW_LOAD_DISK 0
+#define NFP_NSP_APP_FW_LOAD_FLASH 1
+#define NFP_NSP_APP_FW_LOAD_PREF 2
+#define NFP_NSP_APP_FW_LOAD_DEFAULT "2"
+
+/* Define the default value for the 'abi_drv_load_ifc' key */
+#define NFP_NSP_DRV_LOAD_IFC_DEFAULT "0x10ff"
+
/**
* struct nfp_eth_table - ETH table information
* @count: number of table entries
diff --git a/drivers/net/ethernet/packetengines/yellowfin.c b/drivers/net/ethernet/packetengines/yellowfin.c
index 6f8d658..5113ee6 100644
--- a/drivers/net/ethernet/packetengines/yellowfin.c
+++ b/drivers/net/ethernet/packetengines/yellowfin.c
@@ -1258,8 +1258,7 @@ static int yellowfin_close(struct net_device *dev)
yp->rx_skbuff[i] = NULL;
}
for (i = 0; i < TX_RING_SIZE; i++) {
- if (yp->tx_skbuff[i])
- dev_kfree_skb(yp->tx_skbuff[i]);
+ dev_kfree_skb(yp->tx_skbuff[i]);
yp->tx_skbuff[i] = NULL;
}
diff --git a/drivers/net/ethernet/pensando/Kconfig b/drivers/net/ethernet/pensando/Kconfig
new file mode 100644
index 0000000..5ea570b
--- /dev/null
+++ b/drivers/net/ethernet/pensando/Kconfig
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Pensando Systems, Inc
+#
+# Pensando device configuration
+#
+
+config NET_VENDOR_PENSANDO
+ bool "Pensando devices"
+ default y
+ help
+ If you have a network (Ethernet) card belonging to this class, say Y.
+
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Pensando cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_PENSANDO
+
+config IONIC
+ tristate "Pensando Ethernet IONIC Support"
+ depends on 64BIT && PCI
+ help
+ This enables the support for the Pensando family of Ethernet
+ adapters. More specific information on this driver can be
+ found in
+ <file:Documentation/networking/device_drivers/pensando/ionic.rst>.
+
+ To compile this driver as a module, choose M here. The module
+ will be called ionic.
+
+endif # NET_VENDOR_PENSANDO
diff --git a/drivers/net/ethernet/pensando/Makefile b/drivers/net/ethernet/pensando/Makefile
new file mode 100644
index 0000000..21ce749
--- /dev/null
+++ b/drivers/net/ethernet/pensando/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Pensando network device drivers.
+#
+
+obj-$(CONFIG_IONIC) += ionic/
diff --git a/drivers/net/ethernet/pensando/ionic/Makefile b/drivers/net/ethernet/pensando/ionic/Makefile
new file mode 100644
index 0000000..29f304d
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright(c) 2017 - 2019 Pensando Systems, Inc
+
+obj-$(CONFIG_IONIC) := ionic.o
+
+ionic-y := ionic_main.o ionic_bus_pci.o ionic_devlink.o ionic_dev.o \
+ ionic_debugfs.o ionic_lif.o ionic_rx_filter.o ionic_ethtool.o \
+ ionic_txrx.o ionic_stats.o
diff --git a/drivers/net/ethernet/pensando/ionic/ionic.h b/drivers/net/ethernet/pensando/ionic/ionic.h
new file mode 100644
index 0000000..7a70606
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_H_
+#define _IONIC_H_
+
+struct ionic_lif;
+
+#include "ionic_if.h"
+#include "ionic_dev.h"
+#include "ionic_devlink.h"
+
+#define IONIC_DRV_NAME "ionic"
+#define IONIC_DRV_DESCRIPTION "Pensando Ethernet NIC Driver"
+#define IONIC_DRV_VERSION "0.15.0-k"
+
+#define PCI_VENDOR_ID_PENSANDO 0x1dd8
+
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_PF 0x1002
+#define PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF 0x1003
+
+#define IONIC_SUBDEV_ID_NAPLES_25 0x4000
+#define IONIC_SUBDEV_ID_NAPLES_100_4 0x4001
+#define IONIC_SUBDEV_ID_NAPLES_100_8 0x4002
+
+#define DEVCMD_TIMEOUT 10
+
+struct ionic {
+ struct pci_dev *pdev;
+ struct device *dev;
+ struct devlink_port dl_port;
+ struct ionic_dev idev;
+ struct mutex dev_cmd_lock; /* lock for dev_cmd operations */
+ struct dentry *dentry;
+ struct ionic_dev_bar bars[IONIC_BARS_MAX];
+ unsigned int num_bars;
+ struct ionic_identity ident;
+ struct list_head lifs;
+ struct ionic_lif *master_lif;
+ unsigned int nnqs_per_lif;
+ unsigned int neqs_per_lif;
+ unsigned int ntxqs_per_lif;
+ unsigned int nrxqs_per_lif;
+ DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX);
+ unsigned int nintrs;
+ DECLARE_BITMAP(intrs, IONIC_INTR_CTRL_REGS_MAX);
+ struct work_struct nb_work;
+ struct notifier_block nb;
+};
+
+struct ionic_admin_ctx {
+ struct completion work;
+ union ionic_adminq_cmd cmd;
+ union ionic_adminq_comp comp;
+};
+
+int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
+ ionic_cq_done_cb done_cb, void *done_arg);
+
+int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx);
+int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_wait);
+int ionic_set_dma_mask(struct ionic *ionic);
+int ionic_setup(struct ionic *ionic);
+
+int ionic_identify(struct ionic *ionic);
+int ionic_init(struct ionic *ionic);
+int ionic_reset(struct ionic *ionic);
+
+int ionic_port_identify(struct ionic *ionic);
+int ionic_port_init(struct ionic *ionic);
+int ionic_port_reset(struct ionic *ionic);
+
+#endif /* _IONIC_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_bus.h b/drivers/net/ethernet/pensando/ionic/ionic_bus.h
new file mode 100644
index 0000000..2f4d08c
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_bus.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_BUS_H_
+#define _IONIC_BUS_H_
+
+int ionic_bus_get_irq(struct ionic *ionic, unsigned int num);
+const char *ionic_bus_info(struct ionic *ionic);
+int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs);
+void ionic_bus_free_irq_vectors(struct ionic *ionic);
+int ionic_bus_register_driver(void);
+void ionic_bus_unregister_driver(void);
+void __iomem *ionic_bus_map_dbpage(struct ionic *ionic, int page_num);
+void ionic_bus_unmap_dbpage(struct ionic *ionic, void __iomem *page);
+
+#endif /* _IONIC_BUS_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
new file mode 100644
index 0000000..9a9ab8c
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/pci.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_debugfs.h"
+
+/* Supported devices */
+static const struct pci_device_id ionic_id_table[] = {
+ { PCI_VDEVICE(PENSANDO, PCI_DEVICE_ID_PENSANDO_IONIC_ETH_PF) },
+ { PCI_VDEVICE(PENSANDO, PCI_DEVICE_ID_PENSANDO_IONIC_ETH_VF) },
+ { 0, } /* end of table */
+};
+MODULE_DEVICE_TABLE(pci, ionic_id_table);
+
+int ionic_bus_get_irq(struct ionic *ionic, unsigned int num)
+{
+ return pci_irq_vector(ionic->pdev, num);
+}
+
+const char *ionic_bus_info(struct ionic *ionic)
+{
+ return pci_name(ionic->pdev);
+}
+
+int ionic_bus_alloc_irq_vectors(struct ionic *ionic, unsigned int nintrs)
+{
+ return pci_alloc_irq_vectors(ionic->pdev, nintrs, nintrs,
+ PCI_IRQ_MSIX);
+}
+
+void ionic_bus_free_irq_vectors(struct ionic *ionic)
+{
+ pci_free_irq_vectors(ionic->pdev);
+}
+
+static int ionic_map_bars(struct ionic *ionic)
+{
+ struct pci_dev *pdev = ionic->pdev;
+ struct device *dev = ionic->dev;
+ struct ionic_dev_bar *bars;
+ unsigned int i, j;
+
+ bars = ionic->bars;
+ ionic->num_bars = 0;
+
+ for (i = 0, j = 0; i < IONIC_BARS_MAX; i++) {
+ if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM))
+ continue;
+ bars[j].len = pci_resource_len(pdev, i);
+
+ /* only map the whole bar 0 */
+ if (j > 0) {
+ bars[j].vaddr = NULL;
+ } else {
+ bars[j].vaddr = pci_iomap(pdev, i, bars[j].len);
+ if (!bars[j].vaddr) {
+ dev_err(dev,
+ "Cannot memory-map BAR %d, aborting\n",
+ i);
+ return -ENODEV;
+ }
+ }
+
+ bars[j].bus_addr = pci_resource_start(pdev, i);
+ bars[j].res_index = i;
+ ionic->num_bars++;
+ j++;
+ }
+
+ return 0;
+}
+
+static void ionic_unmap_bars(struct ionic *ionic)
+{
+ struct ionic_dev_bar *bars = ionic->bars;
+ unsigned int i;
+
+ for (i = 0; i < IONIC_BARS_MAX; i++) {
+ if (bars[i].vaddr) {
+ iounmap(bars[i].vaddr);
+ bars[i].bus_addr = 0;
+ bars[i].vaddr = NULL;
+ bars[i].len = 0;
+ }
+ }
+}
+
+void __iomem *ionic_bus_map_dbpage(struct ionic *ionic, int page_num)
+{
+ return pci_iomap_range(ionic->pdev,
+ ionic->bars[IONIC_PCI_BAR_DBELL].res_index,
+ (u64)page_num << PAGE_SHIFT, PAGE_SIZE);
+}
+
+void ionic_bus_unmap_dbpage(struct ionic *ionic, void __iomem *page)
+{
+ iounmap(page);
+}
+
+static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ struct device *dev = &pdev->dev;
+ struct ionic *ionic;
+ int err;
+
+ ionic = ionic_devlink_alloc(dev);
+ if (!ionic)
+ return -ENOMEM;
+
+ ionic->pdev = pdev;
+ ionic->dev = dev;
+ pci_set_drvdata(pdev, ionic);
+ mutex_init(&ionic->dev_cmd_lock);
+
+ /* Query system for DMA addressing limitation for the device. */
+ err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(IONIC_ADDR_LEN));
+ if (err) {
+ dev_err(dev, "Unable to obtain 64-bit DMA for consistent allocations, aborting. err=%d\n",
+ err);
+ goto err_out_clear_drvdata;
+ }
+
+ ionic_debugfs_add_dev(ionic);
+
+ /* Setup PCI device */
+ err = pci_enable_device_mem(pdev);
+ if (err) {
+ dev_err(dev, "Cannot enable PCI device: %d, aborting\n", err);
+ goto err_out_debugfs_del_dev;
+ }
+
+ err = pci_request_regions(pdev, IONIC_DRV_NAME);
+ if (err) {
+ dev_err(dev, "Cannot request PCI regions: %d, aborting\n", err);
+ goto err_out_pci_disable_device;
+ }
+
+ pci_set_master(pdev);
+
+ err = ionic_map_bars(ionic);
+ if (err)
+ goto err_out_pci_clear_master;
+
+ /* Configure the device */
+ err = ionic_setup(ionic);
+ if (err) {
+ dev_err(dev, "Cannot setup device: %d, aborting\n", err);
+ goto err_out_unmap_bars;
+ }
+
+ err = ionic_identify(ionic);
+ if (err) {
+ dev_err(dev, "Cannot identify device: %d, aborting\n", err);
+ goto err_out_teardown;
+ }
+
+ err = ionic_init(ionic);
+ if (err) {
+ dev_err(dev, "Cannot init device: %d, aborting\n", err);
+ goto err_out_teardown;
+ }
+
+ /* Configure the ports */
+ err = ionic_port_identify(ionic);
+ if (err) {
+ dev_err(dev, "Cannot identify port: %d, aborting\n", err);
+ goto err_out_reset;
+ }
+
+ err = ionic_port_init(ionic);
+ if (err) {
+ dev_err(dev, "Cannot init port: %d, aborting\n", err);
+ goto err_out_reset;
+ }
+
+ /* Configure LIFs */
+ err = ionic_lif_identify(ionic, IONIC_LIF_TYPE_CLASSIC,
+ &ionic->ident.lif);
+ if (err) {
+ dev_err(dev, "Cannot identify LIFs: %d, aborting\n", err);
+ goto err_out_port_reset;
+ }
+
+ err = ionic_lifs_size(ionic);
+ if (err) {
+ dev_err(dev, "Cannot size LIFs: %d, aborting\n", err);
+ goto err_out_port_reset;
+ }
+
+ err = ionic_lifs_alloc(ionic);
+ if (err) {
+ dev_err(dev, "Cannot allocate LIFs: %d, aborting\n", err);
+ goto err_out_free_irqs;
+ }
+
+ err = ionic_lifs_init(ionic);
+ if (err) {
+ dev_err(dev, "Cannot init LIFs: %d, aborting\n", err);
+ goto err_out_free_lifs;
+ }
+
+ err = ionic_lifs_register(ionic);
+ if (err) {
+ dev_err(dev, "Cannot register LIFs: %d, aborting\n", err);
+ goto err_out_deinit_lifs;
+ }
+
+ err = ionic_devlink_register(ionic);
+ if (err) {
+ dev_err(dev, "Cannot register devlink: %d\n", err);
+ goto err_out_deregister_lifs;
+ }
+
+ return 0;
+
+err_out_deregister_lifs:
+ ionic_lifs_unregister(ionic);
+err_out_deinit_lifs:
+ ionic_lifs_deinit(ionic);
+err_out_free_lifs:
+ ionic_lifs_free(ionic);
+err_out_free_irqs:
+ ionic_bus_free_irq_vectors(ionic);
+err_out_port_reset:
+ ionic_port_reset(ionic);
+err_out_reset:
+ ionic_reset(ionic);
+err_out_teardown:
+ ionic_dev_teardown(ionic);
+err_out_unmap_bars:
+ ionic_unmap_bars(ionic);
+ pci_release_regions(pdev);
+err_out_pci_clear_master:
+ pci_clear_master(pdev);
+err_out_pci_disable_device:
+ pci_disable_device(pdev);
+err_out_debugfs_del_dev:
+ ionic_debugfs_del_dev(ionic);
+err_out_clear_drvdata:
+ mutex_destroy(&ionic->dev_cmd_lock);
+ ionic_devlink_free(ionic);
+
+ return err;
+}
+
+static void ionic_remove(struct pci_dev *pdev)
+{
+ struct ionic *ionic = pci_get_drvdata(pdev);
+
+ if (!ionic)
+ return;
+
+ ionic_devlink_unregister(ionic);
+ ionic_lifs_unregister(ionic);
+ ionic_lifs_deinit(ionic);
+ ionic_lifs_free(ionic);
+ ionic_bus_free_irq_vectors(ionic);
+ ionic_port_reset(ionic);
+ ionic_reset(ionic);
+ ionic_dev_teardown(ionic);
+ ionic_unmap_bars(ionic);
+ pci_release_regions(pdev);
+ pci_clear_master(pdev);
+ pci_disable_device(pdev);
+ ionic_debugfs_del_dev(ionic);
+ mutex_destroy(&ionic->dev_cmd_lock);
+ ionic_devlink_free(ionic);
+}
+
+static struct pci_driver ionic_driver = {
+ .name = IONIC_DRV_NAME,
+ .id_table = ionic_id_table,
+ .probe = ionic_probe,
+ .remove = ionic_remove,
+};
+
+int ionic_bus_register_driver(void)
+{
+ return pci_register_driver(&ionic_driver);
+}
+
+void ionic_bus_unregister_driver(void)
+{
+ pci_unregister_driver(&ionic_driver);
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c
new file mode 100644
index 0000000..7afc4a3
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c
@@ -0,0 +1,248 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/pci.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_debugfs.h"
+
+#ifdef CONFIG_DEBUG_FS
+
+static struct dentry *ionic_dir;
+
+void ionic_debugfs_create(void)
+{
+ ionic_dir = debugfs_create_dir(IONIC_DRV_NAME, NULL);
+}
+
+void ionic_debugfs_destroy(void)
+{
+ debugfs_remove_recursive(ionic_dir);
+}
+
+void ionic_debugfs_add_dev(struct ionic *ionic)
+{
+ ionic->dentry = debugfs_create_dir(ionic_bus_info(ionic), ionic_dir);
+}
+
+void ionic_debugfs_del_dev(struct ionic *ionic)
+{
+ debugfs_remove_recursive(ionic->dentry);
+ ionic->dentry = NULL;
+}
+
+static int identity_show(struct seq_file *seq, void *v)
+{
+ struct ionic *ionic = seq->private;
+ struct ionic_identity *ident;
+
+ ident = &ionic->ident;
+
+ seq_printf(seq, "nlifs: %d\n", ident->dev.nlifs);
+ seq_printf(seq, "nintrs: %d\n", ident->dev.nintrs);
+ seq_printf(seq, "ndbpgs_per_lif: %d\n", ident->dev.ndbpgs_per_lif);
+ seq_printf(seq, "intr_coal_mult: %d\n", ident->dev.intr_coal_mult);
+ seq_printf(seq, "intr_coal_div: %d\n", ident->dev.intr_coal_div);
+
+ seq_printf(seq, "max_ucast_filters: %d\n", ident->lif.eth.max_ucast_filters);
+ seq_printf(seq, "max_mcast_filters: %d\n", ident->lif.eth.max_mcast_filters);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(identity);
+
+void ionic_debugfs_add_ident(struct ionic *ionic)
+{
+ debugfs_create_file("identity", 0400, ionic->dentry,
+ ionic, &identity_fops) ? 0 : -EOPNOTSUPP;
+}
+
+void ionic_debugfs_add_sizes(struct ionic *ionic)
+{
+ debugfs_create_u32("nlifs", 0400, ionic->dentry,
+ (u32 *)&ionic->ident.dev.nlifs);
+ debugfs_create_u32("nintrs", 0400, ionic->dentry, &ionic->nintrs);
+
+ debugfs_create_u32("ntxqs_per_lif", 0400, ionic->dentry,
+ (u32 *)&ionic->ident.lif.eth.config.queue_count[IONIC_QTYPE_TXQ]);
+ debugfs_create_u32("nrxqs_per_lif", 0400, ionic->dentry,
+ (u32 *)&ionic->ident.lif.eth.config.queue_count[IONIC_QTYPE_RXQ]);
+}
+
+static int q_tail_show(struct seq_file *seq, void *v)
+{
+ struct ionic_queue *q = seq->private;
+
+ seq_printf(seq, "%d\n", q->tail->index);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(q_tail);
+
+static int q_head_show(struct seq_file *seq, void *v)
+{
+ struct ionic_queue *q = seq->private;
+
+ seq_printf(seq, "%d\n", q->head->index);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(q_head);
+
+static int cq_tail_show(struct seq_file *seq, void *v)
+{
+ struct ionic_cq *cq = seq->private;
+
+ seq_printf(seq, "%d\n", cq->tail->index);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(cq_tail);
+
+static const struct debugfs_reg32 intr_ctrl_regs[] = {
+ { .name = "coal_init", .offset = 0, },
+ { .name = "mask", .offset = 4, },
+ { .name = "credits", .offset = 8, },
+ { .name = "mask_on_assert", .offset = 12, },
+ { .name = "coal_timer", .offset = 16, },
+};
+
+void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+ struct dentry *q_dentry, *cq_dentry, *intr_dentry, *stats_dentry;
+ struct ionic_dev *idev = &lif->ionic->idev;
+ struct debugfs_regset32 *intr_ctrl_regset;
+ struct ionic_intr_info *intr = &qcq->intr;
+ struct debugfs_blob_wrapper *desc_blob;
+ struct device *dev = lif->ionic->dev;
+ struct ionic_queue *q = &qcq->q;
+ struct ionic_cq *cq = &qcq->cq;
+
+ qcq->dentry = debugfs_create_dir(q->name, lif->dentry);
+
+ debugfs_create_x32("total_size", 0400, qcq->dentry, &qcq->total_size);
+ debugfs_create_x64("base_pa", 0400, qcq->dentry, &qcq->base_pa);
+
+ q_dentry = debugfs_create_dir("q", qcq->dentry);
+
+ debugfs_create_u32("index", 0400, q_dentry, &q->index);
+ debugfs_create_x64("base_pa", 0400, q_dentry, &q->base_pa);
+ if (qcq->flags & IONIC_QCQ_F_SG) {
+ debugfs_create_x64("sg_base_pa", 0400, q_dentry,
+ &q->sg_base_pa);
+ debugfs_create_u32("sg_desc_size", 0400, q_dentry,
+ &q->sg_desc_size);
+ }
+ debugfs_create_u32("num_descs", 0400, q_dentry, &q->num_descs);
+ debugfs_create_u32("desc_size", 0400, q_dentry, &q->desc_size);
+ debugfs_create_u32("pid", 0400, q_dentry, &q->pid);
+ debugfs_create_u32("qid", 0400, q_dentry, &q->hw_index);
+ debugfs_create_u32("qtype", 0400, q_dentry, &q->hw_type);
+ debugfs_create_u64("drop", 0400, q_dentry, &q->drop);
+ debugfs_create_u64("stop", 0400, q_dentry, &q->stop);
+ debugfs_create_u64("wake", 0400, q_dentry, &q->wake);
+
+ debugfs_create_file("tail", 0400, q_dentry, q, &q_tail_fops);
+ debugfs_create_file("head", 0400, q_dentry, q, &q_head_fops);
+
+ desc_blob = devm_kzalloc(dev, sizeof(*desc_blob), GFP_KERNEL);
+ if (!desc_blob)
+ return;
+ desc_blob->data = q->base;
+ desc_blob->size = (unsigned long)q->num_descs * q->desc_size;
+ debugfs_create_blob("desc_blob", 0400, q_dentry, desc_blob);
+
+ if (qcq->flags & IONIC_QCQ_F_SG) {
+ desc_blob = devm_kzalloc(dev, sizeof(*desc_blob), GFP_KERNEL);
+ if (!desc_blob)
+ return;
+ desc_blob->data = q->sg_base;
+ desc_blob->size = (unsigned long)q->num_descs * q->sg_desc_size;
+ debugfs_create_blob("sg_desc_blob", 0400, q_dentry,
+ desc_blob);
+ }
+
+ cq_dentry = debugfs_create_dir("cq", qcq->dentry);
+
+ debugfs_create_x64("base_pa", 0400, cq_dentry, &cq->base_pa);
+ debugfs_create_u32("num_descs", 0400, cq_dentry, &cq->num_descs);
+ debugfs_create_u32("desc_size", 0400, cq_dentry, &cq->desc_size);
+ debugfs_create_u8("done_color", 0400, cq_dentry,
+ (u8 *)&cq->done_color);
+
+ debugfs_create_file("tail", 0400, cq_dentry, cq, &cq_tail_fops);
+
+ desc_blob = devm_kzalloc(dev, sizeof(*desc_blob), GFP_KERNEL);
+ if (!desc_blob)
+ return;
+ desc_blob->data = cq->base;
+ desc_blob->size = (unsigned long)cq->num_descs * cq->desc_size;
+ debugfs_create_blob("desc_blob", 0400, cq_dentry, desc_blob);
+
+ if (qcq->flags & IONIC_QCQ_F_INTR) {
+ intr_dentry = debugfs_create_dir("intr", qcq->dentry);
+
+ debugfs_create_u32("index", 0400, intr_dentry,
+ &intr->index);
+ debugfs_create_u32("vector", 0400, intr_dentry,
+ &intr->vector);
+
+ intr_ctrl_regset = devm_kzalloc(dev, sizeof(*intr_ctrl_regset),
+ GFP_KERNEL);
+ if (!intr_ctrl_regset)
+ return;
+ intr_ctrl_regset->regs = intr_ctrl_regs;
+ intr_ctrl_regset->nregs = ARRAY_SIZE(intr_ctrl_regs);
+ intr_ctrl_regset->base = &idev->intr_ctrl[intr->index];
+
+ debugfs_create_regset32("intr_ctrl", 0400, intr_dentry,
+ intr_ctrl_regset);
+ }
+
+ if (qcq->flags & IONIC_QCQ_F_NOTIFYQ) {
+ stats_dentry = debugfs_create_dir("notifyblock", qcq->dentry);
+
+ debugfs_create_u64("eid", 0400, stats_dentry,
+ (u64 *)&lif->info->status.eid);
+ debugfs_create_u16("link_status", 0400, stats_dentry,
+ (u16 *)&lif->info->status.link_status);
+ debugfs_create_u32("link_speed", 0400, stats_dentry,
+ (u32 *)&lif->info->status.link_speed);
+ debugfs_create_u16("link_down_count", 0400, stats_dentry,
+ (u16 *)&lif->info->status.link_down_count);
+ }
+}
+
+static int netdev_show(struct seq_file *seq, void *v)
+{
+ struct net_device *netdev = seq->private;
+
+ seq_printf(seq, "%s\n", netdev->name);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(netdev);
+
+void ionic_debugfs_add_lif(struct ionic_lif *lif)
+{
+ lif->dentry = debugfs_create_dir(lif->name, lif->ionic->dentry);
+ debugfs_create_file("netdev", 0400, lif->dentry,
+ lif->netdev, &netdev_fops);
+}
+
+void ionic_debugfs_del_lif(struct ionic_lif *lif)
+{
+ debugfs_remove_recursive(lif->dentry);
+ lif->dentry = NULL;
+}
+
+void ionic_debugfs_del_qcq(struct ionic_qcq *qcq)
+{
+ debugfs_remove_recursive(qcq->dentry);
+ qcq->dentry = NULL;
+}
+
+#endif
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.h b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.h
new file mode 100644
index 0000000..c44ebde
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEBUGFS_H_
+#define _IONIC_DEBUGFS_H_
+
+#include <linux/debugfs.h>
+
+#ifdef CONFIG_DEBUG_FS
+
+void ionic_debugfs_create(void);
+void ionic_debugfs_destroy(void);
+void ionic_debugfs_add_dev(struct ionic *ionic);
+void ionic_debugfs_del_dev(struct ionic *ionic);
+void ionic_debugfs_add_ident(struct ionic *ionic);
+void ionic_debugfs_add_sizes(struct ionic *ionic);
+void ionic_debugfs_add_lif(struct ionic_lif *lif);
+void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq);
+void ionic_debugfs_del_lif(struct ionic_lif *lif);
+void ionic_debugfs_del_qcq(struct ionic_qcq *qcq);
+#else
+static inline void ionic_debugfs_create(void) { }
+static inline void ionic_debugfs_destroy(void) { }
+static inline void ionic_debugfs_add_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_del_dev(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_ident(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_sizes(struct ionic *ionic) { }
+static inline void ionic_debugfs_add_lif(struct ionic_lif *lif) { }
+static inline void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq) { }
+static inline void ionic_debugfs_del_lif(struct ionic_lif *lif) { }
+static inline void ionic_debugfs_del_qcq(struct ionic_qcq *qcq) { }
+#endif
+
+#endif /* _IONIC_DEBUGFS_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.c b/drivers/net/ethernet/pensando/ionic/ionic_dev.c
new file mode 100644
index 0000000..d168a64
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.c
@@ -0,0 +1,500 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/io.h>
+#include <linux/slab.h>
+#include <linux/etherdevice.h>
+#include "ionic.h"
+#include "ionic_dev.h"
+#include "ionic_lif.h"
+
+void ionic_init_devinfo(struct ionic *ionic)
+{
+ struct ionic_dev *idev = &ionic->idev;
+
+ idev->dev_info.asic_type = ioread8(&idev->dev_info_regs->asic_type);
+ idev->dev_info.asic_rev = ioread8(&idev->dev_info_regs->asic_rev);
+
+ memcpy_fromio(idev->dev_info.fw_version,
+ idev->dev_info_regs->fw_version,
+ IONIC_DEVINFO_FWVERS_BUFLEN);
+
+ memcpy_fromio(idev->dev_info.serial_num,
+ idev->dev_info_regs->serial_num,
+ IONIC_DEVINFO_SERIAL_BUFLEN);
+
+ idev->dev_info.fw_version[IONIC_DEVINFO_FWVERS_BUFLEN] = 0;
+ idev->dev_info.serial_num[IONIC_DEVINFO_SERIAL_BUFLEN] = 0;
+
+ dev_dbg(ionic->dev, "fw_version %s\n", idev->dev_info.fw_version);
+}
+
+int ionic_dev_setup(struct ionic *ionic)
+{
+ struct ionic_dev_bar *bar = ionic->bars;
+ unsigned int num_bars = ionic->num_bars;
+ struct ionic_dev *idev = &ionic->idev;
+ struct device *dev = ionic->dev;
+ u32 sig;
+
+ /* BAR0: dev_cmd and interrupts */
+ if (num_bars < 1) {
+ dev_err(dev, "No bars found, aborting\n");
+ return -EFAULT;
+ }
+
+ if (bar->len < IONIC_BAR0_SIZE) {
+ dev_err(dev, "Resource bar size %lu too small, aborting\n",
+ bar->len);
+ return -EFAULT;
+ }
+
+ idev->dev_info_regs = bar->vaddr + IONIC_BAR0_DEV_INFO_REGS_OFFSET;
+ idev->dev_cmd_regs = bar->vaddr + IONIC_BAR0_DEV_CMD_REGS_OFFSET;
+ idev->intr_status = bar->vaddr + IONIC_BAR0_INTR_STATUS_OFFSET;
+ idev->intr_ctrl = bar->vaddr + IONIC_BAR0_INTR_CTRL_OFFSET;
+
+ sig = ioread32(&idev->dev_info_regs->signature);
+ if (sig != IONIC_DEV_INFO_SIGNATURE) {
+ dev_err(dev, "Incompatible firmware signature %x", sig);
+ return -EFAULT;
+ }
+
+ ionic_init_devinfo(ionic);
+
+ /* BAR1: doorbells */
+ bar++;
+ if (num_bars < 2) {
+ dev_err(dev, "Doorbell bar missing, aborting\n");
+ return -EFAULT;
+ }
+
+ idev->db_pages = bar->vaddr;
+ idev->phy_db_pages = bar->bus_addr;
+
+ return 0;
+}
+
+void ionic_dev_teardown(struct ionic *ionic)
+{
+ /* place holder */
+}
+
+/* Devcmd Interface */
+u8 ionic_dev_cmd_status(struct ionic_dev *idev)
+{
+ return ioread8(&idev->dev_cmd_regs->comp.comp.status);
+}
+
+bool ionic_dev_cmd_done(struct ionic_dev *idev)
+{
+ return ioread32(&idev->dev_cmd_regs->done) & IONIC_DEV_CMD_DONE;
+}
+
+void ionic_dev_cmd_comp(struct ionic_dev *idev, union ionic_dev_cmd_comp *comp)
+{
+ memcpy_fromio(comp, &idev->dev_cmd_regs->comp, sizeof(*comp));
+}
+
+void ionic_dev_cmd_go(struct ionic_dev *idev, union ionic_dev_cmd *cmd)
+{
+ memcpy_toio(&idev->dev_cmd_regs->cmd, cmd, sizeof(*cmd));
+ iowrite32(0, &idev->dev_cmd_regs->done);
+ iowrite32(1, &idev->dev_cmd_regs->doorbell);
+}
+
+/* Device commands */
+void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver)
+{
+ union ionic_dev_cmd cmd = {
+ .identify.opcode = IONIC_CMD_IDENTIFY,
+ .identify.ver = ver,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_init(struct ionic_dev *idev)
+{
+ union ionic_dev_cmd cmd = {
+ .init.opcode = IONIC_CMD_INIT,
+ .init.type = 0,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_reset(struct ionic_dev *idev)
+{
+ union ionic_dev_cmd cmd = {
+ .reset.opcode = IONIC_CMD_RESET,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+/* Port commands */
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev)
+{
+ union ionic_dev_cmd cmd = {
+ .port_init.opcode = IONIC_CMD_PORT_IDENTIFY,
+ .port_init.index = 0,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_init(struct ionic_dev *idev)
+{
+ union ionic_dev_cmd cmd = {
+ .port_init.opcode = IONIC_CMD_PORT_INIT,
+ .port_init.index = 0,
+ .port_init.info_pa = cpu_to_le64(idev->port_info_pa),
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev)
+{
+ union ionic_dev_cmd cmd = {
+ .port_reset.opcode = IONIC_CMD_PORT_RESET,
+ .port_reset.index = 0,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state)
+{
+ union ionic_dev_cmd cmd = {
+ .port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+ .port_setattr.index = 0,
+ .port_setattr.attr = IONIC_PORT_ATTR_STATE,
+ .port_setattr.state = state,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed)
+{
+ union ionic_dev_cmd cmd = {
+ .port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+ .port_setattr.index = 0,
+ .port_setattr.attr = IONIC_PORT_ATTR_SPEED,
+ .port_setattr.speed = cpu_to_le32(speed),
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable)
+{
+ union ionic_dev_cmd cmd = {
+ .port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+ .port_setattr.index = 0,
+ .port_setattr.attr = IONIC_PORT_ATTR_AUTONEG,
+ .port_setattr.an_enable = an_enable,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type)
+{
+ union ionic_dev_cmd cmd = {
+ .port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+ .port_setattr.index = 0,
+ .port_setattr.attr = IONIC_PORT_ATTR_FEC,
+ .port_setattr.fec_type = fec_type,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type)
+{
+ union ionic_dev_cmd cmd = {
+ .port_setattr.opcode = IONIC_CMD_PORT_SETATTR,
+ .port_setattr.index = 0,
+ .port_setattr.attr = IONIC_PORT_ATTR_PAUSE,
+ .port_setattr.pause_type = pause_type,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+/* LIF commands */
+void ionic_dev_cmd_lif_identify(struct ionic_dev *idev, u8 type, u8 ver)
+{
+ union ionic_dev_cmd cmd = {
+ .lif_identify.opcode = IONIC_CMD_LIF_IDENTIFY,
+ .lif_identify.type = type,
+ .lif_identify.ver = ver,
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
+ dma_addr_t info_pa)
+{
+ union ionic_dev_cmd cmd = {
+ .lif_init.opcode = IONIC_CMD_LIF_INIT,
+ .lif_init.index = cpu_to_le16(lif_index),
+ .lif_init.info_pa = cpu_to_le64(info_pa),
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index)
+{
+ union ionic_dev_cmd cmd = {
+ .lif_init.opcode = IONIC_CMD_LIF_RESET,
+ .lif_init.index = cpu_to_le16(lif_index),
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+void ionic_dev_cmd_adminq_init(struct ionic_dev *idev, struct ionic_qcq *qcq,
+ u16 lif_index, u16 intr_index)
+{
+ struct ionic_queue *q = &qcq->q;
+ struct ionic_cq *cq = &qcq->cq;
+
+ union ionic_dev_cmd cmd = {
+ .q_init.opcode = IONIC_CMD_Q_INIT,
+ .q_init.lif_index = cpu_to_le16(lif_index),
+ .q_init.type = q->type,
+ .q_init.index = cpu_to_le32(q->index),
+ .q_init.flags = cpu_to_le16(IONIC_QINIT_F_IRQ |
+ IONIC_QINIT_F_ENA),
+ .q_init.pid = cpu_to_le16(q->pid),
+ .q_init.intr_index = cpu_to_le16(intr_index),
+ .q_init.ring_size = ilog2(q->num_descs),
+ .q_init.ring_base = cpu_to_le64(q->base_pa),
+ .q_init.cq_ring_base = cpu_to_le64(cq->base_pa),
+ };
+
+ ionic_dev_cmd_go(idev, &cmd);
+}
+
+int ionic_db_page_num(struct ionic_lif *lif, int pid)
+{
+ return (lif->hw_index * lif->dbid_count) + pid;
+}
+
+int ionic_cq_init(struct ionic_lif *lif, struct ionic_cq *cq,
+ struct ionic_intr_info *intr,
+ unsigned int num_descs, size_t desc_size)
+{
+ struct ionic_cq_info *cur;
+ unsigned int ring_size;
+ unsigned int i;
+
+ if (desc_size == 0 || !is_power_of_2(num_descs))
+ return -EINVAL;
+
+ ring_size = ilog2(num_descs);
+ if (ring_size < 2 || ring_size > 16)
+ return -EINVAL;
+
+ cq->lif = lif;
+ cq->bound_intr = intr;
+ cq->num_descs = num_descs;
+ cq->desc_size = desc_size;
+ cq->tail = cq->info;
+ cq->done_color = 1;
+
+ cur = cq->info;
+
+ for (i = 0; i < num_descs; i++) {
+ if (i + 1 == num_descs) {
+ cur->next = cq->info;
+ cur->last = true;
+ } else {
+ cur->next = cur + 1;
+ }
+ cur->index = i;
+ cur++;
+ }
+
+ return 0;
+}
+
+void ionic_cq_map(struct ionic_cq *cq, void *base, dma_addr_t base_pa)
+{
+ struct ionic_cq_info *cur;
+ unsigned int i;
+
+ cq->base = base;
+ cq->base_pa = base_pa;
+
+ for (i = 0, cur = cq->info; i < cq->num_descs; i++, cur++)
+ cur->cq_desc = base + (i * cq->desc_size);
+}
+
+void ionic_cq_bind(struct ionic_cq *cq, struct ionic_queue *q)
+{
+ cq->bound_q = q;
+}
+
+unsigned int ionic_cq_service(struct ionic_cq *cq, unsigned int work_to_do,
+ ionic_cq_cb cb, ionic_cq_done_cb done_cb,
+ void *done_arg)
+{
+ unsigned int work_done = 0;
+
+ if (work_to_do == 0)
+ return 0;
+
+ while (cb(cq, cq->tail)) {
+ if (cq->tail->last)
+ cq->done_color = !cq->done_color;
+ cq->tail = cq->tail->next;
+ DEBUG_STATS_CQE_CNT(cq);
+
+ if (++work_done >= work_to_do)
+ break;
+ }
+
+ if (work_done && done_cb)
+ done_cb(done_arg);
+
+ return work_done;
+}
+
+int ionic_q_init(struct ionic_lif *lif, struct ionic_dev *idev,
+ struct ionic_queue *q, unsigned int index, const char *name,
+ unsigned int num_descs, size_t desc_size,
+ size_t sg_desc_size, unsigned int pid)
+{
+ struct ionic_desc_info *cur;
+ unsigned int ring_size;
+ unsigned int i;
+
+ if (desc_size == 0 || !is_power_of_2(num_descs))
+ return -EINVAL;
+
+ ring_size = ilog2(num_descs);
+ if (ring_size < 2 || ring_size > 16)
+ return -EINVAL;
+
+ q->lif = lif;
+ q->idev = idev;
+ q->index = index;
+ q->num_descs = num_descs;
+ q->desc_size = desc_size;
+ q->sg_desc_size = sg_desc_size;
+ q->tail = q->info;
+ q->head = q->tail;
+ q->pid = pid;
+
+ snprintf(q->name, sizeof(q->name), "L%d-%s%u", lif->index, name, index);
+
+ cur = q->info;
+
+ for (i = 0; i < num_descs; i++) {
+ if (i + 1 == num_descs)
+ cur->next = q->info;
+ else
+ cur->next = cur + 1;
+ cur->index = i;
+ cur->left = num_descs - i;
+ cur++;
+ }
+
+ return 0;
+}
+
+void ionic_q_map(struct ionic_queue *q, void *base, dma_addr_t base_pa)
+{
+ struct ionic_desc_info *cur;
+ unsigned int i;
+
+ q->base = base;
+ q->base_pa = base_pa;
+
+ for (i = 0, cur = q->info; i < q->num_descs; i++, cur++)
+ cur->desc = base + (i * q->desc_size);
+}
+
+void ionic_q_sg_map(struct ionic_queue *q, void *base, dma_addr_t base_pa)
+{
+ struct ionic_desc_info *cur;
+ unsigned int i;
+
+ q->sg_base = base;
+ q->sg_base_pa = base_pa;
+
+ for (i = 0, cur = q->info; i < q->num_descs; i++, cur++)
+ cur->sg_desc = base + (i * q->sg_desc_size);
+}
+
+void ionic_q_post(struct ionic_queue *q, bool ring_doorbell, ionic_desc_cb cb,
+ void *cb_arg)
+{
+ struct device *dev = q->lif->ionic->dev;
+ struct ionic_lif *lif = q->lif;
+
+ q->head->cb = cb;
+ q->head->cb_arg = cb_arg;
+ q->head = q->head->next;
+
+ dev_dbg(dev, "lif=%d qname=%s qid=%d qtype=%d p_index=%d ringdb=%d\n",
+ q->lif->index, q->name, q->hw_type, q->hw_index,
+ q->head->index, ring_doorbell);
+
+ if (ring_doorbell)
+ ionic_dbell_ring(lif->kern_dbpage, q->hw_type,
+ q->dbval | q->head->index);
+}
+
+static bool ionic_q_is_posted(struct ionic_queue *q, unsigned int pos)
+{
+ unsigned int mask, tail, head;
+
+ mask = q->num_descs - 1;
+ tail = q->tail->index;
+ head = q->head->index;
+
+ return ((pos - tail) & mask) < ((head - tail) & mask);
+}
+
+void ionic_q_service(struct ionic_queue *q, struct ionic_cq_info *cq_info,
+ unsigned int stop_index)
+{
+ struct ionic_desc_info *desc_info;
+ ionic_desc_cb cb;
+ void *cb_arg;
+
+ /* check for empty queue */
+ if (q->tail->index == q->head->index)
+ return;
+
+ /* stop index must be for a descriptor that is not yet completed */
+ if (unlikely(!ionic_q_is_posted(q, stop_index)))
+ dev_err(q->lif->ionic->dev,
+ "ionic stop is not posted %s stop %u tail %u head %u\n",
+ q->name, stop_index, q->tail->index, q->head->index);
+
+ do {
+ desc_info = q->tail;
+ q->tail = desc_info->next;
+
+ cb = desc_info->cb;
+ cb_arg = desc_info->cb_arg;
+
+ desc_info->cb = NULL;
+ desc_info->cb_arg = NULL;
+
+ if (cb)
+ cb(q, desc_info, cq_info, cb_arg);
+ } while (desc_info->index != stop_index);
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
new file mode 100644
index 0000000..9610aeb
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
@@ -0,0 +1,299 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEV_H_
+#define _IONIC_DEV_H_
+
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
+
+#include "ionic_if.h"
+#include "ionic_regs.h"
+
+#define IONIC_MIN_MTU ETH_MIN_MTU
+#define IONIC_MAX_MTU 9194
+#define IONIC_MAX_TXRX_DESC 16384
+#define IONIC_MIN_TXRX_DESC 16
+#define IONIC_DEF_TXRX_DESC 4096
+#define IONIC_LIFS_MAX 1024
+#define IONIC_ITR_COAL_USEC_DEFAULT 64
+
+#define IONIC_DEV_CMD_REG_VERSION 1
+#define IONIC_DEV_INFO_REG_COUNT 32
+#define IONIC_DEV_CMD_REG_COUNT 32
+
+struct ionic_dev_bar {
+ void __iomem *vaddr;
+ phys_addr_t bus_addr;
+ unsigned long len;
+ int res_index;
+};
+
+/* Registers */
+static_assert(sizeof(struct ionic_intr) == 32);
+
+static_assert(sizeof(struct ionic_doorbell) == 8);
+static_assert(sizeof(struct ionic_intr_status) == 8);
+static_assert(sizeof(union ionic_dev_regs) == 4096);
+static_assert(sizeof(union ionic_dev_info_regs) == 2048);
+static_assert(sizeof(union ionic_dev_cmd_regs) == 2048);
+static_assert(sizeof(struct ionic_lif_stats) == 1024);
+
+static_assert(sizeof(struct ionic_admin_cmd) == 64);
+static_assert(sizeof(struct ionic_admin_comp) == 16);
+static_assert(sizeof(struct ionic_nop_cmd) == 64);
+static_assert(sizeof(struct ionic_nop_comp) == 16);
+
+/* Device commands */
+static_assert(sizeof(struct ionic_dev_identify_cmd) == 64);
+static_assert(sizeof(struct ionic_dev_identify_comp) == 16);
+static_assert(sizeof(struct ionic_dev_init_cmd) == 64);
+static_assert(sizeof(struct ionic_dev_init_comp) == 16);
+static_assert(sizeof(struct ionic_dev_reset_cmd) == 64);
+static_assert(sizeof(struct ionic_dev_reset_comp) == 16);
+static_assert(sizeof(struct ionic_dev_getattr_cmd) == 64);
+static_assert(sizeof(struct ionic_dev_getattr_comp) == 16);
+static_assert(sizeof(struct ionic_dev_setattr_cmd) == 64);
+static_assert(sizeof(struct ionic_dev_setattr_comp) == 16);
+
+/* Port commands */
+static_assert(sizeof(struct ionic_port_identify_cmd) == 64);
+static_assert(sizeof(struct ionic_port_identify_comp) == 16);
+static_assert(sizeof(struct ionic_port_init_cmd) == 64);
+static_assert(sizeof(struct ionic_port_init_comp) == 16);
+static_assert(sizeof(struct ionic_port_reset_cmd) == 64);
+static_assert(sizeof(struct ionic_port_reset_comp) == 16);
+static_assert(sizeof(struct ionic_port_getattr_cmd) == 64);
+static_assert(sizeof(struct ionic_port_getattr_comp) == 16);
+static_assert(sizeof(struct ionic_port_setattr_cmd) == 64);
+static_assert(sizeof(struct ionic_port_setattr_comp) == 16);
+
+/* LIF commands */
+static_assert(sizeof(struct ionic_lif_init_cmd) == 64);
+static_assert(sizeof(struct ionic_lif_init_comp) == 16);
+static_assert(sizeof(struct ionic_lif_reset_cmd) == 64);
+static_assert(sizeof(ionic_lif_reset_comp) == 16);
+static_assert(sizeof(struct ionic_lif_getattr_cmd) == 64);
+static_assert(sizeof(struct ionic_lif_getattr_comp) == 16);
+static_assert(sizeof(struct ionic_lif_setattr_cmd) == 64);
+static_assert(sizeof(struct ionic_lif_setattr_comp) == 16);
+
+static_assert(sizeof(struct ionic_q_init_cmd) == 64);
+static_assert(sizeof(struct ionic_q_init_comp) == 16);
+static_assert(sizeof(struct ionic_q_control_cmd) == 64);
+static_assert(sizeof(ionic_q_control_comp) == 16);
+
+static_assert(sizeof(struct ionic_rx_mode_set_cmd) == 64);
+static_assert(sizeof(ionic_rx_mode_set_comp) == 16);
+static_assert(sizeof(struct ionic_rx_filter_add_cmd) == 64);
+static_assert(sizeof(struct ionic_rx_filter_add_comp) == 16);
+static_assert(sizeof(struct ionic_rx_filter_del_cmd) == 64);
+static_assert(sizeof(ionic_rx_filter_del_comp) == 16);
+
+/* RDMA commands */
+static_assert(sizeof(struct ionic_rdma_reset_cmd) == 64);
+static_assert(sizeof(struct ionic_rdma_queue_cmd) == 64);
+
+/* Events */
+static_assert(sizeof(struct ionic_notifyq_cmd) == 4);
+static_assert(sizeof(union ionic_notifyq_comp) == 64);
+static_assert(sizeof(struct ionic_notifyq_event) == 64);
+static_assert(sizeof(struct ionic_link_change_event) == 64);
+static_assert(sizeof(struct ionic_reset_event) == 64);
+static_assert(sizeof(struct ionic_heartbeat_event) == 64);
+static_assert(sizeof(struct ionic_log_event) == 64);
+
+/* I/O */
+static_assert(sizeof(struct ionic_txq_desc) == 16);
+static_assert(sizeof(struct ionic_txq_sg_desc) == 128);
+static_assert(sizeof(struct ionic_txq_comp) == 16);
+
+static_assert(sizeof(struct ionic_rxq_desc) == 16);
+static_assert(sizeof(struct ionic_rxq_sg_desc) == 128);
+static_assert(sizeof(struct ionic_rxq_comp) == 16);
+
+struct ionic_devinfo {
+ u8 asic_type;
+ u8 asic_rev;
+ char fw_version[IONIC_DEVINFO_FWVERS_BUFLEN + 1];
+ char serial_num[IONIC_DEVINFO_SERIAL_BUFLEN + 1];
+};
+
+struct ionic_dev {
+ union ionic_dev_info_regs __iomem *dev_info_regs;
+ union ionic_dev_cmd_regs __iomem *dev_cmd_regs;
+
+ u64 __iomem *db_pages;
+ dma_addr_t phy_db_pages;
+
+ struct ionic_intr __iomem *intr_ctrl;
+ u64 __iomem *intr_status;
+
+ u32 port_info_sz;
+ struct ionic_port_info *port_info;
+ dma_addr_t port_info_pa;
+
+ struct ionic_devinfo dev_info;
+};
+
+struct ionic_cq_info {
+ void *cq_desc;
+ struct ionic_cq_info *next;
+ unsigned int index;
+ bool last;
+};
+
+struct ionic_queue;
+struct ionic_qcq;
+struct ionic_desc_info;
+
+typedef void (*ionic_desc_cb)(struct ionic_queue *q,
+ struct ionic_desc_info *desc_info,
+ struct ionic_cq_info *cq_info, void *cb_arg);
+
+struct ionic_desc_info {
+ void *desc;
+ void *sg_desc;
+ struct ionic_desc_info *next;
+ unsigned int index;
+ unsigned int left;
+ ionic_desc_cb cb;
+ void *cb_arg;
+};
+
+#define QUEUE_NAME_MAX_SZ 32
+
+struct ionic_queue {
+ u64 dbell_count;
+ u64 drop;
+ u64 stop;
+ u64 wake;
+ struct ionic_lif *lif;
+ struct ionic_desc_info *info;
+ struct ionic_desc_info *tail;
+ struct ionic_desc_info *head;
+ struct ionic_dev *idev;
+ unsigned int index;
+ unsigned int type;
+ unsigned int hw_index;
+ unsigned int hw_type;
+ u64 dbval;
+ void *base;
+ void *sg_base;
+ dma_addr_t base_pa;
+ dma_addr_t sg_base_pa;
+ unsigned int num_descs;
+ unsigned int desc_size;
+ unsigned int sg_desc_size;
+ unsigned int pid;
+ char name[QUEUE_NAME_MAX_SZ];
+};
+
+#define INTR_INDEX_NOT_ASSIGNED -1
+#define INTR_NAME_MAX_SZ 32
+
+struct ionic_intr_info {
+ char name[INTR_NAME_MAX_SZ];
+ unsigned int index;
+ unsigned int vector;
+ u64 rearm_count;
+ unsigned int cpu;
+ cpumask_t affinity_mask;
+};
+
+struct ionic_cq {
+ void *base;
+ dma_addr_t base_pa;
+ struct ionic_lif *lif;
+ struct ionic_cq_info *info;
+ struct ionic_cq_info *tail;
+ struct ionic_queue *bound_q;
+ struct ionic_intr_info *bound_intr;
+ bool done_color;
+ unsigned int num_descs;
+ u64 compl_count;
+ unsigned int desc_size;
+};
+
+struct ionic;
+
+static inline void ionic_intr_init(struct ionic_dev *idev,
+ struct ionic_intr_info *intr,
+ unsigned long index)
+{
+ ionic_intr_clean(idev->intr_ctrl, index);
+ intr->index = index;
+}
+
+static inline unsigned int ionic_q_space_avail(struct ionic_queue *q)
+{
+ unsigned int avail = q->tail->index;
+
+ if (q->head->index >= avail)
+ avail += q->head->left - 1;
+ else
+ avail -= q->head->index + 1;
+
+ return avail;
+}
+
+static inline bool ionic_q_has_space(struct ionic_queue *q, unsigned int want)
+{
+ return ionic_q_space_avail(q) >= want;
+}
+
+void ionic_init_devinfo(struct ionic *ionic);
+int ionic_dev_setup(struct ionic *ionic);
+void ionic_dev_teardown(struct ionic *ionic);
+
+void ionic_dev_cmd_go(struct ionic_dev *idev, union ionic_dev_cmd *cmd);
+u8 ionic_dev_cmd_status(struct ionic_dev *idev);
+bool ionic_dev_cmd_done(struct ionic_dev *idev);
+void ionic_dev_cmd_comp(struct ionic_dev *idev, union ionic_dev_cmd_comp *comp);
+
+void ionic_dev_cmd_identify(struct ionic_dev *idev, u8 ver);
+void ionic_dev_cmd_init(struct ionic_dev *idev);
+void ionic_dev_cmd_reset(struct ionic_dev *idev);
+
+void ionic_dev_cmd_port_identify(struct ionic_dev *idev);
+void ionic_dev_cmd_port_init(struct ionic_dev *idev);
+void ionic_dev_cmd_port_reset(struct ionic_dev *idev);
+void ionic_dev_cmd_port_state(struct ionic_dev *idev, u8 state);
+void ionic_dev_cmd_port_speed(struct ionic_dev *idev, u32 speed);
+void ionic_dev_cmd_port_autoneg(struct ionic_dev *idev, u8 an_enable);
+void ionic_dev_cmd_port_fec(struct ionic_dev *idev, u8 fec_type);
+void ionic_dev_cmd_port_pause(struct ionic_dev *idev, u8 pause_type);
+
+void ionic_dev_cmd_lif_identify(struct ionic_dev *idev, u8 type, u8 ver);
+void ionic_dev_cmd_lif_init(struct ionic_dev *idev, u16 lif_index,
+ dma_addr_t addr);
+void ionic_dev_cmd_lif_reset(struct ionic_dev *idev, u16 lif_index);
+void ionic_dev_cmd_adminq_init(struct ionic_dev *idev, struct ionic_qcq *qcq,
+ u16 lif_index, u16 intr_index);
+
+int ionic_db_page_num(struct ionic_lif *lif, int pid);
+
+int ionic_cq_init(struct ionic_lif *lif, struct ionic_cq *cq,
+ struct ionic_intr_info *intr,
+ unsigned int num_descs, size_t desc_size);
+void ionic_cq_map(struct ionic_cq *cq, void *base, dma_addr_t base_pa);
+void ionic_cq_bind(struct ionic_cq *cq, struct ionic_queue *q);
+typedef bool (*ionic_cq_cb)(struct ionic_cq *cq, struct ionic_cq_info *cq_info);
+typedef void (*ionic_cq_done_cb)(void *done_arg);
+unsigned int ionic_cq_service(struct ionic_cq *cq, unsigned int work_to_do,
+ ionic_cq_cb cb, ionic_cq_done_cb done_cb,
+ void *done_arg);
+
+int ionic_q_init(struct ionic_lif *lif, struct ionic_dev *idev,
+ struct ionic_queue *q, unsigned int index, const char *name,
+ unsigned int num_descs, size_t desc_size,
+ size_t sg_desc_size, unsigned int pid);
+void ionic_q_map(struct ionic_queue *q, void *base, dma_addr_t base_pa);
+void ionic_q_sg_map(struct ionic_queue *q, void *base, dma_addr_t base_pa);
+void ionic_q_post(struct ionic_queue *q, bool ring_doorbell, ionic_desc_cb cb,
+ void *cb_arg);
+void ionic_q_rewind(struct ionic_queue *q, struct ionic_desc_info *start);
+void ionic_q_service(struct ionic_queue *q, struct ionic_cq_info *cq_info,
+ unsigned int stop_index);
+
+#endif /* _IONIC_DEV_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_devlink.c b/drivers/net/ethernet/pensando/ionic/ionic_devlink.c
new file mode 100644
index 0000000..af1647a
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_devlink.c
@@ -0,0 +1,99 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_devlink.h"
+
+static int ionic_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
+ struct netlink_ext_ack *extack)
+{
+ struct ionic *ionic = devlink_priv(dl);
+ struct ionic_dev *idev = &ionic->idev;
+ char buf[16];
+ int err = 0;
+
+ err = devlink_info_driver_name_put(req, IONIC_DRV_NAME);
+ if (err)
+ goto info_out;
+
+ err = devlink_info_version_running_put(req,
+ DEVLINK_INFO_VERSION_GENERIC_FW,
+ idev->dev_info.fw_version);
+ if (err)
+ goto info_out;
+
+ snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_type);
+ err = devlink_info_version_fixed_put(req,
+ DEVLINK_INFO_VERSION_GENERIC_ASIC_ID,
+ buf);
+ if (err)
+ goto info_out;
+
+ snprintf(buf, sizeof(buf), "0x%x", idev->dev_info.asic_rev);
+ err = devlink_info_version_fixed_put(req,
+ DEVLINK_INFO_VERSION_GENERIC_ASIC_REV,
+ buf);
+ if (err)
+ goto info_out;
+
+ err = devlink_info_serial_number_put(req, idev->dev_info.serial_num);
+
+info_out:
+ return err;
+}
+
+static const struct devlink_ops ionic_dl_ops = {
+ .info_get = ionic_dl_info_get,
+};
+
+struct ionic *ionic_devlink_alloc(struct device *dev)
+{
+ struct devlink *dl;
+
+ dl = devlink_alloc(&ionic_dl_ops, sizeof(struct ionic));
+
+ return devlink_priv(dl);
+}
+
+void ionic_devlink_free(struct ionic *ionic)
+{
+ struct devlink *dl = priv_to_devlink(ionic);
+
+ devlink_free(dl);
+}
+
+int ionic_devlink_register(struct ionic *ionic)
+{
+ struct devlink *dl = priv_to_devlink(ionic);
+ int err;
+
+ err = devlink_register(dl, ionic->dev);
+ if (err) {
+ dev_warn(ionic->dev, "devlink_register failed: %d\n", err);
+ return err;
+ }
+
+ devlink_port_attrs_set(&ionic->dl_port, DEVLINK_PORT_FLAVOUR_PHYSICAL,
+ 0, false, 0, NULL, 0);
+ err = devlink_port_register(dl, &ionic->dl_port, 0);
+ if (err)
+ dev_err(ionic->dev, "devlink_port_register failed: %d\n", err);
+ else
+ devlink_port_type_eth_set(&ionic->dl_port,
+ ionic->master_lif->netdev);
+
+ return err;
+}
+
+void ionic_devlink_unregister(struct ionic *ionic)
+{
+ struct devlink *dl = priv_to_devlink(ionic);
+
+ devlink_port_unregister(&ionic->dl_port);
+ devlink_unregister(dl);
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_devlink.h b/drivers/net/ethernet/pensando/ionic/ionic_devlink.h
new file mode 100644
index 0000000..0690172
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_devlink.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_DEVLINK_H_
+#define _IONIC_DEVLINK_H_
+
+#include <net/devlink.h>
+
+struct ionic *ionic_devlink_alloc(struct device *dev);
+void ionic_devlink_free(struct ionic *ionic);
+int ionic_devlink_register(struct ionic *ionic);
+void ionic_devlink_unregister(struct ionic *ionic);
+
+#endif /* _IONIC_DEVLINK_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
new file mode 100644
index 0000000..7d10265
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -0,0 +1,779 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_ethtool.h"
+#include "ionic_stats.h"
+
+static const char ionic_priv_flags_strings[][ETH_GSTRING_LEN] = {
+#define PRIV_F_SW_DBG_STATS BIT(0)
+ "sw-dbg-stats",
+};
+#define PRIV_FLAGS_COUNT ARRAY_SIZE(ionic_priv_flags_strings)
+
+static void ionic_get_stats_strings(struct ionic_lif *lif, u8 *buf)
+{
+ u32 i;
+
+ for (i = 0; i < ionic_num_stats_grps; i++)
+ ionic_stats_groups[i].get_strings(lif, &buf);
+}
+
+static void ionic_get_stats(struct net_device *netdev,
+ struct ethtool_stats *stats, u64 *buf)
+{
+ struct ionic_lif *lif;
+ u32 i;
+
+ lif = netdev_priv(netdev);
+
+ memset(buf, 0, stats->n_stats * sizeof(*buf));
+ for (i = 0; i < ionic_num_stats_grps; i++)
+ ionic_stats_groups[i].get_values(lif, &buf);
+}
+
+static int ionic_get_stats_count(struct ionic_lif *lif)
+{
+ int i, num_stats = 0;
+
+ for (i = 0; i < ionic_num_stats_grps; i++)
+ num_stats += ionic_stats_groups[i].get_count(lif);
+
+ return num_stats;
+}
+
+static int ionic_get_sset_count(struct net_device *netdev, int sset)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ int count = 0;
+
+ switch (sset) {
+ case ETH_SS_STATS:
+ count = ionic_get_stats_count(lif);
+ break;
+ case ETH_SS_PRIV_FLAGS:
+ count = PRIV_FLAGS_COUNT;
+ break;
+ }
+ return count;
+}
+
+static void ionic_get_strings(struct net_device *netdev,
+ u32 sset, u8 *buf)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ switch (sset) {
+ case ETH_SS_STATS:
+ ionic_get_stats_strings(lif, buf);
+ break;
+ case ETH_SS_PRIV_FLAGS:
+ memcpy(buf, ionic_priv_flags_strings,
+ PRIV_FLAGS_COUNT * ETH_GSTRING_LEN);
+ break;
+ }
+}
+
+static void ionic_get_drvinfo(struct net_device *netdev,
+ struct ethtool_drvinfo *drvinfo)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic *ionic = lif->ionic;
+
+ strlcpy(drvinfo->driver, IONIC_DRV_NAME, sizeof(drvinfo->driver));
+ strlcpy(drvinfo->version, IONIC_DRV_VERSION, sizeof(drvinfo->version));
+ strlcpy(drvinfo->fw_version, ionic->idev.dev_info.fw_version,
+ sizeof(drvinfo->fw_version));
+ strlcpy(drvinfo->bus_info, ionic_bus_info(ionic),
+ sizeof(drvinfo->bus_info));
+}
+
+static int ionic_get_regs_len(struct net_device *netdev)
+{
+ return (IONIC_DEV_INFO_REG_COUNT + IONIC_DEV_CMD_REG_COUNT) * sizeof(u32);
+}
+
+static void ionic_get_regs(struct net_device *netdev, struct ethtool_regs *regs,
+ void *p)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ unsigned int size;
+
+ regs->version = IONIC_DEV_CMD_REG_VERSION;
+
+ size = IONIC_DEV_INFO_REG_COUNT * sizeof(u32);
+ memcpy_fromio(p, lif->ionic->idev.dev_info_regs->words, size);
+
+ size = IONIC_DEV_CMD_REG_COUNT * sizeof(u32);
+ memcpy_fromio(p, lif->ionic->idev.dev_cmd_regs->words, size);
+}
+
+static int ionic_get_link_ksettings(struct net_device *netdev,
+ struct ethtool_link_ksettings *ks)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_dev *idev = &lif->ionic->idev;
+ int copper_seen = 0;
+
+ ethtool_link_ksettings_zero_link_mode(ks, supported);
+
+ /* The port_info data is found in a DMA space that the NIC keeps
+ * up-to-date, so there's no need to request the data from the
+ * NIC, we already have it in our memory space.
+ */
+
+ switch (le16_to_cpu(idev->port_info->status.xcvr.pid)) {
+ /* Copper */
+ case IONIC_XCVR_PID_QSFP_100G_CR4:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 100000baseCR4_Full);
+ copper_seen++;
+ break;
+ case IONIC_XCVR_PID_QSFP_40GBASE_CR4:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 40000baseCR4_Full);
+ copper_seen++;
+ break;
+ case IONIC_XCVR_PID_SFP_25GBASE_CR_S:
+ case IONIC_XCVR_PID_SFP_25GBASE_CR_L:
+ case IONIC_XCVR_PID_SFP_25GBASE_CR_N:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 25000baseCR_Full);
+ copper_seen++;
+ break;
+ case IONIC_XCVR_PID_SFP_10GBASE_AOC:
+ case IONIC_XCVR_PID_SFP_10GBASE_CU:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 10000baseCR_Full);
+ copper_seen++;
+ break;
+
+ /* Fibre */
+ case IONIC_XCVR_PID_QSFP_100G_SR4:
+ case IONIC_XCVR_PID_QSFP_100G_AOC:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 100000baseSR4_Full);
+ break;
+ case IONIC_XCVR_PID_QSFP_100G_LR4:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 100000baseLR4_ER4_Full);
+ break;
+ case IONIC_XCVR_PID_QSFP_100G_ER4:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 100000baseLR4_ER4_Full);
+ break;
+ case IONIC_XCVR_PID_QSFP_40GBASE_SR4:
+ case IONIC_XCVR_PID_QSFP_40GBASE_AOC:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 40000baseSR4_Full);
+ break;
+ case IONIC_XCVR_PID_QSFP_40GBASE_LR4:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 40000baseLR4_Full);
+ break;
+ case IONIC_XCVR_PID_SFP_25GBASE_SR:
+ case IONIC_XCVR_PID_SFP_25GBASE_AOC:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 25000baseSR_Full);
+ break;
+ case IONIC_XCVR_PID_SFP_10GBASE_SR:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 10000baseSR_Full);
+ break;
+ case IONIC_XCVR_PID_SFP_10GBASE_LR:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 10000baseLR_Full);
+ break;
+ case IONIC_XCVR_PID_SFP_10GBASE_LRM:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 10000baseLRM_Full);
+ break;
+ case IONIC_XCVR_PID_SFP_10GBASE_ER:
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 10000baseER_Full);
+ break;
+ case IONIC_XCVR_PID_UNKNOWN:
+ /* This means there's no module plugged in */
+ break;
+ default:
+ dev_info(lif->ionic->dev, "unknown xcvr type pid=%d / 0x%x\n",
+ idev->port_info->status.xcvr.pid,
+ idev->port_info->status.xcvr.pid);
+ break;
+ }
+
+ bitmap_copy(ks->link_modes.advertising, ks->link_modes.supported,
+ __ETHTOOL_LINK_MODE_MASK_NBITS);
+
+ ethtool_link_ksettings_add_link_mode(ks, supported, FEC_BASER);
+ ethtool_link_ksettings_add_link_mode(ks, supported, FEC_RS);
+ if (idev->port_info->config.fec_type == IONIC_PORT_FEC_TYPE_FC)
+ ethtool_link_ksettings_add_link_mode(ks, advertising, FEC_BASER);
+ else if (idev->port_info->config.fec_type == IONIC_PORT_FEC_TYPE_RS)
+ ethtool_link_ksettings_add_link_mode(ks, advertising, FEC_RS);
+
+ ethtool_link_ksettings_add_link_mode(ks, supported, FIBRE);
+ ethtool_link_ksettings_add_link_mode(ks, supported, Pause);
+
+ if (idev->port_info->status.xcvr.phy == IONIC_PHY_TYPE_COPPER ||
+ copper_seen)
+ ks->base.port = PORT_DA;
+ else if (idev->port_info->status.xcvr.phy == IONIC_PHY_TYPE_FIBER)
+ ks->base.port = PORT_FIBRE;
+ else
+ ks->base.port = PORT_NONE;
+
+ if (ks->base.port != PORT_NONE) {
+ ks->base.speed = le32_to_cpu(lif->info->status.link_speed);
+
+ if (le16_to_cpu(lif->info->status.link_status))
+ ks->base.duplex = DUPLEX_FULL;
+ else
+ ks->base.duplex = DUPLEX_UNKNOWN;
+
+ ethtool_link_ksettings_add_link_mode(ks, supported, Autoneg);
+
+ if (idev->port_info->config.an_enable) {
+ ethtool_link_ksettings_add_link_mode(ks, advertising,
+ Autoneg);
+ ks->base.autoneg = AUTONEG_ENABLE;
+ }
+ }
+
+ return 0;
+}
+
+static int ionic_set_link_ksettings(struct net_device *netdev,
+ const struct ethtool_link_ksettings *ks)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic *ionic = lif->ionic;
+ struct ionic_dev *idev;
+ u32 req_rs, req_fc;
+ u8 fec_type;
+ int err = 0;
+
+ idev = &lif->ionic->idev;
+ fec_type = IONIC_PORT_FEC_TYPE_NONE;
+
+ /* set autoneg */
+ if (ks->base.autoneg != idev->port_info->config.an_enable) {
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_port_autoneg(idev, ks->base.autoneg);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+ if (err)
+ return err;
+ }
+
+ /* set speed */
+ if (ks->base.speed != le32_to_cpu(idev->port_info->config.speed)) {
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_port_speed(idev, ks->base.speed);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+ if (err)
+ return err;
+ }
+
+ /* set FEC */
+ req_rs = ethtool_link_ksettings_test_link_mode(ks, advertising, FEC_RS);
+ req_fc = ethtool_link_ksettings_test_link_mode(ks, advertising, FEC_BASER);
+ if (req_rs && req_fc) {
+ netdev_info(netdev, "Only select one FEC mode at a time\n");
+ return -EINVAL;
+ } else if (req_fc) {
+ fec_type = IONIC_PORT_FEC_TYPE_FC;
+ } else if (req_rs) {
+ fec_type = IONIC_PORT_FEC_TYPE_RS;
+ } else if (!(req_rs | req_fc)) {
+ fec_type = IONIC_PORT_FEC_TYPE_NONE;
+ }
+
+ if (fec_type != idev->port_info->config.fec_type) {
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_port_fec(idev, fec_type);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static void ionic_get_pauseparam(struct net_device *netdev,
+ struct ethtool_pauseparam *pause)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ u8 pause_type;
+
+ pause->autoneg = 0;
+
+ pause_type = lif->ionic->idev.port_info->config.pause_type;
+ if (pause_type) {
+ pause->rx_pause = pause_type & IONIC_PAUSE_F_RX ? 1 : 0;
+ pause->tx_pause = pause_type & IONIC_PAUSE_F_TX ? 1 : 0;
+ }
+}
+
+static int ionic_set_pauseparam(struct net_device *netdev,
+ struct ethtool_pauseparam *pause)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic *ionic = lif->ionic;
+ u32 requested_pause;
+ int err;
+
+ if (pause->autoneg)
+ return -EOPNOTSUPP;
+
+ /* change both at the same time */
+ requested_pause = IONIC_PORT_PAUSE_TYPE_LINK;
+ if (pause->rx_pause)
+ requested_pause |= IONIC_PAUSE_F_RX;
+ if (pause->tx_pause)
+ requested_pause |= IONIC_PAUSE_F_TX;
+
+ if (requested_pause == lif->ionic->idev.port_info->config.pause_type)
+ return 0;
+
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_port_pause(&lif->ionic->idev, requested_pause);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int ionic_get_coalesce(struct net_device *netdev,
+ struct ethtool_coalesce *coalesce)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ /* Tx uses Rx interrupt */
+ coalesce->tx_coalesce_usecs = lif->rx_coalesce_usecs;
+ coalesce->rx_coalesce_usecs = lif->rx_coalesce_usecs;
+
+ return 0;
+}
+
+static int ionic_set_coalesce(struct net_device *netdev,
+ struct ethtool_coalesce *coalesce)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_identity *ident;
+ struct ionic_qcq *qcq;
+ unsigned int i;
+ u32 usecs;
+ u32 coal;
+
+ if (coalesce->rx_max_coalesced_frames ||
+ coalesce->rx_coalesce_usecs_irq ||
+ coalesce->rx_max_coalesced_frames_irq ||
+ coalesce->tx_max_coalesced_frames ||
+ coalesce->tx_coalesce_usecs_irq ||
+ coalesce->tx_max_coalesced_frames_irq ||
+ coalesce->stats_block_coalesce_usecs ||
+ coalesce->use_adaptive_rx_coalesce ||
+ coalesce->use_adaptive_tx_coalesce ||
+ coalesce->pkt_rate_low ||
+ coalesce->rx_coalesce_usecs_low ||
+ coalesce->rx_max_coalesced_frames_low ||
+ coalesce->tx_coalesce_usecs_low ||
+ coalesce->tx_max_coalesced_frames_low ||
+ coalesce->pkt_rate_high ||
+ coalesce->rx_coalesce_usecs_high ||
+ coalesce->rx_max_coalesced_frames_high ||
+ coalesce->tx_coalesce_usecs_high ||
+ coalesce->tx_max_coalesced_frames_high ||
+ coalesce->rate_sample_interval)
+ return -EINVAL;
+
+ ident = &lif->ionic->ident;
+ if (ident->dev.intr_coal_div == 0) {
+ netdev_warn(netdev, "bad HW value in dev.intr_coal_div = %d\n",
+ ident->dev.intr_coal_div);
+ return -EIO;
+ }
+
+ /* Tx uses Rx interrupt, so only change Rx */
+ if (coalesce->tx_coalesce_usecs != lif->rx_coalesce_usecs) {
+ netdev_warn(netdev, "only the rx-usecs can be changed\n");
+ return -EINVAL;
+ }
+
+ coal = ionic_coal_usec_to_hw(lif->ionic, coalesce->rx_coalesce_usecs);
+
+ if (coal > IONIC_INTR_CTRL_COAL_MAX)
+ return -ERANGE;
+
+ /* If they asked for non-zero and it resolved to zero, bump it up */
+ if (!coal && coalesce->rx_coalesce_usecs)
+ coal = 1;
+
+ /* Convert it back to get device resolution */
+ usecs = ionic_coal_hw_to_usec(lif->ionic, coal);
+
+ if (usecs != lif->rx_coalesce_usecs) {
+ lif->rx_coalesce_usecs = usecs;
+
+ if (test_bit(IONIC_LIF_UP, lif->state)) {
+ for (i = 0; i < lif->nxqs; i++) {
+ qcq = lif->rxqcqs[i].qcq;
+ ionic_intr_coal_init(lif->ionic->idev.intr_ctrl,
+ qcq->intr.index, coal);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static void ionic_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ ring->tx_max_pending = IONIC_MAX_TXRX_DESC;
+ ring->tx_pending = lif->ntxq_descs;
+ ring->rx_max_pending = IONIC_MAX_TXRX_DESC;
+ ring->rx_pending = lif->nrxq_descs;
+}
+
+static int ionic_set_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ bool running;
+
+ if (ring->rx_mini_pending || ring->rx_jumbo_pending) {
+ netdev_info(netdev, "Changing jumbo or mini descriptors not supported\n");
+ return -EINVAL;
+ }
+
+ if (!is_power_of_2(ring->tx_pending) ||
+ !is_power_of_2(ring->rx_pending)) {
+ netdev_info(netdev, "Descriptor count must be a power of 2\n");
+ return -EINVAL;
+ }
+
+ /* if nothing to do return success */
+ if (ring->tx_pending == lif->ntxq_descs &&
+ ring->rx_pending == lif->nrxq_descs)
+ return 0;
+
+ if (!ionic_wait_for_bit(lif, IONIC_LIF_QUEUE_RESET))
+ return -EBUSY;
+
+ running = test_bit(IONIC_LIF_UP, lif->state);
+ if (running)
+ ionic_stop(netdev);
+
+ lif->ntxq_descs = ring->tx_pending;
+ lif->nrxq_descs = ring->rx_pending;
+
+ if (running)
+ ionic_open(netdev);
+ clear_bit(IONIC_LIF_QUEUE_RESET, lif->state);
+
+ return 0;
+}
+
+static void ionic_get_channels(struct net_device *netdev,
+ struct ethtool_channels *ch)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ /* report maximum channels */
+ ch->max_combined = lif->ionic->ntxqs_per_lif;
+
+ /* report current channels */
+ ch->combined_count = lif->nxqs;
+}
+
+static int ionic_set_channels(struct net_device *netdev,
+ struct ethtool_channels *ch)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ bool running;
+
+ if (!ch->combined_count || ch->other_count ||
+ ch->rx_count || ch->tx_count)
+ return -EINVAL;
+
+ if (ch->combined_count == lif->nxqs)
+ return 0;
+
+ if (!ionic_wait_for_bit(lif, IONIC_LIF_QUEUE_RESET))
+ return -EBUSY;
+
+ running = test_bit(IONIC_LIF_UP, lif->state);
+ if (running)
+ ionic_stop(netdev);
+
+ lif->nxqs = ch->combined_count;
+
+ if (running)
+ ionic_open(netdev);
+ clear_bit(IONIC_LIF_QUEUE_RESET, lif->state);
+
+ return 0;
+}
+
+static u32 ionic_get_priv_flags(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ u32 priv_flags = 0;
+
+ if (test_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state))
+ priv_flags |= PRIV_F_SW_DBG_STATS;
+
+ return priv_flags;
+}
+
+static int ionic_set_priv_flags(struct net_device *netdev, u32 priv_flags)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ u32 flags = lif->flags;
+
+ clear_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state);
+ if (priv_flags & PRIV_F_SW_DBG_STATS)
+ set_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state);
+
+ if (flags != lif->flags)
+ lif->flags = flags;
+
+ return 0;
+}
+
+static int ionic_get_rxnfc(struct net_device *netdev,
+ struct ethtool_rxnfc *info, u32 *rules)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ int err = 0;
+
+ switch (info->cmd) {
+ case ETHTOOL_GRXRINGS:
+ info->data = lif->nxqs;
+ break;
+ default:
+ netdev_err(netdev, "Command parameter %d is not supported\n",
+ info->cmd);
+ err = -EOPNOTSUPP;
+ }
+
+ return err;
+}
+
+static u32 ionic_get_rxfh_indir_size(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ return le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+}
+
+static u32 ionic_get_rxfh_key_size(struct net_device *netdev)
+{
+ return IONIC_RSS_HASH_KEY_SIZE;
+}
+
+static int ionic_get_rxfh(struct net_device *netdev, u32 *indir, u8 *key,
+ u8 *hfunc)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ unsigned int i, tbl_sz;
+
+ if (indir) {
+ tbl_sz = le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+ for (i = 0; i < tbl_sz; i++)
+ indir[i] = lif->rss_ind_tbl[i];
+ }
+
+ if (key)
+ memcpy(key, lif->rss_hash_key, IONIC_RSS_HASH_KEY_SIZE);
+
+ if (hfunc)
+ *hfunc = ETH_RSS_HASH_TOP;
+
+ return 0;
+}
+
+static int ionic_set_rxfh(struct net_device *netdev, const u32 *indir,
+ const u8 *key, const u8 hfunc)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ int err;
+
+ if (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP)
+ return -EOPNOTSUPP;
+
+ err = ionic_lif_rss_config(lif, lif->rss_types, key, indir);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int ionic_set_tunable(struct net_device *dev,
+ const struct ethtool_tunable *tuna,
+ const void *data)
+{
+ struct ionic_lif *lif = netdev_priv(dev);
+
+ switch (tuna->id) {
+ case ETHTOOL_RX_COPYBREAK:
+ lif->rx_copybreak = *(u32 *)data;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int ionic_get_tunable(struct net_device *netdev,
+ const struct ethtool_tunable *tuna, void *data)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ switch (tuna->id) {
+ case ETHTOOL_RX_COPYBREAK:
+ *(u32 *)data = lif->rx_copybreak;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int ionic_get_module_info(struct net_device *netdev,
+ struct ethtool_modinfo *modinfo)
+
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_dev *idev = &lif->ionic->idev;
+ struct ionic_xcvr_status *xcvr;
+
+ xcvr = &idev->port_info->status.xcvr;
+
+ /* report the module data type and length */
+ switch (xcvr->sprom[0]) {
+ case 0x03: /* SFP */
+ modinfo->type = ETH_MODULE_SFF_8079;
+ modinfo->eeprom_len = ETH_MODULE_SFF_8079_LEN;
+ break;
+ case 0x0D: /* QSFP */
+ case 0x11: /* QSFP28 */
+ modinfo->type = ETH_MODULE_SFF_8436;
+ modinfo->eeprom_len = ETH_MODULE_SFF_8436_LEN;
+ break;
+ default:
+ netdev_info(netdev, "unknown xcvr type 0x%02x\n",
+ xcvr->sprom[0]);
+ break;
+ }
+
+ return 0;
+}
+
+static int ionic_get_module_eeprom(struct net_device *netdev,
+ struct ethtool_eeprom *ee,
+ u8 *data)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_dev *idev = &lif->ionic->idev;
+ struct ionic_xcvr_status *xcvr;
+ char tbuf[sizeof(xcvr->sprom)];
+ int count = 10;
+ u32 len;
+
+ /* The NIC keeps the module prom up-to-date in the DMA space
+ * so we can simply copy the module bytes into the data buffer.
+ */
+ xcvr = &idev->port_info->status.xcvr;
+ len = min_t(u32, sizeof(xcvr->sprom), ee->len);
+
+ do {
+ memcpy(data, xcvr->sprom, len);
+ memcpy(tbuf, xcvr->sprom, len);
+
+ /* Let's make sure we got a consistent copy */
+ if (!memcmp(data, tbuf, len))
+ break;
+
+ } while (--count);
+
+ if (!count)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static int ionic_nway_reset(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic *ionic = lif->ionic;
+ int err = 0;
+
+ /* flap the link to force auto-negotiation */
+
+ mutex_lock(&ionic->dev_cmd_lock);
+
+ ionic_dev_cmd_port_state(&ionic->idev, IONIC_PORT_ADMIN_STATE_DOWN);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+
+ if (!err) {
+ ionic_dev_cmd_port_state(&ionic->idev, IONIC_PORT_ADMIN_STATE_UP);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ }
+
+ mutex_unlock(&ionic->dev_cmd_lock);
+
+ return err;
+}
+
+static const struct ethtool_ops ionic_ethtool_ops = {
+ .get_drvinfo = ionic_get_drvinfo,
+ .get_regs_len = ionic_get_regs_len,
+ .get_regs = ionic_get_regs,
+ .get_link = ethtool_op_get_link,
+ .get_link_ksettings = ionic_get_link_ksettings,
+ .get_coalesce = ionic_get_coalesce,
+ .set_coalesce = ionic_set_coalesce,
+ .get_ringparam = ionic_get_ringparam,
+ .set_ringparam = ionic_set_ringparam,
+ .get_channels = ionic_get_channels,
+ .set_channels = ionic_set_channels,
+ .get_strings = ionic_get_strings,
+ .get_ethtool_stats = ionic_get_stats,
+ .get_sset_count = ionic_get_sset_count,
+ .get_priv_flags = ionic_get_priv_flags,
+ .set_priv_flags = ionic_set_priv_flags,
+ .get_rxnfc = ionic_get_rxnfc,
+ .get_rxfh_indir_size = ionic_get_rxfh_indir_size,
+ .get_rxfh_key_size = ionic_get_rxfh_key_size,
+ .get_rxfh = ionic_get_rxfh,
+ .set_rxfh = ionic_set_rxfh,
+ .get_tunable = ionic_get_tunable,
+ .set_tunable = ionic_set_tunable,
+ .get_module_info = ionic_get_module_info,
+ .get_module_eeprom = ionic_get_module_eeprom,
+ .get_pauseparam = ionic_get_pauseparam,
+ .set_pauseparam = ionic_set_pauseparam,
+ .set_link_ksettings = ionic_set_link_ksettings,
+ .nway_reset = ionic_nway_reset,
+};
+
+void ionic_ethtool_set_ops(struct net_device *netdev)
+{
+ netdev->ethtool_ops = &ionic_ethtool_ops;
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.h b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.h
new file mode 100644
index 0000000..38b91b1
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_ETHTOOL_H_
+#define _IONIC_ETHTOOL_H_
+
+void ionic_ethtool_set_ops(struct net_device *netdev);
+
+#endif /* _IONIC_ETHTOOL_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_if.h b/drivers/net/ethernet/pensando/ionic/ionic_if.h
new file mode 100644
index 0000000..5bfdda1
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_if.h
@@ -0,0 +1,2482 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB OR BSD-2-Clause */
+/* Copyright (c) 2017-2019 Pensando Systems, Inc. All rights reserved. */
+
+#ifndef _IONIC_IF_H_
+#define _IONIC_IF_H_
+
+#pragma pack(push, 1)
+
+#define IONIC_DEV_INFO_SIGNATURE 0x44455649 /* 'DEVI' */
+#define IONIC_DEV_INFO_VERSION 1
+#define IONIC_IFNAMSIZ 16
+
+/**
+ * Commands
+ */
+enum ionic_cmd_opcode {
+ IONIC_CMD_NOP = 0,
+
+ /* Device commands */
+ IONIC_CMD_IDENTIFY = 1,
+ IONIC_CMD_INIT = 2,
+ IONIC_CMD_RESET = 3,
+ IONIC_CMD_GETATTR = 4,
+ IONIC_CMD_SETATTR = 5,
+
+ /* Port commands */
+ IONIC_CMD_PORT_IDENTIFY = 10,
+ IONIC_CMD_PORT_INIT = 11,
+ IONIC_CMD_PORT_RESET = 12,
+ IONIC_CMD_PORT_GETATTR = 13,
+ IONIC_CMD_PORT_SETATTR = 14,
+
+ /* LIF commands */
+ IONIC_CMD_LIF_IDENTIFY = 20,
+ IONIC_CMD_LIF_INIT = 21,
+ IONIC_CMD_LIF_RESET = 22,
+ IONIC_CMD_LIF_GETATTR = 23,
+ IONIC_CMD_LIF_SETATTR = 24,
+
+ IONIC_CMD_RX_MODE_SET = 30,
+ IONIC_CMD_RX_FILTER_ADD = 31,
+ IONIC_CMD_RX_FILTER_DEL = 32,
+
+ /* Queue commands */
+ IONIC_CMD_Q_INIT = 40,
+ IONIC_CMD_Q_CONTROL = 41,
+
+ /* RDMA commands */
+ IONIC_CMD_RDMA_RESET_LIF = 50,
+ IONIC_CMD_RDMA_CREATE_EQ = 51,
+ IONIC_CMD_RDMA_CREATE_CQ = 52,
+ IONIC_CMD_RDMA_CREATE_ADMINQ = 53,
+
+ /* QoS commands */
+ IONIC_CMD_QOS_CLASS_IDENTIFY = 240,
+ IONIC_CMD_QOS_CLASS_INIT = 241,
+ IONIC_CMD_QOS_CLASS_RESET = 242,
+
+ /* Firmware commands */
+ IONIC_CMD_FW_DOWNLOAD = 254,
+ IONIC_CMD_FW_CONTROL = 255,
+};
+
+/**
+ * Command Return codes
+ */
+enum ionic_status_code {
+ IONIC_RC_SUCCESS = 0, /* Success */
+ IONIC_RC_EVERSION = 1, /* Incorrect version for request */
+ IONIC_RC_EOPCODE = 2, /* Invalid cmd opcode */
+ IONIC_RC_EIO = 3, /* I/O error */
+ IONIC_RC_EPERM = 4, /* Permission denied */
+ IONIC_RC_EQID = 5, /* Bad qid */
+ IONIC_RC_EQTYPE = 6, /* Bad qtype */
+ IONIC_RC_ENOENT = 7, /* No such element */
+ IONIC_RC_EINTR = 8, /* operation interrupted */
+ IONIC_RC_EAGAIN = 9, /* Try again */
+ IONIC_RC_ENOMEM = 10, /* Out of memory */
+ IONIC_RC_EFAULT = 11, /* Bad address */
+ IONIC_RC_EBUSY = 12, /* Device or resource busy */
+ IONIC_RC_EEXIST = 13, /* object already exists */
+ IONIC_RC_EINVAL = 14, /* Invalid argument */
+ IONIC_RC_ENOSPC = 15, /* No space left or alloc failure */
+ IONIC_RC_ERANGE = 16, /* Parameter out of range */
+ IONIC_RC_BAD_ADDR = 17, /* Descriptor contains a bad ptr */
+ IONIC_RC_DEV_CMD = 18, /* Device cmd attempted on AdminQ */
+ IONIC_RC_ENOSUPP = 19, /* Operation not supported */
+ IONIC_RC_ERROR = 29, /* Generic error */
+
+ IONIC_RC_ERDMA = 30, /* Generic RDMA error */
+};
+
+enum ionic_notifyq_opcode {
+ IONIC_EVENT_LINK_CHANGE = 1,
+ IONIC_EVENT_RESET = 2,
+ IONIC_EVENT_HEARTBEAT = 3,
+ IONIC_EVENT_LOG = 4,
+};
+
+/**
+ * struct cmd - General admin command format
+ * @opcode: Opcode for the command
+ * @lif_index: LIF index
+ * @cmd_data: Opcode-specific command bytes
+ */
+struct ionic_admin_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 lif_index;
+ u8 cmd_data[60];
+};
+
+/**
+ * struct admin_comp - General admin command completion format
+ * @status: The status of the command (enum status_code)
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @cmd_data: Command-specific bytes.
+ * @color: Color bit. (Always 0 for commands issued to the
+ * Device Cmd Registers.)
+ */
+struct ionic_admin_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ u8 cmd_data[11];
+ u8 color;
+#define IONIC_COMP_COLOR_MASK 0x80
+};
+
+static inline u8 color_match(u8 color, u8 done_color)
+{
+ return (!!(color & IONIC_COMP_COLOR_MASK)) == done_color;
+}
+
+/**
+ * struct nop_cmd - NOP command
+ * @opcode: opcode
+ */
+struct ionic_nop_cmd {
+ u8 opcode;
+ u8 rsvd[63];
+};
+
+/**
+ * struct nop_comp - NOP command completion
+ * @status: The status of the command (enum status_code)
+ */
+struct ionic_nop_comp {
+ u8 status;
+ u8 rsvd[15];
+};
+
+/**
+ * struct dev_init_cmd - Device init command
+ * @opcode: opcode
+ * @type: device type
+ */
+struct ionic_dev_init_cmd {
+ u8 opcode;
+ u8 type;
+ u8 rsvd[62];
+};
+
+/**
+ * struct init_comp - Device init command completion
+ * @status: The status of the command (enum status_code)
+ */
+struct ionic_dev_init_comp {
+ u8 status;
+ u8 rsvd[15];
+};
+
+/**
+ * struct dev_reset_cmd - Device reset command
+ * @opcode: opcode
+ */
+struct ionic_dev_reset_cmd {
+ u8 opcode;
+ u8 rsvd[63];
+};
+
+/**
+ * struct reset_comp - Reset command completion
+ * @status: The status of the command (enum status_code)
+ */
+struct ionic_dev_reset_comp {
+ u8 status;
+ u8 rsvd[15];
+};
+
+#define IONIC_IDENTITY_VERSION_1 1
+
+/**
+ * struct dev_identify_cmd - Driver/device identify command
+ * @opcode: opcode
+ * @ver: Highest version of identify supported by driver
+ */
+struct ionic_dev_identify_cmd {
+ u8 opcode;
+ u8 ver;
+ u8 rsvd[62];
+};
+
+/**
+ * struct dev_identify_comp - Driver/device identify command completion
+ * @status: The status of the command (enum status_code)
+ * @ver: Version of identify returned by device
+ */
+struct ionic_dev_identify_comp {
+ u8 status;
+ u8 ver;
+ u8 rsvd[14];
+};
+
+enum ionic_os_type {
+ IONIC_OS_TYPE_LINUX = 1,
+ IONIC_OS_TYPE_WIN = 2,
+ IONIC_OS_TYPE_DPDK = 3,
+ IONIC_OS_TYPE_FREEBSD = 4,
+ IONIC_OS_TYPE_IPXE = 5,
+ IONIC_OS_TYPE_ESXI = 6,
+};
+
+/**
+ * union drv_identity - driver identity information
+ * @os_type: OS type (see enum os_type)
+ * @os_dist: OS distribution, numeric format
+ * @os_dist_str: OS distribution, string format
+ * @kernel_ver: Kernel version, numeric format
+ * @kernel_ver_str: Kernel version, string format
+ * @driver_ver_str: Driver version, string format
+ */
+union ionic_drv_identity {
+ struct {
+ __le32 os_type;
+ __le32 os_dist;
+ char os_dist_str[128];
+ __le32 kernel_ver;
+ char kernel_ver_str[32];
+ char driver_ver_str[32];
+ };
+ __le32 words[512];
+};
+
+/**
+ * union dev_identity - device identity information
+ * @version: Version of device identify
+ * @type: Identify type (0 for now)
+ * @nports: Number of ports provisioned
+ * @nlifs: Number of LIFs provisioned
+ * @nintrs: Number of interrupts provisioned
+ * @ndbpgs_per_lif: Number of doorbell pages per LIF
+ * @intr_coal_mult: Interrupt coalescing multiplication factor.
+ * Scale user-supplied interrupt coalescing
+ * value in usecs to device units using:
+ * device units = usecs * mult / div
+ * @intr_coal_div: Interrupt coalescing division factor.
+ * Scale user-supplied interrupt coalescing
+ * value in usecs to device units using:
+ * device units = usecs * mult / div
+ *
+ */
+union ionic_dev_identity {
+ struct {
+ u8 version;
+ u8 type;
+ u8 rsvd[2];
+ u8 nports;
+ u8 rsvd2[3];
+ __le32 nlifs;
+ __le32 nintrs;
+ __le32 ndbpgs_per_lif;
+ __le32 intr_coal_mult;
+ __le32 intr_coal_div;
+ };
+ __le32 words[512];
+};
+
+enum ionic_lif_type {
+ IONIC_LIF_TYPE_CLASSIC = 0,
+ IONIC_LIF_TYPE_MACVLAN = 1,
+ IONIC_LIF_TYPE_NETQUEUE = 2,
+};
+
+/**
+ * struct lif_identify_cmd - lif identify command
+ * @opcode: opcode
+ * @type: lif type (enum lif_type)
+ * @ver: version of identify returned by device
+ */
+struct ionic_lif_identify_cmd {
+ u8 opcode;
+ u8 type;
+ u8 ver;
+ u8 rsvd[61];
+};
+
+/**
+ * struct lif_identify_comp - lif identify command completion
+ * @status: status of the command (enum status_code)
+ * @ver: version of identify returned by device
+ */
+struct ionic_lif_identify_comp {
+ u8 status;
+ u8 ver;
+ u8 rsvd2[14];
+};
+
+enum ionic_lif_capability {
+ IONIC_LIF_CAP_ETH = BIT(0),
+ IONIC_LIF_CAP_RDMA = BIT(1),
+};
+
+/**
+ * Logical Queue Types
+ */
+enum ionic_logical_qtype {
+ IONIC_QTYPE_ADMINQ = 0,
+ IONIC_QTYPE_NOTIFYQ = 1,
+ IONIC_QTYPE_RXQ = 2,
+ IONIC_QTYPE_TXQ = 3,
+ IONIC_QTYPE_EQ = 4,
+ IONIC_QTYPE_MAX = 16,
+};
+
+/**
+ * struct lif_logical_qtype - Descriptor of logical to hardware queue type.
+ * @qtype: Hardware Queue Type.
+ * @qid_count: Number of Queue IDs of the logical type.
+ * @qid_base: Minimum Queue ID of the logical type.
+ */
+struct ionic_lif_logical_qtype {
+ u8 qtype;
+ u8 rsvd[3];
+ __le32 qid_count;
+ __le32 qid_base;
+};
+
+enum ionic_lif_state {
+ IONIC_LIF_DISABLE = 0,
+ IONIC_LIF_ENABLE = 1,
+ IONIC_LIF_HANG_RESET = 2,
+};
+
+/**
+ * LIF configuration
+ * @state: lif state (enum lif_state)
+ * @name: lif name
+ * @mtu: mtu
+ * @mac: station mac address
+ * @features: features (enum eth_hw_features)
+ * @queue_count: queue counts per queue-type
+ */
+union ionic_lif_config {
+ struct {
+ u8 state;
+ u8 rsvd[3];
+ char name[IONIC_IFNAMSIZ];
+ __le32 mtu;
+ u8 mac[6];
+ u8 rsvd2[2];
+ __le64 features;
+ __le32 queue_count[IONIC_QTYPE_MAX];
+ };
+ __le32 words[64];
+};
+
+/**
+ * struct lif_identity - lif identity information (type-specific)
+ *
+ * @capabilities LIF capabilities
+ *
+ * Ethernet:
+ * @version: Ethernet identify structure version.
+ * @features: Ethernet features supported on this lif type.
+ * @max_ucast_filters: Number of perfect unicast addresses supported.
+ * @max_mcast_filters: Number of perfect multicast addresses supported.
+ * @min_frame_size: Minimum size of frames to be sent
+ * @max_frame_size: Maximim size of frames to be sent
+ * @config: LIF config struct with features, mtu, mac, q counts
+ *
+ * RDMA:
+ * @version: RDMA version of opcodes and queue descriptors.
+ * @qp_opcodes: Number of rdma queue pair opcodes supported.
+ * @admin_opcodes: Number of rdma admin opcodes supported.
+ * @npts_per_lif: Page table size per lif
+ * @nmrs_per_lif: Number of memory regions per lif
+ * @nahs_per_lif: Number of address handles per lif
+ * @max_stride: Max work request stride.
+ * @cl_stride: Cache line stride.
+ * @pte_stride: Page table entry stride.
+ * @rrq_stride: Remote RQ work request stride.
+ * @rsq_stride: Remote SQ work request stride.
+ * @dcqcn_profiles: Number of DCQCN profiles
+ * @aq_qtype: RDMA Admin Qtype.
+ * @sq_qtype: RDMA Send Qtype.
+ * @rq_qtype: RDMA Receive Qtype.
+ * @cq_qtype: RDMA Completion Qtype.
+ * @eq_qtype: RDMA Event Qtype.
+ */
+union ionic_lif_identity {
+ struct {
+ __le64 capabilities;
+
+ struct {
+ u8 version;
+ u8 rsvd[3];
+ __le32 max_ucast_filters;
+ __le32 max_mcast_filters;
+ __le16 rss_ind_tbl_sz;
+ __le32 min_frame_size;
+ __le32 max_frame_size;
+ u8 rsvd2[106];
+ union ionic_lif_config config;
+ } eth;
+
+ struct {
+ u8 version;
+ u8 qp_opcodes;
+ u8 admin_opcodes;
+ u8 rsvd;
+ __le32 npts_per_lif;
+ __le32 nmrs_per_lif;
+ __le32 nahs_per_lif;
+ u8 max_stride;
+ u8 cl_stride;
+ u8 pte_stride;
+ u8 rrq_stride;
+ u8 rsq_stride;
+ u8 dcqcn_profiles;
+ u8 rsvd_dimensions[10];
+ struct ionic_lif_logical_qtype aq_qtype;
+ struct ionic_lif_logical_qtype sq_qtype;
+ struct ionic_lif_logical_qtype rq_qtype;
+ struct ionic_lif_logical_qtype cq_qtype;
+ struct ionic_lif_logical_qtype eq_qtype;
+ } rdma;
+ };
+ __le32 words[512];
+};
+
+/**
+ * struct lif_init_cmd - LIF init command
+ * @opcode: opcode
+ * @type: LIF type (enum lif_type)
+ * @index: LIF index
+ * @info_pa: destination address for lif info (struct lif_info)
+ */
+struct ionic_lif_init_cmd {
+ u8 opcode;
+ u8 type;
+ __le16 index;
+ __le32 rsvd;
+ __le64 info_pa;
+ u8 rsvd2[48];
+};
+
+/**
+ * struct lif_init_comp - LIF init command completion
+ * @status: The status of the command (enum status_code)
+ */
+struct ionic_lif_init_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 hw_index;
+ u8 rsvd2[12];
+};
+
+/**
+ * struct q_init_cmd - Queue init command
+ * @opcode: opcode
+ * @type: Logical queue type
+ * @ver: Queue version (defines opcode/descriptor scope)
+ * @lif_index: LIF index
+ * @index: (lif, qtype) relative admin queue index
+ * @intr_index: Interrupt control register index
+ * @pid: Process ID
+ * @flags:
+ * IRQ: Interrupt requested on completion
+ * ENA: Enable the queue. If ENA=0 the queue is initialized
+ * but remains disabled, to be later enabled with the
+ * Queue Enable command. If ENA=1, then queue is
+ * initialized and then enabled.
+ * SG: Enable Scatter-Gather on the queue.
+ * in number of descs. The actual ring size is
+ * (1 << ring_size). For example, to
+ * select a ring size of 64 descriptors write
+ * ring_size = 6. The minimum ring_size value is 2
+ * for a ring size of 4 descriptors. The maximum
+ * ring_size value is 16 for a ring size of 64k
+ * descriptors. Values of ring_size <2 and >16 are
+ * reserved.
+ * EQ: Enable the Event Queue
+ * @cos: Class of service for this queue.
+ * @ring_size: Queue ring size, encoded as a log2(size)
+ * @ring_base: Queue ring base address
+ * @cq_ring_base: Completion queue ring base address
+ * @sg_ring_base: Scatter/Gather ring base address
+ * @eq_index: Event queue index
+ */
+struct ionic_q_init_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 lif_index;
+ u8 type;
+ u8 ver;
+ u8 rsvd1[2];
+ __le32 index;
+ __le16 pid;
+ __le16 intr_index;
+ __le16 flags;
+#define IONIC_QINIT_F_IRQ 0x01 /* Request interrupt on completion */
+#define IONIC_QINIT_F_ENA 0x02 /* Enable the queue */
+#define IONIC_QINIT_F_SG 0x04 /* Enable scatter/gather on the queue */
+#define IONIC_QINIT_F_EQ 0x08 /* Enable event queue */
+#define IONIC_QINIT_F_DEBUG 0x80 /* Enable queue debugging */
+ u8 cos;
+ u8 ring_size;
+ __le64 ring_base;
+ __le64 cq_ring_base;
+ __le64 sg_ring_base;
+ __le32 eq_index;
+ u8 rsvd2[16];
+};
+
+/**
+ * struct q_init_comp - Queue init command completion
+ * @status: The status of the command (enum status_code)
+ * @ver: Queue version (defines opcode/descriptor scope)
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @hw_index: Hardware Queue ID
+ * @hw_type: Hardware Queue type
+ * @color: Color
+ */
+struct ionic_q_init_comp {
+ u8 status;
+ u8 ver;
+ __le16 comp_index;
+ __le32 hw_index;
+ u8 hw_type;
+ u8 rsvd2[6];
+ u8 color;
+};
+
+/* the device's internal addressing uses up to 52 bits */
+#define IONIC_ADDR_LEN 52
+#define IONIC_ADDR_MASK (BIT_ULL(IONIC_ADDR_LEN) - 1)
+
+enum ionic_txq_desc_opcode {
+ IONIC_TXQ_DESC_OPCODE_CSUM_NONE = 0,
+ IONIC_TXQ_DESC_OPCODE_CSUM_PARTIAL = 1,
+ IONIC_TXQ_DESC_OPCODE_CSUM_HW = 2,
+ IONIC_TXQ_DESC_OPCODE_TSO = 3,
+};
+
+/**
+ * struct txq_desc - Ethernet Tx queue descriptor format
+ * @opcode: Tx operation, see TXQ_DESC_OPCODE_*:
+ *
+ * IONIC_TXQ_DESC_OPCODE_CSUM_NONE:
+ *
+ * Non-offload send. No segmentation,
+ * fragmentation or checksum calc/insertion is
+ * performed by device; packet is prepared
+ * to send by software stack and requires
+ * no further manipulation from device.
+ *
+ * IONIC_TXQ_DESC_OPCODE_CSUM_PARTIAL:
+ *
+ * Offload 16-bit L4 checksum
+ * calculation/insertion. The device will
+ * calculate the L4 checksum value and
+ * insert the result in the packet's L4
+ * header checksum field. The L4 checksum
+ * is calculated starting at @csum_start bytes
+ * into the packet to the end of the packet.
+ * The checksum insertion position is given
+ * in @csum_offset. This feature is only
+ * applicable to protocols such as TCP, UDP
+ * and ICMP where a standard (i.e. the
+ * 'IP-style' checksum) one's complement
+ * 16-bit checksum is used, using an IP
+ * pseudo-header to seed the calculation.
+ * Software will preload the L4 checksum
+ * field with the IP pseudo-header checksum.
+ *
+ * For tunnel encapsulation, @csum_start and
+ * @csum_offset refer to the inner L4
+ * header. Supported tunnels encapsulations
+ * are: IPIP, GRE, and UDP. If the @encap
+ * is clear, no further processing by the
+ * device is required; software will
+ * calculate the outer header checksums. If
+ * the @encap is set, the device will
+ * offload the outer header checksums using
+ * LCO (local checksum offload) (see
+ * Documentation/networking/checksum-
+ * offloads.txt for more info).
+ *
+ * IONIC_TXQ_DESC_OPCODE_CSUM_HW:
+ *
+ * Offload 16-bit checksum computation to hardware.
+ * If @csum_l3 is set then the packet's L3 checksum is
+ * updated. Similarly, if @csum_l4 is set the the L4
+ * checksum is updated. If @encap is set then encap header
+ * checksums are also updated.
+ *
+ * IONIC_TXQ_DESC_OPCODE_TSO:
+ *
+ * Device preforms TCP segmentation offload
+ * (TSO). @hdr_len is the number of bytes
+ * to the end of TCP header (the offset to
+ * the TCP payload). @mss is the desired
+ * MSS, the TCP payload length for each
+ * segment. The device will calculate/
+ * insert IP (IPv4 only) and TCP checksums
+ * for each segment. In the first data
+ * buffer containing the header template,
+ * the driver will set IPv4 checksum to 0
+ * and preload TCP checksum with the IP
+ * pseudo header calculated with IP length = 0.
+ *
+ * Supported tunnel encapsulations are IPIP,
+ * layer-3 GRE, and UDP. @hdr_len includes
+ * both outer and inner headers. The driver
+ * will set IPv4 checksum to zero and
+ * preload TCP checksum with IP pseudo
+ * header on the inner header.
+ *
+ * TCP ECN offload is supported. The device
+ * will set CWR flag in the first segment if
+ * CWR is set in the template header, and
+ * clear CWR in remaining segments.
+ * @flags:
+ * vlan:
+ * Insert an L2 VLAN header using @vlan_tci.
+ * encap:
+ * Calculate encap header checksum.
+ * csum_l3:
+ * Compute L3 header checksum.
+ * csum_l4:
+ * Compute L4 header checksum.
+ * tso_sot:
+ * TSO start
+ * tso_eot:
+ * TSO end
+ * @num_sg_elems: Number of scatter-gather elements in SG
+ * descriptor
+ * @addr: First data buffer's DMA address.
+ * (Subsequent data buffers are on txq_sg_desc).
+ * @len: First data buffer's length, in bytes
+ * @vlan_tci: VLAN tag to insert in the packet (if requested
+ * by @V-bit). Includes .1p and .1q tags
+ * @hdr_len: Length of packet headers, including
+ * encapsulating outer header, if applicable.
+ * Valid for opcodes TXQ_DESC_OPCODE_CALC_CSUM and
+ * TXQ_DESC_OPCODE_TSO. Should be set to zero for
+ * all other modes. For
+ * TXQ_DESC_OPCODE_CALC_CSUM, @hdr_len is length
+ * of headers up to inner-most L4 header. For
+ * TXQ_DESC_OPCODE_TSO, @hdr_len is up to
+ * inner-most L4 payload, so inclusive of
+ * inner-most L4 header.
+ * @mss: Desired MSS value for TSO. Only applicable for
+ * TXQ_DESC_OPCODE_TSO.
+ * @csum_start: Offset into inner-most L3 header of checksum
+ * @csum_offset: Offset into inner-most L4 header of checksum
+ */
+
+#define IONIC_TXQ_DESC_OPCODE_MASK 0xf
+#define IONIC_TXQ_DESC_OPCODE_SHIFT 4
+#define IONIC_TXQ_DESC_FLAGS_MASK 0xf
+#define IONIC_TXQ_DESC_FLAGS_SHIFT 0
+#define IONIC_TXQ_DESC_NSGE_MASK 0xf
+#define IONIC_TXQ_DESC_NSGE_SHIFT 8
+#define IONIC_TXQ_DESC_ADDR_MASK (BIT_ULL(IONIC_ADDR_LEN) - 1)
+#define IONIC_TXQ_DESC_ADDR_SHIFT 12
+
+/* common flags */
+#define IONIC_TXQ_DESC_FLAG_VLAN 0x1
+#define IONIC_TXQ_DESC_FLAG_ENCAP 0x2
+
+/* flags for csum_hw opcode */
+#define IONIC_TXQ_DESC_FLAG_CSUM_L3 0x4
+#define IONIC_TXQ_DESC_FLAG_CSUM_L4 0x8
+
+/* flags for tso opcode */
+#define IONIC_TXQ_DESC_FLAG_TSO_SOT 0x4
+#define IONIC_TXQ_DESC_FLAG_TSO_EOT 0x8
+
+struct ionic_txq_desc {
+ __le64 cmd;
+ __le16 len;
+ union {
+ __le16 vlan_tci;
+ __le16 hword0;
+ };
+ union {
+ __le16 csum_start;
+ __le16 hdr_len;
+ __le16 hword1;
+ };
+ union {
+ __le16 csum_offset;
+ __le16 mss;
+ __le16 hword2;
+ };
+};
+
+static inline u64 encode_txq_desc_cmd(u8 opcode, u8 flags,
+ u8 nsge, u64 addr)
+{
+ u64 cmd;
+
+ cmd = (opcode & IONIC_TXQ_DESC_OPCODE_MASK) << IONIC_TXQ_DESC_OPCODE_SHIFT;
+ cmd |= (flags & IONIC_TXQ_DESC_FLAGS_MASK) << IONIC_TXQ_DESC_FLAGS_SHIFT;
+ cmd |= (nsge & IONIC_TXQ_DESC_NSGE_MASK) << IONIC_TXQ_DESC_NSGE_SHIFT;
+ cmd |= (addr & IONIC_TXQ_DESC_ADDR_MASK) << IONIC_TXQ_DESC_ADDR_SHIFT;
+
+ return cmd;
+};
+
+static inline void decode_txq_desc_cmd(u64 cmd, u8 *opcode, u8 *flags,
+ u8 *nsge, u64 *addr)
+{
+ *opcode = (cmd >> IONIC_TXQ_DESC_OPCODE_SHIFT) & IONIC_TXQ_DESC_OPCODE_MASK;
+ *flags = (cmd >> IONIC_TXQ_DESC_FLAGS_SHIFT) & IONIC_TXQ_DESC_FLAGS_MASK;
+ *nsge = (cmd >> IONIC_TXQ_DESC_NSGE_SHIFT) & IONIC_TXQ_DESC_NSGE_MASK;
+ *addr = (cmd >> IONIC_TXQ_DESC_ADDR_SHIFT) & IONIC_TXQ_DESC_ADDR_MASK;
+};
+
+#define IONIC_TX_MAX_SG_ELEMS 8
+#define IONIC_RX_MAX_SG_ELEMS 8
+
+/**
+ * struct txq_sg_desc - Transmit scatter-gather (SG) list
+ * @addr: DMA address of SG element data buffer
+ * @len: Length of SG element data buffer, in bytes
+ */
+struct ionic_txq_sg_desc {
+ struct ionic_txq_sg_elem {
+ __le64 addr;
+ __le16 len;
+ __le16 rsvd[3];
+ } elems[IONIC_TX_MAX_SG_ELEMS];
+};
+
+/**
+ * struct txq_comp - Ethernet transmit queue completion descriptor
+ * @status: The status of the command (enum status_code)
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @color: Color bit.
+ */
+struct ionic_txq_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ u8 rsvd2[11];
+ u8 color;
+};
+
+enum ionic_rxq_desc_opcode {
+ IONIC_RXQ_DESC_OPCODE_SIMPLE = 0,
+ IONIC_RXQ_DESC_OPCODE_SG = 1,
+};
+
+/**
+ * struct rxq_desc - Ethernet Rx queue descriptor format
+ * @opcode: Rx operation, see RXQ_DESC_OPCODE_*:
+ *
+ * RXQ_DESC_OPCODE_SIMPLE:
+ *
+ * Receive full packet into data buffer
+ * starting at @addr. Results of
+ * receive, including actual bytes received,
+ * are recorded in Rx completion descriptor.
+ *
+ * @len: Data buffer's length, in bytes.
+ * @addr: Data buffer's DMA address
+ */
+struct ionic_rxq_desc {
+ u8 opcode;
+ u8 rsvd[5];
+ __le16 len;
+ __le64 addr;
+};
+
+/**
+ * struct rxq_sg_desc - Receive scatter-gather (SG) list
+ * @addr: DMA address of SG element data buffer
+ * @len: Length of SG element data buffer, in bytes
+ */
+struct ionic_rxq_sg_desc {
+ struct ionic_rxq_sg_elem {
+ __le64 addr;
+ __le16 len;
+ __le16 rsvd[3];
+ } elems[IONIC_RX_MAX_SG_ELEMS];
+};
+
+/**
+ * struct rxq_comp - Ethernet receive queue completion descriptor
+ * @status: The status of the command (enum status_code)
+ * @num_sg_elems: Number of SG elements used by this descriptor
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @rss_hash: 32-bit RSS hash
+ * @csum: 16-bit sum of the packet's L2 payload.
+ * If the packet's L2 payload is odd length, an extra
+ * zero-value byte is included in the @csum calculation but
+ * not included in @len.
+ * @vlan_tci: VLAN tag stripped from the packet. Valid if @VLAN is
+ * set. Includes .1p and .1q tags.
+ * @len: Received packet length, in bytes. Excludes FCS.
+ * @csum_calc L2 payload checksum is computed or not
+ * @csum_tcp_ok: The TCP checksum calculated by the device
+ * matched the checksum in the receive packet's
+ * TCP header
+ * @csum_tcp_bad: The TCP checksum calculated by the device did
+ * not match the checksum in the receive packet's
+ * TCP header.
+ * @csum_udp_ok: The UDP checksum calculated by the device
+ * matched the checksum in the receive packet's
+ * UDP header
+ * @csum_udp_bad: The UDP checksum calculated by the device did
+ * not match the checksum in the receive packet's
+ * UDP header.
+ * @csum_ip_ok: The IPv4 checksum calculated by the device
+ * matched the checksum in the receive packet's
+ * first IPv4 header. If the receive packet
+ * contains both a tunnel IPv4 header and a
+ * transport IPv4 header, the device validates the
+ * checksum for the both IPv4 headers.
+ * @csum_ip_bad: The IPv4 checksum calculated by the device did
+ * not match the checksum in the receive packet's
+ * first IPv4 header. If the receive packet
+ * contains both a tunnel IPv4 header and a
+ * transport IPv4 header, the device validates the
+ * checksum for both IP headers.
+ * @VLAN: VLAN header was stripped and placed in @vlan_tci.
+ * @pkt_type: Packet type
+ * @color: Color bit.
+ */
+struct ionic_rxq_comp {
+ u8 status;
+ u8 num_sg_elems;
+ __le16 comp_index;
+ __le32 rss_hash;
+ __le16 csum;
+ __le16 vlan_tci;
+ __le16 len;
+ u8 csum_flags;
+#define IONIC_RXQ_COMP_CSUM_F_TCP_OK 0x01
+#define IONIC_RXQ_COMP_CSUM_F_TCP_BAD 0x02
+#define IONIC_RXQ_COMP_CSUM_F_UDP_OK 0x04
+#define IONIC_RXQ_COMP_CSUM_F_UDP_BAD 0x08
+#define IONIC_RXQ_COMP_CSUM_F_IP_OK 0x10
+#define IONIC_RXQ_COMP_CSUM_F_IP_BAD 0x20
+#define IONIC_RXQ_COMP_CSUM_F_VLAN 0x40
+#define IONIC_RXQ_COMP_CSUM_F_CALC 0x80
+ u8 pkt_type_color;
+#define IONIC_RXQ_COMP_PKT_TYPE_MASK 0x0f
+};
+
+enum ionic_pkt_type {
+ IONIC_PKT_TYPE_NON_IP = 0x000,
+ IONIC_PKT_TYPE_IPV4 = 0x001,
+ IONIC_PKT_TYPE_IPV4_TCP = 0x003,
+ IONIC_PKT_TYPE_IPV4_UDP = 0x005,
+ IONIC_PKT_TYPE_IPV6 = 0x008,
+ IONIC_PKT_TYPE_IPV6_TCP = 0x018,
+ IONIC_PKT_TYPE_IPV6_UDP = 0x028,
+};
+
+enum ionic_eth_hw_features {
+ IONIC_ETH_HW_VLAN_TX_TAG = BIT(0),
+ IONIC_ETH_HW_VLAN_RX_STRIP = BIT(1),
+ IONIC_ETH_HW_VLAN_RX_FILTER = BIT(2),
+ IONIC_ETH_HW_RX_HASH = BIT(3),
+ IONIC_ETH_HW_RX_CSUM = BIT(4),
+ IONIC_ETH_HW_TX_SG = BIT(5),
+ IONIC_ETH_HW_RX_SG = BIT(6),
+ IONIC_ETH_HW_TX_CSUM = BIT(7),
+ IONIC_ETH_HW_TSO = BIT(8),
+ IONIC_ETH_HW_TSO_IPV6 = BIT(9),
+ IONIC_ETH_HW_TSO_ECN = BIT(10),
+ IONIC_ETH_HW_TSO_GRE = BIT(11),
+ IONIC_ETH_HW_TSO_GRE_CSUM = BIT(12),
+ IONIC_ETH_HW_TSO_IPXIP4 = BIT(13),
+ IONIC_ETH_HW_TSO_IPXIP6 = BIT(14),
+ IONIC_ETH_HW_TSO_UDP = BIT(15),
+ IONIC_ETH_HW_TSO_UDP_CSUM = BIT(16),
+};
+
+/**
+ * struct q_control_cmd - Queue control command
+ * @opcode: opcode
+ * @type: Queue type
+ * @lif_index: LIF index
+ * @index: Queue index
+ * @oper: Operation (enum q_control_oper)
+ */
+struct ionic_q_control_cmd {
+ u8 opcode;
+ u8 type;
+ __le16 lif_index;
+ __le32 index;
+ u8 oper;
+ u8 rsvd[55];
+};
+
+typedef struct ionic_admin_comp ionic_q_control_comp;
+
+enum q_control_oper {
+ IONIC_Q_DISABLE = 0,
+ IONIC_Q_ENABLE = 1,
+ IONIC_Q_HANG_RESET = 2,
+};
+
+/**
+ * Physical connection type
+ */
+enum ionic_phy_type {
+ IONIC_PHY_TYPE_NONE = 0,
+ IONIC_PHY_TYPE_COPPER = 1,
+ IONIC_PHY_TYPE_FIBER = 2,
+};
+
+/**
+ * Transceiver status
+ */
+enum ionic_xcvr_state {
+ IONIC_XCVR_STATE_REMOVED = 0,
+ IONIC_XCVR_STATE_INSERTED = 1,
+ IONIC_XCVR_STATE_PENDING = 2,
+ IONIC_XCVR_STATE_SPROM_READ = 3,
+ IONIC_XCVR_STATE_SPROM_READ_ERR = 4,
+};
+
+/**
+ * Supported link modes
+ */
+enum ionic_xcvr_pid {
+ IONIC_XCVR_PID_UNKNOWN = 0,
+
+ /* CU */
+ IONIC_XCVR_PID_QSFP_100G_CR4 = 1,
+ IONIC_XCVR_PID_QSFP_40GBASE_CR4 = 2,
+ IONIC_XCVR_PID_SFP_25GBASE_CR_S = 3,
+ IONIC_XCVR_PID_SFP_25GBASE_CR_L = 4,
+ IONIC_XCVR_PID_SFP_25GBASE_CR_N = 5,
+
+ /* Fiber */
+ IONIC_XCVR_PID_QSFP_100G_AOC = 50,
+ IONIC_XCVR_PID_QSFP_100G_ACC = 51,
+ IONIC_XCVR_PID_QSFP_100G_SR4 = 52,
+ IONIC_XCVR_PID_QSFP_100G_LR4 = 53,
+ IONIC_XCVR_PID_QSFP_100G_ER4 = 54,
+ IONIC_XCVR_PID_QSFP_40GBASE_ER4 = 55,
+ IONIC_XCVR_PID_QSFP_40GBASE_SR4 = 56,
+ IONIC_XCVR_PID_QSFP_40GBASE_LR4 = 57,
+ IONIC_XCVR_PID_QSFP_40GBASE_AOC = 58,
+ IONIC_XCVR_PID_SFP_25GBASE_SR = 59,
+ IONIC_XCVR_PID_SFP_25GBASE_LR = 60,
+ IONIC_XCVR_PID_SFP_25GBASE_ER = 61,
+ IONIC_XCVR_PID_SFP_25GBASE_AOC = 62,
+ IONIC_XCVR_PID_SFP_10GBASE_SR = 63,
+ IONIC_XCVR_PID_SFP_10GBASE_LR = 64,
+ IONIC_XCVR_PID_SFP_10GBASE_LRM = 65,
+ IONIC_XCVR_PID_SFP_10GBASE_ER = 66,
+ IONIC_XCVR_PID_SFP_10GBASE_AOC = 67,
+ IONIC_XCVR_PID_SFP_10GBASE_CU = 68,
+ IONIC_XCVR_PID_QSFP_100G_CWDM4 = 69,
+ IONIC_XCVR_PID_QSFP_100G_PSM4 = 70,
+};
+
+/**
+ * Port types
+ */
+enum ionic_port_type {
+ IONIC_PORT_TYPE_NONE = 0, /* port type not configured */
+ IONIC_PORT_TYPE_ETH = 1, /* port carries ethernet traffic (inband) */
+ IONIC_PORT_TYPE_MGMT = 2, /* port carries mgmt traffic (out-of-band) */
+};
+
+/**
+ * Port config state
+ */
+enum ionic_port_admin_state {
+ IONIC_PORT_ADMIN_STATE_NONE = 0, /* port admin state not configured */
+ IONIC_PORT_ADMIN_STATE_DOWN = 1, /* port is admin disabled */
+ IONIC_PORT_ADMIN_STATE_UP = 2, /* port is admin enabled */
+};
+
+/**
+ * Port operational status
+ */
+enum ionic_port_oper_status {
+ IONIC_PORT_OPER_STATUS_NONE = 0, /* port is disabled */
+ IONIC_PORT_OPER_STATUS_UP = 1, /* port is linked up */
+ IONIC_PORT_OPER_STATUS_DOWN = 2, /* port link status is down */
+};
+
+/**
+ * Ethernet Forward error correction (fec) modes
+ */
+enum ionic_port_fec_type {
+ IONIC_PORT_FEC_TYPE_NONE = 0, /* Disabled */
+ IONIC_PORT_FEC_TYPE_FC = 1, /* FireCode */
+ IONIC_PORT_FEC_TYPE_RS = 2, /* ReedSolomon */
+};
+
+/**
+ * Ethernet pause (flow control) modes
+ */
+enum ionic_port_pause_type {
+ IONIC_PORT_PAUSE_TYPE_NONE = 0, /* Disable Pause */
+ IONIC_PORT_PAUSE_TYPE_LINK = 1, /* Link level pause */
+ IONIC_PORT_PAUSE_TYPE_PFC = 2, /* Priority-Flow control */
+};
+
+/**
+ * Loopback modes
+ */
+enum ionic_port_loopback_mode {
+ IONIC_PORT_LOOPBACK_MODE_NONE = 0, /* Disable loopback */
+ IONIC_PORT_LOOPBACK_MODE_MAC = 1, /* MAC loopback */
+ IONIC_PORT_LOOPBACK_MODE_PHY = 2, /* PHY/Serdes loopback */
+};
+
+/**
+ * Transceiver Status information
+ * @state: Transceiver status (enum xcvr_state)
+ * @phy: Physical connection type (enum phy_type)
+ * @pid: Transceiver link mode (enum pid)
+ * @sprom: Transceiver sprom contents
+ */
+struct ionic_xcvr_status {
+ u8 state;
+ u8 phy;
+ __le16 pid;
+ u8 sprom[256];
+};
+
+/**
+ * Port configuration
+ * @speed: port speed (in Mbps)
+ * @mtu: mtu
+ * @state: port admin state (enum port_admin_state)
+ * @an_enable: autoneg enable
+ * @fec_type: fec type (enum port_fec_type)
+ * @pause_type: pause type (enum port_pause_type)
+ * @loopback_mode: loopback mode (enum port_loopback_mode)
+ */
+union ionic_port_config {
+ struct {
+#define IONIC_SPEED_100G 100000 /* 100G in Mbps */
+#define IONIC_SPEED_50G 50000 /* 50G in Mbps */
+#define IONIC_SPEED_40G 40000 /* 40G in Mbps */
+#define IONIC_SPEED_25G 25000 /* 25G in Mbps */
+#define IONIC_SPEED_10G 10000 /* 10G in Mbps */
+#define IONIC_SPEED_1G 1000 /* 1G in Mbps */
+ __le32 speed;
+ __le32 mtu;
+ u8 state;
+ u8 an_enable;
+ u8 fec_type;
+#define IONIC_PAUSE_TYPE_MASK 0x0f
+#define IONIC_PAUSE_FLAGS_MASK 0xf0
+#define IONIC_PAUSE_F_TX 0x10
+#define IONIC_PAUSE_F_RX 0x20
+ u8 pause_type;
+ u8 loopback_mode;
+ };
+ __le32 words[64];
+};
+
+/**
+ * Port Status information
+ * @status: link status (enum port_oper_status)
+ * @id: port id
+ * @speed: link speed (in Mbps)
+ * @xcvr: tranceiver status
+ */
+struct ionic_port_status {
+ __le32 id;
+ __le32 speed;
+ u8 status;
+ u8 rsvd[51];
+ struct ionic_xcvr_status xcvr;
+};
+
+/**
+ * struct port_identify_cmd - Port identify command
+ * @opcode: opcode
+ * @index: port index
+ * @ver: Highest version of identify supported by driver
+ */
+struct ionic_port_identify_cmd {
+ u8 opcode;
+ u8 index;
+ u8 ver;
+ u8 rsvd[61];
+};
+
+/**
+ * struct port_identify_comp - Port identify command completion
+ * @status: The status of the command (enum status_code)
+ * @ver: Version of identify returned by device
+ */
+struct ionic_port_identify_comp {
+ u8 status;
+ u8 ver;
+ u8 rsvd[14];
+};
+
+/**
+ * struct port_init_cmd - Port initialization command
+ * @opcode: opcode
+ * @index: port index
+ * @info_pa: destination address for port info (struct port_info)
+ */
+struct ionic_port_init_cmd {
+ u8 opcode;
+ u8 index;
+ u8 rsvd[6];
+ __le64 info_pa;
+ u8 rsvd2[48];
+};
+
+/**
+ * struct port_init_comp - Port initialization command completion
+ * @status: The status of the command (enum status_code)
+ */
+struct ionic_port_init_comp {
+ u8 status;
+ u8 rsvd[15];
+};
+
+/**
+ * struct port_reset_cmd - Port reset command
+ * @opcode: opcode
+ * @index: port index
+ */
+struct ionic_port_reset_cmd {
+ u8 opcode;
+ u8 index;
+ u8 rsvd[62];
+};
+
+/**
+ * struct port_reset_comp - Port reset command completion
+ * @status: The status of the command (enum status_code)
+ */
+struct ionic_port_reset_comp {
+ u8 status;
+ u8 rsvd[15];
+};
+
+/**
+ * enum stats_ctl_cmd - List of commands for stats control
+ */
+enum ionic_stats_ctl_cmd {
+ IONIC_STATS_CTL_RESET = 0,
+};
+
+
+/**
+ * enum ionic_port_attr - List of device attributes
+ */
+enum ionic_port_attr {
+ IONIC_PORT_ATTR_STATE = 0,
+ IONIC_PORT_ATTR_SPEED = 1,
+ IONIC_PORT_ATTR_MTU = 2,
+ IONIC_PORT_ATTR_AUTONEG = 3,
+ IONIC_PORT_ATTR_FEC = 4,
+ IONIC_PORT_ATTR_PAUSE = 5,
+ IONIC_PORT_ATTR_LOOPBACK = 6,
+ IONIC_PORT_ATTR_STATS_CTRL = 7,
+};
+
+/**
+ * struct port_setattr_cmd - Set port attributes on the NIC
+ * @opcode: Opcode
+ * @index: port index
+ * @attr: Attribute type (enum ionic_port_attr)
+ */
+struct ionic_port_setattr_cmd {
+ u8 opcode;
+ u8 index;
+ u8 attr;
+ u8 rsvd;
+ union {
+ u8 state;
+ __le32 speed;
+ __le32 mtu;
+ u8 an_enable;
+ u8 fec_type;
+ u8 pause_type;
+ u8 loopback_mode;
+ u8 stats_ctl;
+ u8 rsvd2[60];
+ };
+};
+
+/**
+ * struct port_setattr_comp - Port set attr command completion
+ * @status: The status of the command (enum status_code)
+ * @color: Color bit
+ */
+struct ionic_port_setattr_comp {
+ u8 status;
+ u8 rsvd[14];
+ u8 color;
+};
+
+/**
+ * struct port_getattr_cmd - Get port attributes from the NIC
+ * @opcode: Opcode
+ * @index: port index
+ * @attr: Attribute type (enum ionic_port_attr)
+ */
+struct ionic_port_getattr_cmd {
+ u8 opcode;
+ u8 index;
+ u8 attr;
+ u8 rsvd[61];
+};
+
+/**
+ * struct port_getattr_comp - Port get attr command completion
+ * @status: The status of the command (enum status_code)
+ * @color: Color bit
+ */
+struct ionic_port_getattr_comp {
+ u8 status;
+ u8 rsvd[3];
+ union {
+ u8 state;
+ __le32 speed;
+ __le32 mtu;
+ u8 an_enable;
+ u8 fec_type;
+ u8 pause_type;
+ u8 loopback_mode;
+ u8 rsvd2[11];
+ };
+ u8 color;
+};
+
+/**
+ * struct lif_status - Lif status register
+ * @eid: most recent NotifyQ event id
+ * @port_num: port the lif is connected to
+ * @link_status: port status (enum port_oper_status)
+ * @link_speed: speed of link in Mbps
+ * @link_down_count: number of times link status changes
+ */
+struct ionic_lif_status {
+ __le64 eid;
+ u8 port_num;
+ u8 rsvd;
+ __le16 link_status;
+ __le32 link_speed; /* units of 1Mbps: eg 10000 = 10Gbps */
+ __le16 link_down_count;
+ u8 rsvd2[46];
+};
+
+/**
+ * struct lif_reset_cmd - LIF reset command
+ * @opcode: opcode
+ * @index: LIF index
+ */
+struct ionic_lif_reset_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 index;
+ __le32 rsvd2[15];
+};
+
+typedef struct ionic_admin_comp ionic_lif_reset_comp;
+
+enum ionic_dev_state {
+ IONIC_DEV_DISABLE = 0,
+ IONIC_DEV_ENABLE = 1,
+ IONIC_DEV_HANG_RESET = 2,
+};
+
+/**
+ * enum dev_attr - List of device attributes
+ */
+enum ionic_dev_attr {
+ IONIC_DEV_ATTR_STATE = 0,
+ IONIC_DEV_ATTR_NAME = 1,
+ IONIC_DEV_ATTR_FEATURES = 2,
+};
+
+/**
+ * struct dev_setattr_cmd - Set Device attributes on the NIC
+ * @opcode: Opcode
+ * @attr: Attribute type (enum dev_attr)
+ * @state: Device state (enum dev_state)
+ * @name: The bus info, e.g. PCI slot-device-function, 0 terminated
+ * @features: Device features
+ */
+struct ionic_dev_setattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 rsvd;
+ union {
+ u8 state;
+ char name[IONIC_IFNAMSIZ];
+ __le64 features;
+ u8 rsvd2[60];
+ };
+};
+
+/**
+ * struct dev_setattr_comp - Device set attr command completion
+ * @status: The status of the command (enum status_code)
+ * @features: Device features
+ * @color: Color bit
+ */
+struct ionic_dev_setattr_comp {
+ u8 status;
+ u8 rsvd[3];
+ union {
+ __le64 features;
+ u8 rsvd2[11];
+ };
+ u8 color;
+};
+
+/**
+ * struct dev_getattr_cmd - Get Device attributes from the NIC
+ * @opcode: opcode
+ * @attr: Attribute type (enum dev_attr)
+ */
+struct ionic_dev_getattr_cmd {
+ u8 opcode;
+ u8 attr;
+ u8 rsvd[62];
+};
+
+/**
+ * struct dev_setattr_comp - Device set attr command completion
+ * @status: The status of the command (enum status_code)
+ * @features: Device features
+ * @color: Color bit
+ */
+struct ionic_dev_getattr_comp {
+ u8 status;
+ u8 rsvd[3];
+ union {
+ __le64 features;
+ u8 rsvd2[11];
+ };
+ u8 color;
+};
+
+/**
+ * RSS parameters
+ */
+#define IONIC_RSS_HASH_KEY_SIZE 40
+
+enum ionic_rss_hash_types {
+ IONIC_RSS_TYPE_IPV4 = BIT(0),
+ IONIC_RSS_TYPE_IPV4_TCP = BIT(1),
+ IONIC_RSS_TYPE_IPV4_UDP = BIT(2),
+ IONIC_RSS_TYPE_IPV6 = BIT(3),
+ IONIC_RSS_TYPE_IPV6_TCP = BIT(4),
+ IONIC_RSS_TYPE_IPV6_UDP = BIT(5),
+};
+
+/**
+ * enum lif_attr - List of LIF attributes
+ */
+enum ionic_lif_attr {
+ IONIC_LIF_ATTR_STATE = 0,
+ IONIC_LIF_ATTR_NAME = 1,
+ IONIC_LIF_ATTR_MTU = 2,
+ IONIC_LIF_ATTR_MAC = 3,
+ IONIC_LIF_ATTR_FEATURES = 4,
+ IONIC_LIF_ATTR_RSS = 5,
+ IONIC_LIF_ATTR_STATS_CTRL = 6,
+};
+
+/**
+ * struct lif_setattr_cmd - Set LIF attributes on the NIC
+ * @opcode: Opcode
+ * @type: Attribute type (enum lif_attr)
+ * @index: LIF index
+ * @state: lif state (enum lif_state)
+ * @name: The netdev name string, 0 terminated
+ * @mtu: Mtu
+ * @mac: Station mac
+ * @features: Features (enum eth_hw_features)
+ * @rss: RSS properties
+ * @types: The hash types to enable (see rss_hash_types).
+ * @key: The hash secret key.
+ * @addr: Address for the indirection table shared memory.
+ * @stats_ctl: stats control commands (enum stats_ctl_cmd)
+ */
+struct ionic_lif_setattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 index;
+ union {
+ u8 state;
+ char name[IONIC_IFNAMSIZ];
+ __le32 mtu;
+ u8 mac[6];
+ __le64 features;
+ struct {
+ __le16 types;
+ u8 key[IONIC_RSS_HASH_KEY_SIZE];
+ u8 rsvd[6];
+ __le64 addr;
+ } rss;
+ u8 stats_ctl;
+ u8 rsvd[60];
+ };
+};
+
+/**
+ * struct lif_setattr_comp - LIF set attr command completion
+ * @status: The status of the command (enum status_code)
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @features: features (enum eth_hw_features)
+ * @color: Color bit
+ */
+struct ionic_lif_setattr_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ union {
+ __le64 features;
+ u8 rsvd2[11];
+ };
+ u8 color;
+};
+
+/**
+ * struct lif_getattr_cmd - Get LIF attributes from the NIC
+ * @opcode: Opcode
+ * @attr: Attribute type (enum lif_attr)
+ * @index: LIF index
+ */
+struct ionic_lif_getattr_cmd {
+ u8 opcode;
+ u8 attr;
+ __le16 index;
+ u8 rsvd[60];
+};
+
+/**
+ * struct lif_getattr_comp - LIF get attr command completion
+ * @status: The status of the command (enum status_code)
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @state: lif state (enum lif_state)
+ * @name: The netdev name string, 0 terminated
+ * @mtu: Mtu
+ * @mac: Station mac
+ * @features: Features (enum eth_hw_features)
+ * @color: Color bit
+ */
+struct ionic_lif_getattr_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ union {
+ u8 state;
+ __le32 mtu;
+ u8 mac[6];
+ __le64 features;
+ u8 rsvd2[11];
+ };
+ u8 color;
+};
+
+enum ionic_rx_mode {
+ IONIC_RX_MODE_F_UNICAST = BIT(0),
+ IONIC_RX_MODE_F_MULTICAST = BIT(1),
+ IONIC_RX_MODE_F_BROADCAST = BIT(2),
+ IONIC_RX_MODE_F_PROMISC = BIT(3),
+ IONIC_RX_MODE_F_ALLMULTI = BIT(4),
+};
+
+/**
+ * struct rx_mode_set_cmd - Set LIF's Rx mode command
+ * @opcode: opcode
+ * @lif_index: LIF index
+ * @rx_mode: Rx mode flags:
+ * IONIC_RX_MODE_F_UNICAST: Accept known unicast packets.
+ * IONIC_RX_MODE_F_MULTICAST: Accept known multicast packets.
+ * IONIC_RX_MODE_F_BROADCAST: Accept broadcast packets.
+ * IONIC_RX_MODE_F_PROMISC: Accept any packets.
+ * IONIC_RX_MODE_F_ALLMULTI: Accept any multicast packets.
+ */
+struct ionic_rx_mode_set_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 lif_index;
+ __le16 rx_mode;
+ __le16 rsvd2[29];
+};
+
+typedef struct ionic_admin_comp ionic_rx_mode_set_comp;
+
+enum ionic_rx_filter_match_type {
+ IONIC_RX_FILTER_MATCH_VLAN = 0,
+ IONIC_RX_FILTER_MATCH_MAC,
+ IONIC_RX_FILTER_MATCH_MAC_VLAN,
+};
+
+/**
+ * struct rx_filter_add_cmd - Add LIF Rx filter command
+ * @opcode: opcode
+ * @qtype: Queue type
+ * @lif_index: LIF index
+ * @qid: Queue ID
+ * @match: Rx filter match type. (See IONIC_RX_FILTER_MATCH_xxx)
+ * @vlan: VLAN ID
+ * @addr: MAC address (network-byte order)
+ */
+struct ionic_rx_filter_add_cmd {
+ u8 opcode;
+ u8 qtype;
+ __le16 lif_index;
+ __le32 qid;
+ __le16 match;
+ union {
+ struct {
+ __le16 vlan;
+ } vlan;
+ struct {
+ u8 addr[6];
+ } mac;
+ struct {
+ __le16 vlan;
+ u8 addr[6];
+ } mac_vlan;
+ u8 rsvd[54];
+ };
+};
+
+/**
+ * struct rx_filter_add_comp - Add LIF Rx filter command completion
+ * @status: The status of the command (enum status_code)
+ * @comp_index: The index in the descriptor ring for which this
+ * is the completion.
+ * @filter_id: Filter ID
+ * @color: Color bit.
+ */
+struct ionic_rx_filter_add_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ __le32 filter_id;
+ u8 rsvd2[7];
+ u8 color;
+};
+
+/**
+ * struct rx_filter_del_cmd - Delete LIF Rx filter command
+ * @opcode: opcode
+ * @lif_index: LIF index
+ * @filter_id: Filter ID
+ */
+struct ionic_rx_filter_del_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 lif_index;
+ __le32 filter_id;
+ u8 rsvd2[56];
+};
+
+typedef struct ionic_admin_comp ionic_rx_filter_del_comp;
+
+/**
+ * struct qos_identify_cmd - QoS identify command
+ * @opcode: opcode
+ * @ver: Highest version of identify supported by driver
+ *
+ */
+struct ionic_qos_identify_cmd {
+ u8 opcode;
+ u8 ver;
+ u8 rsvd[62];
+};
+
+/**
+ * struct qos_identify_comp - QoS identify command completion
+ * @status: The status of the command (enum status_code)
+ * @ver: Version of identify returned by device
+ */
+struct ionic_qos_identify_comp {
+ u8 status;
+ u8 ver;
+ u8 rsvd[14];
+};
+
+#define IONIC_QOS_CLASS_MAX 7
+#define IONIC_QOS_CLASS_NAME_SZ 32
+#define IONIC_QOS_DSCP_MAX_VALUES 64
+
+/**
+ * enum qos_class
+ */
+enum ionic_qos_class {
+ IONIC_QOS_CLASS_DEFAULT = 0,
+ IONIC_QOS_CLASS_USER_DEFINED_1 = 1,
+ IONIC_QOS_CLASS_USER_DEFINED_2 = 2,
+ IONIC_QOS_CLASS_USER_DEFINED_3 = 3,
+ IONIC_QOS_CLASS_USER_DEFINED_4 = 4,
+ IONIC_QOS_CLASS_USER_DEFINED_5 = 5,
+ IONIC_QOS_CLASS_USER_DEFINED_6 = 6,
+};
+
+/**
+ * enum qos_class_type - Traffic classification criteria
+ */
+enum ionic_qos_class_type {
+ IONIC_QOS_CLASS_TYPE_NONE = 0,
+ IONIC_QOS_CLASS_TYPE_PCP = 1, /* Dot1Q pcp */
+ IONIC_QOS_CLASS_TYPE_DSCP = 2, /* IP dscp */
+};
+
+/**
+ * enum qos_sched_type - Qos class scheduling type
+ */
+enum ionic_qos_sched_type {
+ IONIC_QOS_SCHED_TYPE_STRICT = 0, /* Strict priority */
+ IONIC_QOS_SCHED_TYPE_DWRR = 1, /* Deficit weighted round-robin */
+};
+
+/**
+ * union qos_config - Qos configuration structure
+ * @flags: Configuration flags
+ * IONIC_QOS_CONFIG_F_ENABLE enable
+ * IONIC_QOS_CONFIG_F_DROP drop/nodrop
+ * IONIC_QOS_CONFIG_F_RW_DOT1Q_PCP enable dot1q pcp rewrite
+ * IONIC_QOS_CONFIG_F_RW_IP_DSCP enable ip dscp rewrite
+ * @sched_type: Qos class scheduling type (enum qos_sched_type)
+ * @class_type: Qos class type (enum qos_class_type)
+ * @pause_type: Qos pause type (enum qos_pause_type)
+ * @name: Qos class name
+ * @mtu: MTU of the class
+ * @pfc_dot1q_pcp: Pcp value for pause frames (valid iff F_NODROP)
+ * @dwrr_weight: Qos class scheduling weight
+ * @strict_rlmt: Rate limit for strict priority scheduling
+ * @rw_dot1q_pcp: Rewrite dot1q pcp to this value (valid iff F_RW_DOT1Q_PCP)
+ * @rw_ip_dscp: Rewrite ip dscp to this value (valid iff F_RW_IP_DSCP)
+ * @dot1q_pcp: Dot1q pcp value
+ * @ndscp: Number of valid dscp values in the ip_dscp field
+ * @ip_dscp: IP dscp values
+ */
+union ionic_qos_config {
+ struct {
+#define IONIC_QOS_CONFIG_F_ENABLE BIT(0)
+#define IONIC_QOS_CONFIG_F_DROP BIT(1)
+#define IONIC_QOS_CONFIG_F_RW_DOT1Q_PCP BIT(2)
+#define IONIC_QOS_CONFIG_F_RW_IP_DSCP BIT(3)
+ u8 flags;
+ u8 sched_type;
+ u8 class_type;
+ u8 pause_type;
+ char name[IONIC_QOS_CLASS_NAME_SZ];
+ __le32 mtu;
+ /* flow control */
+ u8 pfc_cos;
+ /* scheduler */
+ union {
+ u8 dwrr_weight;
+ __le64 strict_rlmt;
+ };
+ /* marking */
+ union {
+ u8 rw_dot1q_pcp;
+ u8 rw_ip_dscp;
+ };
+ /* classification */
+ union {
+ u8 dot1q_pcp;
+ struct {
+ u8 ndscp;
+ u8 ip_dscp[IONIC_QOS_DSCP_MAX_VALUES];
+ };
+ };
+ };
+ __le32 words[64];
+};
+
+/**
+ * union qos_identity - QoS identity structure
+ * @version: Version of the identify structure
+ * @type: QoS system type
+ * @nclasses: Number of usable QoS classes
+ * @config: Current configuration of classes
+ */
+union ionic_qos_identity {
+ struct {
+ u8 version;
+ u8 type;
+ u8 rsvd[62];
+ union ionic_qos_config config[IONIC_QOS_CLASS_MAX];
+ };
+ __le32 words[512];
+};
+
+/**
+ * struct qos_init_cmd - QoS config init command
+ * @opcode: Opcode
+ * @group: Qos class id
+ * @info_pa: destination address for qos info
+ */
+struct ionic_qos_init_cmd {
+ u8 opcode;
+ u8 group;
+ u8 rsvd[6];
+ __le64 info_pa;
+ u8 rsvd1[48];
+};
+
+typedef struct ionic_admin_comp ionic_qos_init_comp;
+
+/**
+ * struct qos_reset_cmd - Qos config reset command
+ * @opcode: Opcode
+ */
+struct ionic_qos_reset_cmd {
+ u8 opcode;
+ u8 group;
+ u8 rsvd[62];
+};
+
+typedef struct ionic_admin_comp ionic_qos_reset_comp;
+
+/**
+ * struct fw_download_cmd - Firmware download command
+ * @opcode: opcode
+ * @addr: dma address of the firmware buffer
+ * @offset: offset of the firmware buffer within the full image
+ * @length: number of valid bytes in the firmware buffer
+ */
+struct ionic_fw_download_cmd {
+ u8 opcode;
+ u8 rsvd[3];
+ __le32 offset;
+ __le64 addr;
+ __le32 length;
+};
+
+typedef struct ionic_admin_comp ionic_fw_download_comp;
+
+enum ionic_fw_control_oper {
+ IONIC_FW_RESET = 0, /* Reset firmware */
+ IONIC_FW_INSTALL = 1, /* Install firmware */
+ IONIC_FW_ACTIVATE = 2, /* Activate firmware */
+};
+
+/**
+ * struct fw_control_cmd - Firmware control command
+ * @opcode: opcode
+ * @oper: firmware control operation (enum fw_control_oper)
+ * @slot: slot to activate
+ */
+struct ionic_fw_control_cmd {
+ u8 opcode;
+ u8 rsvd[3];
+ u8 oper;
+ u8 slot;
+ u8 rsvd1[58];
+};
+
+/**
+ * struct fw_control_comp - Firmware control copletion
+ * @opcode: opcode
+ * @slot: slot where the firmware was installed
+ */
+struct ionic_fw_control_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ u8 slot;
+ u8 rsvd1[10];
+ u8 color;
+};
+
+/******************************************************************
+ ******************* RDMA Commands ********************************
+ ******************************************************************/
+
+/**
+ * struct rdma_reset_cmd - Reset RDMA LIF cmd
+ * @opcode: opcode
+ * @lif_index: lif index
+ *
+ * There is no rdma specific dev command completion struct. Completion uses
+ * the common struct admin_comp. Only the status is indicated. Nonzero status
+ * means the LIF does not support rdma.
+ **/
+struct ionic_rdma_reset_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 lif_index;
+ u8 rsvd2[60];
+};
+
+/**
+ * struct rdma_queue_cmd - Create RDMA Queue command
+ * @opcode: opcode, 52, 53
+ * @lif_index lif index
+ * @qid_ver: (qid | (rdma version << 24))
+ * @cid: intr, eq_id, or cq_id
+ * @dbid: doorbell page id
+ * @depth_log2: log base two of queue depth
+ * @stride_log2: log base two of queue stride
+ * @dma_addr: address of the queue memory
+ * @xxx_table_index: temporary, but should not need pgtbl for contig. queues.
+ *
+ * The same command struct is used to create an rdma event queue, completion
+ * queue, or rdma admin queue. The cid is an interrupt number for an event
+ * queue, an event queue id for a completion queue, or a completion queue id
+ * for an rdma admin queue.
+ *
+ * The queue created via a dev command must be contiguous in dma space.
+ *
+ * The dev commands are intended only to be used during driver initialization,
+ * to create queues supporting the rdma admin queue. Other queues, and other
+ * types of rdma resources like memory regions, will be created and registered
+ * via the rdma admin queue, and will support a more complete interface
+ * providing scatter gather lists for larger, scattered queue buffers and
+ * memory registration.
+ *
+ * There is no rdma specific dev command completion struct. Completion uses
+ * the common struct admin_comp. Only the status is indicated.
+ **/
+struct ionic_rdma_queue_cmd {
+ u8 opcode;
+ u8 rsvd;
+ __le16 lif_index;
+ __le32 qid_ver;
+ __le32 cid;
+ __le16 dbid;
+ u8 depth_log2;
+ u8 stride_log2;
+ __le64 dma_addr;
+ u8 rsvd2[36];
+ __le32 xxx_table_index;
+};
+
+/******************************************************************
+ ******************* Notify Events ********************************
+ ******************************************************************/
+
+/**
+ * struct notifyq_event
+ * @eid: event number
+ * @ecode: event code
+ * @data: unspecified data about the event
+ *
+ * This is the generic event report struct from which the other
+ * actual events will be formed.
+ */
+struct ionic_notifyq_event {
+ __le64 eid;
+ __le16 ecode;
+ u8 data[54];
+};
+
+/**
+ * struct link_change_event
+ * @eid: event number
+ * @ecode: event code = EVENT_OPCODE_LINK_CHANGE
+ * @link_status: link up or down, with error bits (enum port_status)
+ * @link_speed: speed of the network link
+ *
+ * Sent when the network link state changes between UP and DOWN
+ */
+struct ionic_link_change_event {
+ __le64 eid;
+ __le16 ecode;
+ __le16 link_status;
+ __le32 link_speed; /* units of 1Mbps: e.g. 10000 = 10Gbps */
+ u8 rsvd[48];
+};
+
+/**
+ * struct reset_event
+ * @eid: event number
+ * @ecode: event code = EVENT_OPCODE_RESET
+ * @reset_code: reset type
+ * @state: 0=pending, 1=complete, 2=error
+ *
+ * Sent when the NIC or some subsystem is going to be or
+ * has been reset.
+ */
+struct ionic_reset_event {
+ __le64 eid;
+ __le16 ecode;
+ u8 reset_code;
+ u8 state;
+ u8 rsvd[52];
+};
+
+/**
+ * struct heartbeat_event
+ * @eid: event number
+ * @ecode: event code = EVENT_OPCODE_HEARTBEAT
+ *
+ * Sent periodically by the NIC to indicate continued health
+ */
+struct ionic_heartbeat_event {
+ __le64 eid;
+ __le16 ecode;
+ u8 rsvd[54];
+};
+
+/**
+ * struct log_event
+ * @eid: event number
+ * @ecode: event code = EVENT_OPCODE_LOG
+ * @data: log data
+ *
+ * Sent to notify the driver of an internal error.
+ */
+struct ionic_log_event {
+ __le64 eid;
+ __le16 ecode;
+ u8 data[54];
+};
+
+/**
+ * struct port_stats
+ */
+struct ionic_port_stats {
+ __le64 frames_rx_ok;
+ __le64 frames_rx_all;
+ __le64 frames_rx_bad_fcs;
+ __le64 frames_rx_bad_all;
+ __le64 octets_rx_ok;
+ __le64 octets_rx_all;
+ __le64 frames_rx_unicast;
+ __le64 frames_rx_multicast;
+ __le64 frames_rx_broadcast;
+ __le64 frames_rx_pause;
+ __le64 frames_rx_bad_length;
+ __le64 frames_rx_undersized;
+ __le64 frames_rx_oversized;
+ __le64 frames_rx_fragments;
+ __le64 frames_rx_jabber;
+ __le64 frames_rx_pripause;
+ __le64 frames_rx_stomped_crc;
+ __le64 frames_rx_too_long;
+ __le64 frames_rx_vlan_good;
+ __le64 frames_rx_dropped;
+ __le64 frames_rx_less_than_64b;
+ __le64 frames_rx_64b;
+ __le64 frames_rx_65b_127b;
+ __le64 frames_rx_128b_255b;
+ __le64 frames_rx_256b_511b;
+ __le64 frames_rx_512b_1023b;
+ __le64 frames_rx_1024b_1518b;
+ __le64 frames_rx_1519b_2047b;
+ __le64 frames_rx_2048b_4095b;
+ __le64 frames_rx_4096b_8191b;
+ __le64 frames_rx_8192b_9215b;
+ __le64 frames_rx_other;
+ __le64 frames_tx_ok;
+ __le64 frames_tx_all;
+ __le64 frames_tx_bad;
+ __le64 octets_tx_ok;
+ __le64 octets_tx_total;
+ __le64 frames_tx_unicast;
+ __le64 frames_tx_multicast;
+ __le64 frames_tx_broadcast;
+ __le64 frames_tx_pause;
+ __le64 frames_tx_pripause;
+ __le64 frames_tx_vlan;
+ __le64 frames_tx_less_than_64b;
+ __le64 frames_tx_64b;
+ __le64 frames_tx_65b_127b;
+ __le64 frames_tx_128b_255b;
+ __le64 frames_tx_256b_511b;
+ __le64 frames_tx_512b_1023b;
+ __le64 frames_tx_1024b_1518b;
+ __le64 frames_tx_1519b_2047b;
+ __le64 frames_tx_2048b_4095b;
+ __le64 frames_tx_4096b_8191b;
+ __le64 frames_tx_8192b_9215b;
+ __le64 frames_tx_other;
+ __le64 frames_tx_pri_0;
+ __le64 frames_tx_pri_1;
+ __le64 frames_tx_pri_2;
+ __le64 frames_tx_pri_3;
+ __le64 frames_tx_pri_4;
+ __le64 frames_tx_pri_5;
+ __le64 frames_tx_pri_6;
+ __le64 frames_tx_pri_7;
+ __le64 frames_rx_pri_0;
+ __le64 frames_rx_pri_1;
+ __le64 frames_rx_pri_2;
+ __le64 frames_rx_pri_3;
+ __le64 frames_rx_pri_4;
+ __le64 frames_rx_pri_5;
+ __le64 frames_rx_pri_6;
+ __le64 frames_rx_pri_7;
+ __le64 tx_pripause_0_1us_count;
+ __le64 tx_pripause_1_1us_count;
+ __le64 tx_pripause_2_1us_count;
+ __le64 tx_pripause_3_1us_count;
+ __le64 tx_pripause_4_1us_count;
+ __le64 tx_pripause_5_1us_count;
+ __le64 tx_pripause_6_1us_count;
+ __le64 tx_pripause_7_1us_count;
+ __le64 rx_pripause_0_1us_count;
+ __le64 rx_pripause_1_1us_count;
+ __le64 rx_pripause_2_1us_count;
+ __le64 rx_pripause_3_1us_count;
+ __le64 rx_pripause_4_1us_count;
+ __le64 rx_pripause_5_1us_count;
+ __le64 rx_pripause_6_1us_count;
+ __le64 rx_pripause_7_1us_count;
+ __le64 rx_pause_1us_count;
+ __le64 frames_tx_truncated;
+};
+
+struct ionic_mgmt_port_stats {
+ __le64 frames_rx_ok;
+ __le64 frames_rx_all;
+ __le64 frames_rx_bad_fcs;
+ __le64 frames_rx_bad_all;
+ __le64 octets_rx_ok;
+ __le64 octets_rx_all;
+ __le64 frames_rx_unicast;
+ __le64 frames_rx_multicast;
+ __le64 frames_rx_broadcast;
+ __le64 frames_rx_pause;
+ __le64 frames_rx_bad_length0;
+ __le64 frames_rx_undersized1;
+ __le64 frames_rx_oversized2;
+ __le64 frames_rx_fragments3;
+ __le64 frames_rx_jabber4;
+ __le64 frames_rx_64b5;
+ __le64 frames_rx_65b_127b6;
+ __le64 frames_rx_128b_255b7;
+ __le64 frames_rx_256b_511b8;
+ __le64 frames_rx_512b_1023b9;
+ __le64 frames_rx_1024b_1518b0;
+ __le64 frames_rx_gt_1518b1;
+ __le64 frames_rx_fifo_full2;
+ __le64 frames_tx_ok3;
+ __le64 frames_tx_all4;
+ __le64 frames_tx_bad5;
+ __le64 octets_tx_ok6;
+ __le64 octets_tx_total7;
+ __le64 frames_tx_unicast8;
+ __le64 frames_tx_multicast9;
+ __le64 frames_tx_broadcast0;
+ __le64 frames_tx_pause1;
+};
+
+/**
+ * struct port_identity - port identity structure
+ * @version: identity structure version
+ * @type: type of port (enum port_type)
+ * @num_lanes: number of lanes for the port
+ * @autoneg: autoneg supported
+ * @min_frame_size: minimum frame size supported
+ * @max_frame_size: maximum frame size supported
+ * @fec_type: supported fec types
+ * @pause_type: supported pause types
+ * @loopback_mode: supported loopback mode
+ * @speeds: supported speeds
+ * @config: current port configuration
+ */
+union ionic_port_identity {
+ struct {
+ u8 version;
+ u8 type;
+ u8 num_lanes;
+ u8 autoneg;
+ __le32 min_frame_size;
+ __le32 max_frame_size;
+ u8 fec_type[4];
+ u8 pause_type[2];
+ u8 loopback_mode[2];
+ __le32 speeds[16];
+ u8 rsvd2[44];
+ union ionic_port_config config;
+ };
+ __le32 words[512];
+};
+
+/**
+ * struct port_info - port info structure
+ * @port_status: port status
+ * @port_stats: port stats
+ */
+struct ionic_port_info {
+ union ionic_port_config config;
+ struct ionic_port_status status;
+ struct ionic_port_stats stats;
+};
+
+/**
+ * struct lif_stats
+ */
+struct ionic_lif_stats {
+ /* RX */
+ __le64 rx_ucast_bytes;
+ __le64 rx_ucast_packets;
+ __le64 rx_mcast_bytes;
+ __le64 rx_mcast_packets;
+ __le64 rx_bcast_bytes;
+ __le64 rx_bcast_packets;
+ __le64 rsvd0;
+ __le64 rsvd1;
+ /* RX drops */
+ __le64 rx_ucast_drop_bytes;
+ __le64 rx_ucast_drop_packets;
+ __le64 rx_mcast_drop_bytes;
+ __le64 rx_mcast_drop_packets;
+ __le64 rx_bcast_drop_bytes;
+ __le64 rx_bcast_drop_packets;
+ __le64 rx_dma_error;
+ __le64 rsvd2;
+ /* TX */
+ __le64 tx_ucast_bytes;
+ __le64 tx_ucast_packets;
+ __le64 tx_mcast_bytes;
+ __le64 tx_mcast_packets;
+ __le64 tx_bcast_bytes;
+ __le64 tx_bcast_packets;
+ __le64 rsvd3;
+ __le64 rsvd4;
+ /* TX drops */
+ __le64 tx_ucast_drop_bytes;
+ __le64 tx_ucast_drop_packets;
+ __le64 tx_mcast_drop_bytes;
+ __le64 tx_mcast_drop_packets;
+ __le64 tx_bcast_drop_bytes;
+ __le64 tx_bcast_drop_packets;
+ __le64 tx_dma_error;
+ __le64 rsvd5;
+ /* Rx Queue/Ring drops */
+ __le64 rx_queue_disabled;
+ __le64 rx_queue_empty;
+ __le64 rx_queue_error;
+ __le64 rx_desc_fetch_error;
+ __le64 rx_desc_data_error;
+ __le64 rsvd6;
+ __le64 rsvd7;
+ __le64 rsvd8;
+ /* Tx Queue/Ring drops */
+ __le64 tx_queue_disabled;
+ __le64 tx_queue_error;
+ __le64 tx_desc_fetch_error;
+ __le64 tx_desc_data_error;
+ __le64 rsvd9;
+ __le64 rsvd10;
+ __le64 rsvd11;
+ __le64 rsvd12;
+
+ /* RDMA/ROCE TX */
+ __le64 tx_rdma_ucast_bytes;
+ __le64 tx_rdma_ucast_packets;
+ __le64 tx_rdma_mcast_bytes;
+ __le64 tx_rdma_mcast_packets;
+ __le64 tx_rdma_cnp_packets;
+ __le64 rsvd13;
+ __le64 rsvd14;
+ __le64 rsvd15;
+
+ /* RDMA/ROCE RX */
+ __le64 rx_rdma_ucast_bytes;
+ __le64 rx_rdma_ucast_packets;
+ __le64 rx_rdma_mcast_bytes;
+ __le64 rx_rdma_mcast_packets;
+ __le64 rx_rdma_cnp_packets;
+ __le64 rx_rdma_ecn_packets;
+ __le64 rsvd16;
+ __le64 rsvd17;
+
+ __le64 rsvd18;
+ __le64 rsvd19;
+ __le64 rsvd20;
+ __le64 rsvd21;
+ __le64 rsvd22;
+ __le64 rsvd23;
+ __le64 rsvd24;
+ __le64 rsvd25;
+
+ __le64 rsvd26;
+ __le64 rsvd27;
+ __le64 rsvd28;
+ __le64 rsvd29;
+ __le64 rsvd30;
+ __le64 rsvd31;
+ __le64 rsvd32;
+ __le64 rsvd33;
+
+ __le64 rsvd34;
+ __le64 rsvd35;
+ __le64 rsvd36;
+ __le64 rsvd37;
+ __le64 rsvd38;
+ __le64 rsvd39;
+ __le64 rsvd40;
+ __le64 rsvd41;
+
+ __le64 rsvd42;
+ __le64 rsvd43;
+ __le64 rsvd44;
+ __le64 rsvd45;
+ __le64 rsvd46;
+ __le64 rsvd47;
+ __le64 rsvd48;
+ __le64 rsvd49;
+
+ /* RDMA/ROCE REQ Error/Debugs (768 - 895) */
+ __le64 rdma_req_rx_pkt_seq_err;
+ __le64 rdma_req_rx_rnr_retry_err;
+ __le64 rdma_req_rx_remote_access_err;
+ __le64 rdma_req_rx_remote_inv_req_err;
+ __le64 rdma_req_rx_remote_oper_err;
+ __le64 rdma_req_rx_implied_nak_seq_err;
+ __le64 rdma_req_rx_cqe_err;
+ __le64 rdma_req_rx_cqe_flush_err;
+
+ __le64 rdma_req_rx_dup_responses;
+ __le64 rdma_req_rx_invalid_packets;
+ __le64 rdma_req_tx_local_access_err;
+ __le64 rdma_req_tx_local_oper_err;
+ __le64 rdma_req_tx_memory_mgmt_err;
+ __le64 rsvd52;
+ __le64 rsvd53;
+ __le64 rsvd54;
+
+ /* RDMA/ROCE RESP Error/Debugs (896 - 1023) */
+ __le64 rdma_resp_rx_dup_requests;
+ __le64 rdma_resp_rx_out_of_buffer;
+ __le64 rdma_resp_rx_out_of_seq_pkts;
+ __le64 rdma_resp_rx_cqe_err;
+ __le64 rdma_resp_rx_cqe_flush_err;
+ __le64 rdma_resp_rx_local_len_err;
+ __le64 rdma_resp_rx_inv_request_err;
+ __le64 rdma_resp_rx_local_qp_oper_err;
+
+ __le64 rdma_resp_rx_out_of_atomic_resource;
+ __le64 rdma_resp_tx_pkt_seq_err;
+ __le64 rdma_resp_tx_remote_inv_req_err;
+ __le64 rdma_resp_tx_remote_access_err;
+ __le64 rdma_resp_tx_remote_oper_err;
+ __le64 rdma_resp_tx_rnr_retry_err;
+ __le64 rsvd57;
+ __le64 rsvd58;
+};
+
+/**
+ * struct lif_info - lif info structure
+ */
+struct ionic_lif_info {
+ union ionic_lif_config config;
+ struct ionic_lif_status status;
+ struct ionic_lif_stats stats;
+};
+
+union ionic_dev_cmd {
+ u32 words[16];
+ struct ionic_admin_cmd cmd;
+ struct ionic_nop_cmd nop;
+
+ struct ionic_dev_identify_cmd identify;
+ struct ionic_dev_init_cmd init;
+ struct ionic_dev_reset_cmd reset;
+ struct ionic_dev_getattr_cmd getattr;
+ struct ionic_dev_setattr_cmd setattr;
+
+ struct ionic_port_identify_cmd port_identify;
+ struct ionic_port_init_cmd port_init;
+ struct ionic_port_reset_cmd port_reset;
+ struct ionic_port_getattr_cmd port_getattr;
+ struct ionic_port_setattr_cmd port_setattr;
+
+ struct ionic_lif_identify_cmd lif_identify;
+ struct ionic_lif_init_cmd lif_init;
+ struct ionic_lif_reset_cmd lif_reset;
+
+ struct ionic_qos_identify_cmd qos_identify;
+ struct ionic_qos_init_cmd qos_init;
+ struct ionic_qos_reset_cmd qos_reset;
+
+ struct ionic_q_init_cmd q_init;
+};
+
+union ionic_dev_cmd_comp {
+ u32 words[4];
+ u8 status;
+ struct ionic_admin_comp comp;
+ struct ionic_nop_comp nop;
+
+ struct ionic_dev_identify_comp identify;
+ struct ionic_dev_init_comp init;
+ struct ionic_dev_reset_comp reset;
+ struct ionic_dev_getattr_comp getattr;
+ struct ionic_dev_setattr_comp setattr;
+
+ struct ionic_port_identify_comp port_identify;
+ struct ionic_port_init_comp port_init;
+ struct ionic_port_reset_comp port_reset;
+ struct ionic_port_getattr_comp port_getattr;
+ struct ionic_port_setattr_comp port_setattr;
+
+ struct ionic_lif_identify_comp lif_identify;
+ struct ionic_lif_init_comp lif_init;
+ ionic_lif_reset_comp lif_reset;
+
+ struct ionic_qos_identify_comp qos_identify;
+ ionic_qos_init_comp qos_init;
+ ionic_qos_reset_comp qos_reset;
+
+ struct ionic_q_init_comp q_init;
+};
+
+/**
+ * union dev_info - Device info register format (read-only)
+ * @signature: Signature value of 0x44455649 ('DEVI').
+ * @version: Current version of info.
+ * @asic_type: Asic type.
+ * @asic_rev: Asic revision.
+ * @fw_status: Firmware status.
+ * @fw_heartbeat: Firmware heartbeat counter.
+ * @serial_num: Serial number.
+ * @fw_version: Firmware version.
+ */
+union ionic_dev_info_regs {
+#define IONIC_DEVINFO_FWVERS_BUFLEN 32
+#define IONIC_DEVINFO_SERIAL_BUFLEN 32
+ struct {
+ u32 signature;
+ u8 version;
+ u8 asic_type;
+ u8 asic_rev;
+ u8 fw_status;
+ u32 fw_heartbeat;
+ char fw_version[IONIC_DEVINFO_FWVERS_BUFLEN];
+ char serial_num[IONIC_DEVINFO_SERIAL_BUFLEN];
+ };
+ u32 words[512];
+};
+
+/**
+ * union dev_cmd_regs - Device command register format (read-write)
+ * @doorbell: Device Cmd Doorbell, write-only.
+ * Write a 1 to signal device to process cmd,
+ * poll done for completion.
+ * @done: Done indicator, bit 0 == 1 when command is complete.
+ * @cmd: Opcode-specific command bytes
+ * @comp: Opcode-specific response bytes
+ * @data: Opcode-specific side-data
+ */
+union ionic_dev_cmd_regs {
+ struct {
+ u32 doorbell;
+ u32 done;
+ union ionic_dev_cmd cmd;
+ union ionic_dev_cmd_comp comp;
+ u8 rsvd[48];
+ u32 data[478];
+ };
+ u32 words[512];
+};
+
+/**
+ * union dev_regs - Device register format in for bar 0 page 0
+ * @info: Device info registers
+ * @devcmd: Device command registers
+ */
+union ionic_dev_regs {
+ struct {
+ union ionic_dev_info_regs info;
+ union ionic_dev_cmd_regs devcmd;
+ };
+ __le32 words[1024];
+};
+
+union ionic_adminq_cmd {
+ struct ionic_admin_cmd cmd;
+ struct ionic_nop_cmd nop;
+ struct ionic_q_init_cmd q_init;
+ struct ionic_q_control_cmd q_control;
+ struct ionic_lif_setattr_cmd lif_setattr;
+ struct ionic_lif_getattr_cmd lif_getattr;
+ struct ionic_rx_mode_set_cmd rx_mode_set;
+ struct ionic_rx_filter_add_cmd rx_filter_add;
+ struct ionic_rx_filter_del_cmd rx_filter_del;
+ struct ionic_rdma_reset_cmd rdma_reset;
+ struct ionic_rdma_queue_cmd rdma_queue;
+ struct ionic_fw_download_cmd fw_download;
+ struct ionic_fw_control_cmd fw_control;
+};
+
+union ionic_adminq_comp {
+ struct ionic_admin_comp comp;
+ struct ionic_nop_comp nop;
+ struct ionic_q_init_comp q_init;
+ struct ionic_lif_setattr_comp lif_setattr;
+ struct ionic_lif_getattr_comp lif_getattr;
+ struct ionic_rx_filter_add_comp rx_filter_add;
+ struct ionic_fw_control_comp fw_control;
+};
+
+#define IONIC_BARS_MAX 6
+#define IONIC_PCI_BAR_DBELL 1
+
+/* BAR0 */
+#define IONIC_BAR0_SIZE 0x8000
+
+#define IONIC_BAR0_DEV_INFO_REGS_OFFSET 0x0000
+#define IONIC_BAR0_DEV_CMD_REGS_OFFSET 0x0800
+#define IONIC_BAR0_DEV_CMD_DATA_REGS_OFFSET 0x0c00
+#define IONIC_BAR0_INTR_STATUS_OFFSET 0x1000
+#define IONIC_BAR0_INTR_CTRL_OFFSET 0x2000
+#define IONIC_DEV_CMD_DONE 0x00000001
+
+#define IONIC_ASIC_TYPE_CAPRI 0
+
+/**
+ * struct doorbell - Doorbell register layout
+ * @p_index: Producer index
+ * @ring: Selects the specific ring of the queue to update.
+ * Type-specific meaning:
+ * ring=0: Default producer/consumer queue.
+ * ring=1: (CQ, EQ) Re-Arm queue. RDMA CQs
+ * send events to EQs when armed. EQs send
+ * interrupts when armed.
+ * @qid: The queue id selects the queue destination for the
+ * producer index and flags.
+ */
+struct ionic_doorbell {
+ __le16 p_index;
+ u8 ring;
+ u8 qid_lo;
+ __le16 qid_hi;
+ u16 rsvd2;
+};
+
+struct ionic_intr_status {
+ u32 status[2];
+};
+
+struct ionic_notifyq_cmd {
+ __le32 data; /* Not used but needed for qcq structure */
+};
+
+union ionic_notifyq_comp {
+ struct ionic_notifyq_event event;
+ struct ionic_link_change_event link_change;
+ struct ionic_reset_event reset;
+ struct ionic_heartbeat_event heartbeat;
+ struct ionic_log_event log;
+};
+
+/* Deprecate */
+struct ionic_identity {
+ union ionic_drv_identity drv;
+ union ionic_dev_identity dev;
+ union ionic_lif_identity lif;
+ union ionic_port_identity port;
+ union ionic_qos_identity qos;
+};
+
+#pragma pack(pop)
+
+#endif /* _IONIC_IF_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
new file mode 100644
index 0000000..db7c827
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -0,0 +1,2274 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/rtnetlink.h>
+#include <linux/interrupt.h>
+#include <linux/pci.h>
+#include <linux/cpumask.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_txrx.h"
+#include "ionic_ethtool.h"
+#include "ionic_debugfs.h"
+
+static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode);
+static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr);
+static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr);
+static void ionic_link_status_check(struct ionic_lif *lif);
+
+static void ionic_lif_deferred_work(struct work_struct *work)
+{
+ struct ionic_lif *lif = container_of(work, struct ionic_lif, deferred.work);
+ struct ionic_deferred *def = &lif->deferred;
+ struct ionic_deferred_work *w = NULL;
+
+ spin_lock_bh(&def->lock);
+ if (!list_empty(&def->list)) {
+ w = list_first_entry(&def->list,
+ struct ionic_deferred_work, list);
+ list_del(&w->list);
+ }
+ spin_unlock_bh(&def->lock);
+
+ if (w) {
+ switch (w->type) {
+ case IONIC_DW_TYPE_RX_MODE:
+ ionic_lif_rx_mode(lif, w->rx_mode);
+ break;
+ case IONIC_DW_TYPE_RX_ADDR_ADD:
+ ionic_lif_addr_add(lif, w->addr);
+ break;
+ case IONIC_DW_TYPE_RX_ADDR_DEL:
+ ionic_lif_addr_del(lif, w->addr);
+ break;
+ case IONIC_DW_TYPE_LINK_STATUS:
+ ionic_link_status_check(lif);
+ break;
+ default:
+ break;
+ }
+ kfree(w);
+ schedule_work(&def->work);
+ }
+}
+
+static void ionic_lif_deferred_enqueue(struct ionic_deferred *def,
+ struct ionic_deferred_work *work)
+{
+ spin_lock_bh(&def->lock);
+ list_add_tail(&work->list, &def->list);
+ spin_unlock_bh(&def->lock);
+ schedule_work(&def->work);
+}
+
+static void ionic_link_status_check(struct ionic_lif *lif)
+{
+ struct net_device *netdev = lif->netdev;
+ u16 link_status;
+ bool link_up;
+
+ link_status = le16_to_cpu(lif->info->status.link_status);
+ link_up = link_status == IONIC_PORT_OPER_STATUS_UP;
+
+ /* filter out the no-change cases */
+ if (link_up == netif_carrier_ok(netdev))
+ goto link_out;
+
+ if (link_up) {
+ netdev_info(netdev, "Link up - %d Gbps\n",
+ le32_to_cpu(lif->info->status.link_speed) / 1000);
+
+ if (test_bit(IONIC_LIF_UP, lif->state)) {
+ netif_tx_wake_all_queues(lif->netdev);
+ netif_carrier_on(netdev);
+ }
+ } else {
+ netdev_info(netdev, "Link down\n");
+
+ /* carrier off first to avoid watchdog timeout */
+ netif_carrier_off(netdev);
+ if (test_bit(IONIC_LIF_UP, lif->state))
+ netif_tx_stop_all_queues(netdev);
+ }
+
+link_out:
+ clear_bit(IONIC_LIF_LINK_CHECK_REQUESTED, lif->state);
+}
+
+static void ionic_link_status_check_request(struct ionic_lif *lif)
+{
+ struct ionic_deferred_work *work;
+
+ /* we only need one request outstanding at a time */
+ if (test_and_set_bit(IONIC_LIF_LINK_CHECK_REQUESTED, lif->state))
+ return;
+
+ if (in_interrupt()) {
+ work = kzalloc(sizeof(*work), GFP_ATOMIC);
+ if (!work)
+ return;
+
+ work->type = IONIC_DW_TYPE_LINK_STATUS;
+ ionic_lif_deferred_enqueue(&lif->deferred, work);
+ } else {
+ ionic_link_status_check(lif);
+ }
+}
+
+static irqreturn_t ionic_isr(int irq, void *data)
+{
+ struct napi_struct *napi = data;
+
+ napi_schedule_irqoff(napi);
+
+ return IRQ_HANDLED;
+}
+
+static int ionic_request_irq(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+ struct ionic_intr_info *intr = &qcq->intr;
+ struct device *dev = lif->ionic->dev;
+ struct ionic_queue *q = &qcq->q;
+ const char *name;
+
+ if (lif->registered)
+ name = lif->netdev->name;
+ else
+ name = dev_name(dev);
+
+ snprintf(intr->name, sizeof(intr->name),
+ "%s-%s-%s", IONIC_DRV_NAME, name, q->name);
+
+ return devm_request_irq(dev, intr->vector, ionic_isr,
+ 0, intr->name, &qcq->napi);
+}
+
+static int ionic_intr_alloc(struct ionic_lif *lif, struct ionic_intr_info *intr)
+{
+ struct ionic *ionic = lif->ionic;
+ int index;
+
+ index = find_first_zero_bit(ionic->intrs, ionic->nintrs);
+ if (index == ionic->nintrs) {
+ netdev_warn(lif->netdev, "%s: no intr, index=%d nintrs=%d\n",
+ __func__, index, ionic->nintrs);
+ return -ENOSPC;
+ }
+
+ set_bit(index, ionic->intrs);
+ ionic_intr_init(&ionic->idev, intr, index);
+
+ return 0;
+}
+
+static void ionic_intr_free(struct ionic_lif *lif, int index)
+{
+ if (index != INTR_INDEX_NOT_ASSIGNED && index < lif->ionic->nintrs)
+ clear_bit(index, lif->ionic->intrs);
+}
+
+static int ionic_qcq_enable(struct ionic_qcq *qcq)
+{
+ struct ionic_queue *q = &qcq->q;
+ struct ionic_lif *lif = q->lif;
+ struct ionic_dev *idev;
+ struct device *dev;
+
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.q_control = {
+ .opcode = IONIC_CMD_Q_CONTROL,
+ .lif_index = cpu_to_le16(lif->index),
+ .type = q->type,
+ .index = cpu_to_le32(q->index),
+ .oper = IONIC_Q_ENABLE,
+ },
+ };
+
+ idev = &lif->ionic->idev;
+ dev = lif->ionic->dev;
+
+ dev_dbg(dev, "q_enable.index %d q_enable.qtype %d\n",
+ ctx.cmd.q_control.index, ctx.cmd.q_control.type);
+
+ if (qcq->flags & IONIC_QCQ_F_INTR) {
+ irq_set_affinity_hint(qcq->intr.vector,
+ &qcq->intr.affinity_mask);
+ napi_enable(&qcq->napi);
+ ionic_intr_clean(idev->intr_ctrl, qcq->intr.index);
+ ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
+ IONIC_INTR_MASK_CLEAR);
+ }
+
+ return ionic_adminq_post_wait(lif, &ctx);
+}
+
+static int ionic_qcq_disable(struct ionic_qcq *qcq)
+{
+ struct ionic_queue *q = &qcq->q;
+ struct ionic_lif *lif = q->lif;
+ struct ionic_dev *idev;
+ struct device *dev;
+
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.q_control = {
+ .opcode = IONIC_CMD_Q_CONTROL,
+ .lif_index = cpu_to_le16(lif->index),
+ .type = q->type,
+ .index = cpu_to_le32(q->index),
+ .oper = IONIC_Q_DISABLE,
+ },
+ };
+
+ idev = &lif->ionic->idev;
+ dev = lif->ionic->dev;
+
+ dev_dbg(dev, "q_disable.index %d q_disable.qtype %d\n",
+ ctx.cmd.q_control.index, ctx.cmd.q_control.type);
+
+ if (qcq->flags & IONIC_QCQ_F_INTR) {
+ ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
+ IONIC_INTR_MASK_SET);
+ synchronize_irq(qcq->intr.vector);
+ irq_set_affinity_hint(qcq->intr.vector, NULL);
+ napi_disable(&qcq->napi);
+ }
+
+ return ionic_adminq_post_wait(lif, &ctx);
+}
+
+static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+ struct ionic_dev *idev = &lif->ionic->idev;
+ struct device *dev = lif->ionic->dev;
+
+ if (!qcq)
+ return;
+
+ ionic_debugfs_del_qcq(qcq);
+
+ if (!(qcq->flags & IONIC_QCQ_F_INITED))
+ return;
+
+ if (qcq->flags & IONIC_QCQ_F_INTR) {
+ ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
+ IONIC_INTR_MASK_SET);
+ devm_free_irq(dev, qcq->intr.vector, &qcq->napi);
+ netif_napi_del(&qcq->napi);
+ }
+
+ qcq->flags &= ~IONIC_QCQ_F_INITED;
+}
+
+static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+ struct device *dev = lif->ionic->dev;
+
+ if (!qcq)
+ return;
+
+ dma_free_coherent(dev, qcq->total_size, qcq->base, qcq->base_pa);
+ qcq->base = NULL;
+ qcq->base_pa = 0;
+
+ if (qcq->flags & IONIC_QCQ_F_INTR)
+ ionic_intr_free(lif, qcq->intr.index);
+
+ devm_kfree(dev, qcq->cq.info);
+ qcq->cq.info = NULL;
+ devm_kfree(dev, qcq->q.info);
+ qcq->q.info = NULL;
+ devm_kfree(dev, qcq);
+}
+
+static void ionic_qcqs_free(struct ionic_lif *lif)
+{
+ struct device *dev = lif->ionic->dev;
+ unsigned int i;
+
+ if (lif->notifyqcq) {
+ ionic_qcq_free(lif, lif->notifyqcq);
+ lif->notifyqcq = NULL;
+ }
+
+ if (lif->adminqcq) {
+ ionic_qcq_free(lif, lif->adminqcq);
+ lif->adminqcq = NULL;
+ }
+
+ for (i = 0; i < lif->nxqs; i++)
+ if (lif->rxqcqs[i].stats)
+ devm_kfree(dev, lif->rxqcqs[i].stats);
+
+ devm_kfree(dev, lif->rxqcqs);
+ lif->rxqcqs = NULL;
+
+ for (i = 0; i < lif->nxqs; i++)
+ if (lif->txqcqs[i].stats)
+ devm_kfree(dev, lif->txqcqs[i].stats);
+
+ devm_kfree(dev, lif->txqcqs);
+ lif->txqcqs = NULL;
+}
+
+static void ionic_link_qcq_interrupts(struct ionic_qcq *src_qcq,
+ struct ionic_qcq *n_qcq)
+{
+ if (WARN_ON(n_qcq->flags & IONIC_QCQ_F_INTR)) {
+ ionic_intr_free(n_qcq->cq.lif, n_qcq->intr.index);
+ n_qcq->flags &= ~IONIC_QCQ_F_INTR;
+ }
+
+ n_qcq->intr.vector = src_qcq->intr.vector;
+ n_qcq->intr.index = src_qcq->intr.index;
+}
+
+static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type,
+ unsigned int index,
+ const char *name, unsigned int flags,
+ unsigned int num_descs, unsigned int desc_size,
+ unsigned int cq_desc_size,
+ unsigned int sg_desc_size,
+ unsigned int pid, struct ionic_qcq **qcq)
+{
+ struct ionic_dev *idev = &lif->ionic->idev;
+ u32 q_size, cq_size, sg_size, total_size;
+ struct device *dev = lif->ionic->dev;
+ void *q_base, *cq_base, *sg_base;
+ dma_addr_t cq_base_pa = 0;
+ dma_addr_t sg_base_pa = 0;
+ dma_addr_t q_base_pa = 0;
+ struct ionic_qcq *new;
+ int err;
+
+ *qcq = NULL;
+
+ q_size = num_descs * desc_size;
+ cq_size = num_descs * cq_desc_size;
+ sg_size = num_descs * sg_desc_size;
+
+ total_size = ALIGN(q_size, PAGE_SIZE) + ALIGN(cq_size, PAGE_SIZE);
+ /* Note: aligning q_size/cq_size is not enough due to cq_base
+ * address aligning as q_base could be not aligned to the page.
+ * Adding PAGE_SIZE.
+ */
+ total_size += PAGE_SIZE;
+ if (flags & IONIC_QCQ_F_SG) {
+ total_size += ALIGN(sg_size, PAGE_SIZE);
+ total_size += PAGE_SIZE;
+ }
+
+ new = devm_kzalloc(dev, sizeof(*new), GFP_KERNEL);
+ if (!new) {
+ netdev_err(lif->netdev, "Cannot allocate queue structure\n");
+ err = -ENOMEM;
+ goto err_out;
+ }
+
+ new->flags = flags;
+
+ new->q.info = devm_kzalloc(dev, sizeof(*new->q.info) * num_descs,
+ GFP_KERNEL);
+ if (!new->q.info) {
+ netdev_err(lif->netdev, "Cannot allocate queue info\n");
+ err = -ENOMEM;
+ goto err_out;
+ }
+
+ new->q.type = type;
+
+ err = ionic_q_init(lif, idev, &new->q, index, name, num_descs,
+ desc_size, sg_desc_size, pid);
+ if (err) {
+ netdev_err(lif->netdev, "Cannot initialize queue\n");
+ goto err_out;
+ }
+
+ if (flags & IONIC_QCQ_F_INTR) {
+ err = ionic_intr_alloc(lif, &new->intr);
+ if (err) {
+ netdev_warn(lif->netdev, "no intr for %s: %d\n",
+ name, err);
+ goto err_out;
+ }
+
+ err = ionic_bus_get_irq(lif->ionic, new->intr.index);
+ if (err < 0) {
+ netdev_warn(lif->netdev, "no vector for %s: %d\n",
+ name, err);
+ goto err_out_free_intr;
+ }
+ new->intr.vector = err;
+ ionic_intr_mask_assert(idev->intr_ctrl, new->intr.index,
+ IONIC_INTR_MASK_SET);
+
+ new->intr.cpu = new->intr.index % num_online_cpus();
+ if (cpu_online(new->intr.cpu))
+ cpumask_set_cpu(new->intr.cpu,
+ &new->intr.affinity_mask);
+ } else {
+ new->intr.index = INTR_INDEX_NOT_ASSIGNED;
+ }
+
+ new->cq.info = devm_kzalloc(dev, sizeof(*new->cq.info) * num_descs,
+ GFP_KERNEL);
+ if (!new->cq.info) {
+ netdev_err(lif->netdev, "Cannot allocate completion queue info\n");
+ err = -ENOMEM;
+ goto err_out_free_intr;
+ }
+
+ err = ionic_cq_init(lif, &new->cq, &new->intr, num_descs, cq_desc_size);
+ if (err) {
+ netdev_err(lif->netdev, "Cannot initialize completion queue\n");
+ goto err_out_free_intr;
+ }
+
+ new->base = dma_alloc_coherent(dev, total_size, &new->base_pa,
+ GFP_KERNEL);
+ if (!new->base) {
+ netdev_err(lif->netdev, "Cannot allocate queue DMA memory\n");
+ err = -ENOMEM;
+ goto err_out_free_intr;
+ }
+
+ new->total_size = total_size;
+
+ q_base = new->base;
+ q_base_pa = new->base_pa;
+
+ cq_base = (void *)ALIGN((uintptr_t)q_base + q_size, PAGE_SIZE);
+ cq_base_pa = ALIGN(q_base_pa + q_size, PAGE_SIZE);
+
+ if (flags & IONIC_QCQ_F_SG) {
+ sg_base = (void *)ALIGN((uintptr_t)cq_base + cq_size,
+ PAGE_SIZE);
+ sg_base_pa = ALIGN(cq_base_pa + cq_size, PAGE_SIZE);
+ ionic_q_sg_map(&new->q, sg_base, sg_base_pa);
+ }
+
+ ionic_q_map(&new->q, q_base, q_base_pa);
+ ionic_cq_map(&new->cq, cq_base, cq_base_pa);
+ ionic_cq_bind(&new->cq, &new->q);
+
+ *qcq = new;
+
+ return 0;
+
+err_out_free_intr:
+ ionic_intr_free(lif, new->intr.index);
+err_out:
+ dev_err(dev, "qcq alloc of %s%d failed %d\n", name, index, err);
+ return err;
+}
+
+static int ionic_qcqs_alloc(struct ionic_lif *lif)
+{
+ struct device *dev = lif->ionic->dev;
+ unsigned int q_list_size;
+ unsigned int flags;
+ int err;
+ int i;
+
+ flags = IONIC_QCQ_F_INTR;
+ err = ionic_qcq_alloc(lif, IONIC_QTYPE_ADMINQ, 0, "admin", flags,
+ IONIC_ADMINQ_LENGTH,
+ sizeof(struct ionic_admin_cmd),
+ sizeof(struct ionic_admin_comp),
+ 0, lif->kern_pid, &lif->adminqcq);
+ if (err)
+ return err;
+
+ if (lif->ionic->nnqs_per_lif) {
+ flags = IONIC_QCQ_F_NOTIFYQ;
+ err = ionic_qcq_alloc(lif, IONIC_QTYPE_NOTIFYQ, 0, "notifyq",
+ flags, IONIC_NOTIFYQ_LENGTH,
+ sizeof(struct ionic_notifyq_cmd),
+ sizeof(union ionic_notifyq_comp),
+ 0, lif->kern_pid, &lif->notifyqcq);
+ if (err)
+ goto err_out_free_adminqcq;
+
+ /* Let the notifyq ride on the adminq interrupt */
+ ionic_link_qcq_interrupts(lif->adminqcq, lif->notifyqcq);
+ }
+
+ q_list_size = sizeof(*lif->txqcqs) * lif->nxqs;
+ err = -ENOMEM;
+ lif->txqcqs = devm_kzalloc(dev, q_list_size, GFP_KERNEL);
+ if (!lif->txqcqs)
+ goto err_out_free_notifyqcq;
+ for (i = 0; i < lif->nxqs; i++) {
+ lif->txqcqs[i].stats = devm_kzalloc(dev,
+ sizeof(struct ionic_q_stats),
+ GFP_KERNEL);
+ if (!lif->txqcqs[i].stats)
+ goto err_out_free_tx_stats;
+ }
+
+ lif->rxqcqs = devm_kzalloc(dev, q_list_size, GFP_KERNEL);
+ if (!lif->rxqcqs)
+ goto err_out_free_tx_stats;
+ for (i = 0; i < lif->nxqs; i++) {
+ lif->rxqcqs[i].stats = devm_kzalloc(dev,
+ sizeof(struct ionic_q_stats),
+ GFP_KERNEL);
+ if (!lif->rxqcqs[i].stats)
+ goto err_out_free_rx_stats;
+ }
+
+ return 0;
+
+err_out_free_rx_stats:
+ for (i = 0; i < lif->nxqs; i++)
+ if (lif->rxqcqs[i].stats)
+ devm_kfree(dev, lif->rxqcqs[i].stats);
+ devm_kfree(dev, lif->rxqcqs);
+ lif->rxqcqs = NULL;
+err_out_free_tx_stats:
+ for (i = 0; i < lif->nxqs; i++)
+ if (lif->txqcqs[i].stats)
+ devm_kfree(dev, lif->txqcqs[i].stats);
+ devm_kfree(dev, lif->txqcqs);
+ lif->txqcqs = NULL;
+err_out_free_notifyqcq:
+ if (lif->notifyqcq) {
+ ionic_qcq_free(lif, lif->notifyqcq);
+ lif->notifyqcq = NULL;
+ }
+err_out_free_adminqcq:
+ ionic_qcq_free(lif, lif->adminqcq);
+ lif->adminqcq = NULL;
+
+ return err;
+}
+
+static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+ struct device *dev = lif->ionic->dev;
+ struct ionic_queue *q = &qcq->q;
+ struct ionic_cq *cq = &qcq->cq;
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.q_init = {
+ .opcode = IONIC_CMD_Q_INIT,
+ .lif_index = cpu_to_le16(lif->index),
+ .type = q->type,
+ .index = cpu_to_le32(q->index),
+ .flags = cpu_to_le16(IONIC_QINIT_F_IRQ |
+ IONIC_QINIT_F_SG),
+ .intr_index = cpu_to_le16(lif->rxqcqs[q->index].qcq->intr.index),
+ .pid = cpu_to_le16(q->pid),
+ .ring_size = ilog2(q->num_descs),
+ .ring_base = cpu_to_le64(q->base_pa),
+ .cq_ring_base = cpu_to_le64(cq->base_pa),
+ .sg_ring_base = cpu_to_le64(q->sg_base_pa),
+ },
+ };
+ int err;
+
+ dev_dbg(dev, "txq_init.pid %d\n", ctx.cmd.q_init.pid);
+ dev_dbg(dev, "txq_init.index %d\n", ctx.cmd.q_init.index);
+ dev_dbg(dev, "txq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
+ dev_dbg(dev, "txq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ q->hw_type = ctx.comp.q_init.hw_type;
+ q->hw_index = le32_to_cpu(ctx.comp.q_init.hw_index);
+ q->dbval = IONIC_DBELL_QID(q->hw_index);
+
+ dev_dbg(dev, "txq->hw_type %d\n", q->hw_type);
+ dev_dbg(dev, "txq->hw_index %d\n", q->hw_index);
+
+ qcq->flags |= IONIC_QCQ_F_INITED;
+
+ ionic_debugfs_add_qcq(lif, qcq);
+
+ return 0;
+}
+
+static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
+{
+ struct device *dev = lif->ionic->dev;
+ struct ionic_queue *q = &qcq->q;
+ struct ionic_cq *cq = &qcq->cq;
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.q_init = {
+ .opcode = IONIC_CMD_Q_INIT,
+ .lif_index = cpu_to_le16(lif->index),
+ .type = q->type,
+ .index = cpu_to_le32(q->index),
+ .flags = cpu_to_le16(IONIC_QINIT_F_IRQ),
+ .intr_index = cpu_to_le16(cq->bound_intr->index),
+ .pid = cpu_to_le16(q->pid),
+ .ring_size = ilog2(q->num_descs),
+ .ring_base = cpu_to_le64(q->base_pa),
+ .cq_ring_base = cpu_to_le64(cq->base_pa),
+ },
+ };
+ int err;
+
+ dev_dbg(dev, "rxq_init.pid %d\n", ctx.cmd.q_init.pid);
+ dev_dbg(dev, "rxq_init.index %d\n", ctx.cmd.q_init.index);
+ dev_dbg(dev, "rxq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
+ dev_dbg(dev, "rxq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ q->hw_type = ctx.comp.q_init.hw_type;
+ q->hw_index = le32_to_cpu(ctx.comp.q_init.hw_index);
+ q->dbval = IONIC_DBELL_QID(q->hw_index);
+
+ dev_dbg(dev, "rxq->hw_type %d\n", q->hw_type);
+ dev_dbg(dev, "rxq->hw_index %d\n", q->hw_index);
+
+ netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi,
+ NAPI_POLL_WEIGHT);
+
+ err = ionic_request_irq(lif, qcq);
+ if (err) {
+ netif_napi_del(&qcq->napi);
+ return err;
+ }
+
+ qcq->flags |= IONIC_QCQ_F_INITED;
+
+ ionic_debugfs_add_qcq(lif, qcq);
+
+ return 0;
+}
+
+static bool ionic_notifyq_service(struct ionic_cq *cq,
+ struct ionic_cq_info *cq_info)
+{
+ union ionic_notifyq_comp *comp = cq_info->cq_desc;
+ struct net_device *netdev;
+ struct ionic_queue *q;
+ struct ionic_lif *lif;
+ u64 eid;
+
+ q = cq->bound_q;
+ lif = q->info[0].cb_arg;
+ netdev = lif->netdev;
+ eid = le64_to_cpu(comp->event.eid);
+
+ /* Have we run out of new completions to process? */
+ if (eid <= lif->last_eid)
+ return false;
+
+ lif->last_eid = eid;
+
+ dev_dbg(lif->ionic->dev, "notifyq event:\n");
+ dynamic_hex_dump("event ", DUMP_PREFIX_OFFSET, 16, 1,
+ comp, sizeof(*comp), true);
+
+ switch (le16_to_cpu(comp->event.ecode)) {
+ case IONIC_EVENT_LINK_CHANGE:
+ ionic_link_status_check_request(lif);
+ break;
+ case IONIC_EVENT_RESET:
+ netdev_info(netdev, "Notifyq IONIC_EVENT_RESET eid=%lld\n",
+ eid);
+ netdev_info(netdev, " reset_code=%d state=%d\n",
+ comp->reset.reset_code,
+ comp->reset.state);
+ break;
+ default:
+ netdev_warn(netdev, "Notifyq unknown event ecode=%d eid=%lld\n",
+ comp->event.ecode, eid);
+ break;
+ }
+
+ return true;
+}
+
+static int ionic_notifyq_clean(struct ionic_lif *lif, int budget)
+{
+ struct ionic_dev *idev = &lif->ionic->idev;
+ struct ionic_cq *cq = &lif->notifyqcq->cq;
+ u32 work_done;
+
+ work_done = ionic_cq_service(cq, budget, ionic_notifyq_service,
+ NULL, NULL);
+ if (work_done)
+ ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index,
+ work_done, IONIC_INTR_CRED_RESET_COALESCE);
+
+ return work_done;
+}
+
+static bool ionic_adminq_service(struct ionic_cq *cq,
+ struct ionic_cq_info *cq_info)
+{
+ struct ionic_admin_comp *comp = cq_info->cq_desc;
+
+ if (!color_match(comp->color, cq->done_color))
+ return false;
+
+ ionic_q_service(cq->bound_q, cq_info, le16_to_cpu(comp->comp_index));
+
+ return true;
+}
+
+static int ionic_adminq_napi(struct napi_struct *napi, int budget)
+{
+ struct ionic_lif *lif = napi_to_cq(napi)->lif;
+ int n_work = 0;
+ int a_work = 0;
+
+ if (likely(lif->notifyqcq && lif->notifyqcq->flags & IONIC_QCQ_F_INITED))
+ n_work = ionic_notifyq_clean(lif, budget);
+ a_work = ionic_napi(napi, budget, ionic_adminq_service, NULL, NULL);
+
+ return max(n_work, a_work);
+}
+
+static void ionic_get_stats64(struct net_device *netdev,
+ struct rtnl_link_stats64 *ns)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_lif_stats *ls;
+
+ memset(ns, 0, sizeof(*ns));
+ ls = &lif->info->stats;
+
+ ns->rx_packets = le64_to_cpu(ls->rx_ucast_packets) +
+ le64_to_cpu(ls->rx_mcast_packets) +
+ le64_to_cpu(ls->rx_bcast_packets);
+
+ ns->tx_packets = le64_to_cpu(ls->tx_ucast_packets) +
+ le64_to_cpu(ls->tx_mcast_packets) +
+ le64_to_cpu(ls->tx_bcast_packets);
+
+ ns->rx_bytes = le64_to_cpu(ls->rx_ucast_bytes) +
+ le64_to_cpu(ls->rx_mcast_bytes) +
+ le64_to_cpu(ls->rx_bcast_bytes);
+
+ ns->tx_bytes = le64_to_cpu(ls->tx_ucast_bytes) +
+ le64_to_cpu(ls->tx_mcast_bytes) +
+ le64_to_cpu(ls->tx_bcast_bytes);
+
+ ns->rx_dropped = le64_to_cpu(ls->rx_ucast_drop_packets) +
+ le64_to_cpu(ls->rx_mcast_drop_packets) +
+ le64_to_cpu(ls->rx_bcast_drop_packets);
+
+ ns->tx_dropped = le64_to_cpu(ls->tx_ucast_drop_packets) +
+ le64_to_cpu(ls->tx_mcast_drop_packets) +
+ le64_to_cpu(ls->tx_bcast_drop_packets);
+
+ ns->multicast = le64_to_cpu(ls->rx_mcast_packets);
+
+ ns->rx_over_errors = le64_to_cpu(ls->rx_queue_empty);
+
+ ns->rx_missed_errors = le64_to_cpu(ls->rx_dma_error) +
+ le64_to_cpu(ls->rx_queue_disabled) +
+ le64_to_cpu(ls->rx_desc_fetch_error) +
+ le64_to_cpu(ls->rx_desc_data_error);
+
+ ns->tx_aborted_errors = le64_to_cpu(ls->tx_dma_error) +
+ le64_to_cpu(ls->tx_queue_disabled) +
+ le64_to_cpu(ls->tx_desc_fetch_error) +
+ le64_to_cpu(ls->tx_desc_data_error);
+
+ ns->rx_errors = ns->rx_over_errors +
+ ns->rx_missed_errors;
+
+ ns->tx_errors = ns->tx_aborted_errors;
+}
+
+static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr)
+{
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.rx_filter_add = {
+ .opcode = IONIC_CMD_RX_FILTER_ADD,
+ .lif_index = cpu_to_le16(lif->index),
+ .match = cpu_to_le16(IONIC_RX_FILTER_MATCH_MAC),
+ },
+ };
+ struct ionic_rx_filter *f;
+ int err;
+
+ /* don't bother if we already have it */
+ spin_lock_bh(&lif->rx_filters.lock);
+ f = ionic_rx_filter_by_addr(lif, addr);
+ spin_unlock_bh(&lif->rx_filters.lock);
+ if (f)
+ return 0;
+
+ netdev_dbg(lif->netdev, "rx_filter add ADDR %pM (id %d)\n", addr,
+ ctx.comp.rx_filter_add.filter_id);
+
+ memcpy(ctx.cmd.rx_filter_add.mac.addr, addr, ETH_ALEN);
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ return ionic_rx_filter_save(lif, 0, IONIC_RXQ_INDEX_ANY, 0, &ctx);
+}
+
+static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr)
+{
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.rx_filter_del = {
+ .opcode = IONIC_CMD_RX_FILTER_DEL,
+ .lif_index = cpu_to_le16(lif->index),
+ },
+ };
+ struct ionic_rx_filter *f;
+ int err;
+
+ spin_lock_bh(&lif->rx_filters.lock);
+ f = ionic_rx_filter_by_addr(lif, addr);
+ if (!f) {
+ spin_unlock_bh(&lif->rx_filters.lock);
+ return -ENOENT;
+ }
+
+ ctx.cmd.rx_filter_del.filter_id = cpu_to_le32(f->filter_id);
+ ionic_rx_filter_free(lif, f);
+ spin_unlock_bh(&lif->rx_filters.lock);
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ netdev_dbg(lif->netdev, "rx_filter del ADDR %pM (id %d)\n", addr,
+ ctx.cmd.rx_filter_del.filter_id);
+
+ return 0;
+}
+
+static int ionic_lif_addr(struct ionic_lif *lif, const u8 *addr, bool add)
+{
+ struct ionic *ionic = lif->ionic;
+ struct ionic_deferred_work *work;
+ unsigned int nmfilters;
+ unsigned int nufilters;
+
+ if (add) {
+ /* Do we have space for this filter? We test the counters
+ * here before checking the need for deferral so that we
+ * can return an overflow error to the stack.
+ */
+ nmfilters = le32_to_cpu(ionic->ident.lif.eth.max_mcast_filters);
+ nufilters = le32_to_cpu(ionic->ident.lif.eth.max_ucast_filters);
+
+ if ((is_multicast_ether_addr(addr) && lif->nmcast < nmfilters))
+ lif->nmcast++;
+ else if (!is_multicast_ether_addr(addr) &&
+ lif->nucast < nufilters)
+ lif->nucast++;
+ else
+ return -ENOSPC;
+ } else {
+ if (is_multicast_ether_addr(addr) && lif->nmcast)
+ lif->nmcast--;
+ else if (!is_multicast_ether_addr(addr) && lif->nucast)
+ lif->nucast--;
+ }
+
+ if (in_interrupt()) {
+ work = kzalloc(sizeof(*work), GFP_ATOMIC);
+ if (!work) {
+ netdev_err(lif->netdev, "%s OOM\n", __func__);
+ return -ENOMEM;
+ }
+ work->type = add ? IONIC_DW_TYPE_RX_ADDR_ADD :
+ IONIC_DW_TYPE_RX_ADDR_DEL;
+ memcpy(work->addr, addr, ETH_ALEN);
+ netdev_dbg(lif->netdev, "deferred: rx_filter %s %pM\n",
+ add ? "add" : "del", addr);
+ ionic_lif_deferred_enqueue(&lif->deferred, work);
+ } else {
+ netdev_dbg(lif->netdev, "rx_filter %s %pM\n",
+ add ? "add" : "del", addr);
+ if (add)
+ return ionic_lif_addr_add(lif, addr);
+ else
+ return ionic_lif_addr_del(lif, addr);
+ }
+
+ return 0;
+}
+
+static int ionic_addr_add(struct net_device *netdev, const u8 *addr)
+{
+ return ionic_lif_addr(netdev_priv(netdev), addr, true);
+}
+
+static int ionic_addr_del(struct net_device *netdev, const u8 *addr)
+{
+ return ionic_lif_addr(netdev_priv(netdev), addr, false);
+}
+
+static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
+{
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.rx_mode_set = {
+ .opcode = IONIC_CMD_RX_MODE_SET,
+ .lif_index = cpu_to_le16(lif->index),
+ .rx_mode = cpu_to_le16(rx_mode),
+ },
+ };
+ char buf[128];
+ int err;
+ int i;
+#define REMAIN(__x) (sizeof(buf) - (__x))
+
+ i = snprintf(buf, sizeof(buf), "rx_mode 0x%04x -> 0x%04x:",
+ lif->rx_mode, rx_mode);
+ if (rx_mode & IONIC_RX_MODE_F_UNICAST)
+ i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_UNICAST");
+ if (rx_mode & IONIC_RX_MODE_F_MULTICAST)
+ i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_MULTICAST");
+ if (rx_mode & IONIC_RX_MODE_F_BROADCAST)
+ i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_BROADCAST");
+ if (rx_mode & IONIC_RX_MODE_F_PROMISC)
+ i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_PROMISC");
+ if (rx_mode & IONIC_RX_MODE_F_ALLMULTI)
+ i += snprintf(&buf[i], REMAIN(i), " RX_MODE_F_ALLMULTI");
+ netdev_dbg(lif->netdev, "lif%d %s\n", lif->index, buf);
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ netdev_warn(lif->netdev, "set rx_mode 0x%04x failed: %d\n",
+ rx_mode, err);
+ else
+ lif->rx_mode = rx_mode;
+}
+
+static void _ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
+{
+ struct ionic_deferred_work *work;
+
+ if (in_interrupt()) {
+ work = kzalloc(sizeof(*work), GFP_ATOMIC);
+ if (!work) {
+ netdev_err(lif->netdev, "%s OOM\n", __func__);
+ return;
+ }
+ work->type = IONIC_DW_TYPE_RX_MODE;
+ work->rx_mode = rx_mode;
+ netdev_dbg(lif->netdev, "deferred: rx_mode\n");
+ ionic_lif_deferred_enqueue(&lif->deferred, work);
+ } else {
+ ionic_lif_rx_mode(lif, rx_mode);
+ }
+}
+
+static void ionic_set_rx_mode(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_identity *ident;
+ unsigned int nfilters;
+ unsigned int rx_mode;
+
+ ident = &lif->ionic->ident;
+
+ rx_mode = IONIC_RX_MODE_F_UNICAST;
+ rx_mode |= (netdev->flags & IFF_MULTICAST) ? IONIC_RX_MODE_F_MULTICAST : 0;
+ rx_mode |= (netdev->flags & IFF_BROADCAST) ? IONIC_RX_MODE_F_BROADCAST : 0;
+ rx_mode |= (netdev->flags & IFF_PROMISC) ? IONIC_RX_MODE_F_PROMISC : 0;
+ rx_mode |= (netdev->flags & IFF_ALLMULTI) ? IONIC_RX_MODE_F_ALLMULTI : 0;
+
+ /* sync unicast addresses
+ * next check to see if we're in an overflow state
+ * if so, we track that we overflowed and enable NIC PROMISC
+ * else if the overflow is set and not needed
+ * we remove our overflow flag and check the netdev flags
+ * to see if we can disable NIC PROMISC
+ */
+ __dev_uc_sync(netdev, ionic_addr_add, ionic_addr_del);
+ nfilters = le32_to_cpu(ident->lif.eth.max_ucast_filters);
+ if (netdev_uc_count(netdev) + 1 > nfilters) {
+ rx_mode |= IONIC_RX_MODE_F_PROMISC;
+ lif->uc_overflow = true;
+ } else if (lif->uc_overflow) {
+ lif->uc_overflow = false;
+ if (!(netdev->flags & IFF_PROMISC))
+ rx_mode &= ~IONIC_RX_MODE_F_PROMISC;
+ }
+
+ /* same for multicast */
+ __dev_mc_sync(netdev, ionic_addr_add, ionic_addr_del);
+ nfilters = le32_to_cpu(ident->lif.eth.max_mcast_filters);
+ if (netdev_mc_count(netdev) > nfilters) {
+ rx_mode |= IONIC_RX_MODE_F_ALLMULTI;
+ lif->mc_overflow = true;
+ } else if (lif->mc_overflow) {
+ lif->mc_overflow = false;
+ if (!(netdev->flags & IFF_ALLMULTI))
+ rx_mode &= ~IONIC_RX_MODE_F_ALLMULTI;
+ }
+
+ if (lif->rx_mode != rx_mode)
+ _ionic_lif_rx_mode(lif, rx_mode);
+}
+
+static __le64 ionic_netdev_features_to_nic(netdev_features_t features)
+{
+ u64 wanted = 0;
+
+ if (features & NETIF_F_HW_VLAN_CTAG_TX)
+ wanted |= IONIC_ETH_HW_VLAN_TX_TAG;
+ if (features & NETIF_F_HW_VLAN_CTAG_RX)
+ wanted |= IONIC_ETH_HW_VLAN_RX_STRIP;
+ if (features & NETIF_F_HW_VLAN_CTAG_FILTER)
+ wanted |= IONIC_ETH_HW_VLAN_RX_FILTER;
+ if (features & NETIF_F_RXHASH)
+ wanted |= IONIC_ETH_HW_RX_HASH;
+ if (features & NETIF_F_RXCSUM)
+ wanted |= IONIC_ETH_HW_RX_CSUM;
+ if (features & NETIF_F_SG)
+ wanted |= IONIC_ETH_HW_TX_SG;
+ if (features & NETIF_F_HW_CSUM)
+ wanted |= IONIC_ETH_HW_TX_CSUM;
+ if (features & NETIF_F_TSO)
+ wanted |= IONIC_ETH_HW_TSO;
+ if (features & NETIF_F_TSO6)
+ wanted |= IONIC_ETH_HW_TSO_IPV6;
+ if (features & NETIF_F_TSO_ECN)
+ wanted |= IONIC_ETH_HW_TSO_ECN;
+ if (features & NETIF_F_GSO_GRE)
+ wanted |= IONIC_ETH_HW_TSO_GRE;
+ if (features & NETIF_F_GSO_GRE_CSUM)
+ wanted |= IONIC_ETH_HW_TSO_GRE_CSUM;
+ if (features & NETIF_F_GSO_IPXIP4)
+ wanted |= IONIC_ETH_HW_TSO_IPXIP4;
+ if (features & NETIF_F_GSO_IPXIP6)
+ wanted |= IONIC_ETH_HW_TSO_IPXIP6;
+ if (features & NETIF_F_GSO_UDP_TUNNEL)
+ wanted |= IONIC_ETH_HW_TSO_UDP;
+ if (features & NETIF_F_GSO_UDP_TUNNEL_CSUM)
+ wanted |= IONIC_ETH_HW_TSO_UDP_CSUM;
+
+ return cpu_to_le64(wanted);
+}
+
+static int ionic_set_nic_features(struct ionic_lif *lif,
+ netdev_features_t features)
+{
+ struct device *dev = lif->ionic->dev;
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.lif_setattr = {
+ .opcode = IONIC_CMD_LIF_SETATTR,
+ .index = cpu_to_le16(lif->index),
+ .attr = IONIC_LIF_ATTR_FEATURES,
+ },
+ };
+ u64 vlan_flags = IONIC_ETH_HW_VLAN_TX_TAG |
+ IONIC_ETH_HW_VLAN_RX_STRIP |
+ IONIC_ETH_HW_VLAN_RX_FILTER;
+ int err;
+
+ ctx.cmd.lif_setattr.features = ionic_netdev_features_to_nic(features);
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ lif->hw_features = le64_to_cpu(ctx.cmd.lif_setattr.features &
+ ctx.comp.lif_setattr.features);
+
+ if ((vlan_flags & features) &&
+ !(vlan_flags & le64_to_cpu(ctx.comp.lif_setattr.features)))
+ dev_info_once(lif->ionic->dev, "NIC is not supporting vlan offload, likely in SmartNIC mode\n");
+
+ if (lif->hw_features & IONIC_ETH_HW_VLAN_TX_TAG)
+ dev_dbg(dev, "feature ETH_HW_VLAN_TX_TAG\n");
+ if (lif->hw_features & IONIC_ETH_HW_VLAN_RX_STRIP)
+ dev_dbg(dev, "feature ETH_HW_VLAN_RX_STRIP\n");
+ if (lif->hw_features & IONIC_ETH_HW_VLAN_RX_FILTER)
+ dev_dbg(dev, "feature ETH_HW_VLAN_RX_FILTER\n");
+ if (lif->hw_features & IONIC_ETH_HW_RX_HASH)
+ dev_dbg(dev, "feature ETH_HW_RX_HASH\n");
+ if (lif->hw_features & IONIC_ETH_HW_TX_SG)
+ dev_dbg(dev, "feature ETH_HW_TX_SG\n");
+ if (lif->hw_features & IONIC_ETH_HW_TX_CSUM)
+ dev_dbg(dev, "feature ETH_HW_TX_CSUM\n");
+ if (lif->hw_features & IONIC_ETH_HW_RX_CSUM)
+ dev_dbg(dev, "feature ETH_HW_RX_CSUM\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO)
+ dev_dbg(dev, "feature ETH_HW_TSO\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_IPV6)
+ dev_dbg(dev, "feature ETH_HW_TSO_IPV6\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_ECN)
+ dev_dbg(dev, "feature ETH_HW_TSO_ECN\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_GRE)
+ dev_dbg(dev, "feature ETH_HW_TSO_GRE\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_GRE_CSUM)
+ dev_dbg(dev, "feature ETH_HW_TSO_GRE_CSUM\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_IPXIP4)
+ dev_dbg(dev, "feature ETH_HW_TSO_IPXIP4\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_IPXIP6)
+ dev_dbg(dev, "feature ETH_HW_TSO_IPXIP6\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_UDP)
+ dev_dbg(dev, "feature ETH_HW_TSO_UDP\n");
+ if (lif->hw_features & IONIC_ETH_HW_TSO_UDP_CSUM)
+ dev_dbg(dev, "feature ETH_HW_TSO_UDP_CSUM\n");
+
+ return 0;
+}
+
+static int ionic_init_nic_features(struct ionic_lif *lif)
+{
+ struct net_device *netdev = lif->netdev;
+ netdev_features_t features;
+ int err;
+
+ /* set up what we expect to support by default */
+ features = NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_CTAG_RX |
+ NETIF_F_HW_VLAN_CTAG_FILTER |
+ NETIF_F_RXHASH |
+ NETIF_F_SG |
+ NETIF_F_HW_CSUM |
+ NETIF_F_RXCSUM |
+ NETIF_F_TSO |
+ NETIF_F_TSO6 |
+ NETIF_F_TSO_ECN;
+
+ err = ionic_set_nic_features(lif, features);
+ if (err)
+ return err;
+
+ /* tell the netdev what we actually can support */
+ netdev->features |= NETIF_F_HIGHDMA;
+
+ if (lif->hw_features & IONIC_ETH_HW_VLAN_TX_TAG)
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_TX;
+ if (lif->hw_features & IONIC_ETH_HW_VLAN_RX_STRIP)
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_RX;
+ if (lif->hw_features & IONIC_ETH_HW_VLAN_RX_FILTER)
+ netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER;
+ if (lif->hw_features & IONIC_ETH_HW_RX_HASH)
+ netdev->hw_features |= NETIF_F_RXHASH;
+ if (lif->hw_features & IONIC_ETH_HW_TX_SG)
+ netdev->hw_features |= NETIF_F_SG;
+
+ if (lif->hw_features & IONIC_ETH_HW_TX_CSUM)
+ netdev->hw_enc_features |= NETIF_F_HW_CSUM;
+ if (lif->hw_features & IONIC_ETH_HW_RX_CSUM)
+ netdev->hw_enc_features |= NETIF_F_RXCSUM;
+ if (lif->hw_features & IONIC_ETH_HW_TSO)
+ netdev->hw_enc_features |= NETIF_F_TSO;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_IPV6)
+ netdev->hw_enc_features |= NETIF_F_TSO6;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_ECN)
+ netdev->hw_enc_features |= NETIF_F_TSO_ECN;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_GRE)
+ netdev->hw_enc_features |= NETIF_F_GSO_GRE;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_GRE_CSUM)
+ netdev->hw_enc_features |= NETIF_F_GSO_GRE_CSUM;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_IPXIP4)
+ netdev->hw_enc_features |= NETIF_F_GSO_IPXIP4;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_IPXIP6)
+ netdev->hw_enc_features |= NETIF_F_GSO_IPXIP6;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_UDP)
+ netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL;
+ if (lif->hw_features & IONIC_ETH_HW_TSO_UDP_CSUM)
+ netdev->hw_enc_features |= NETIF_F_GSO_UDP_TUNNEL_CSUM;
+
+ netdev->hw_features |= netdev->hw_enc_features;
+ netdev->features |= netdev->hw_features;
+
+ netdev->priv_flags |= IFF_UNICAST_FLT;
+
+ return 0;
+}
+
+static int ionic_set_features(struct net_device *netdev,
+ netdev_features_t features)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ int err;
+
+ netdev_dbg(netdev, "%s: lif->features=0x%08llx new_features=0x%08llx\n",
+ __func__, (u64)lif->netdev->features, (u64)features);
+
+ err = ionic_set_nic_features(lif, features);
+
+ return err;
+}
+
+static int ionic_set_mac_address(struct net_device *netdev, void *sa)
+{
+ struct sockaddr *addr = sa;
+ u8 *mac;
+ int err;
+
+ mac = (u8 *)addr->sa_data;
+ if (ether_addr_equal(netdev->dev_addr, mac))
+ return 0;
+
+ err = eth_prepare_mac_addr_change(netdev, addr);
+ if (err)
+ return err;
+
+ if (!is_zero_ether_addr(netdev->dev_addr)) {
+ netdev_info(netdev, "deleting mac addr %pM\n",
+ netdev->dev_addr);
+ ionic_addr_del(netdev, netdev->dev_addr);
+ }
+
+ eth_commit_mac_addr_change(netdev, addr);
+ netdev_info(netdev, "updating mac addr %pM\n", mac);
+
+ return ionic_addr_add(netdev, mac);
+}
+
+static int ionic_change_mtu(struct net_device *netdev, int new_mtu)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.lif_setattr = {
+ .opcode = IONIC_CMD_LIF_SETATTR,
+ .index = cpu_to_le16(lif->index),
+ .attr = IONIC_LIF_ATTR_MTU,
+ .mtu = cpu_to_le32(new_mtu),
+ },
+ };
+ int err;
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ netdev->mtu = new_mtu;
+ err = ionic_reset_queues(lif);
+
+ return err;
+}
+
+static void ionic_tx_timeout_work(struct work_struct *ws)
+{
+ struct ionic_lif *lif = container_of(ws, struct ionic_lif, tx_timeout_work);
+
+ netdev_info(lif->netdev, "Tx Timeout recovery\n");
+
+ rtnl_lock();
+ ionic_reset_queues(lif);
+ rtnl_unlock();
+}
+
+static void ionic_tx_timeout(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+
+ schedule_work(&lif->tx_timeout_work);
+}
+
+static int ionic_vlan_rx_add_vid(struct net_device *netdev, __be16 proto,
+ u16 vid)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.rx_filter_add = {
+ .opcode = IONIC_CMD_RX_FILTER_ADD,
+ .lif_index = cpu_to_le16(lif->index),
+ .match = cpu_to_le16(IONIC_RX_FILTER_MATCH_VLAN),
+ .vlan.vlan = cpu_to_le16(vid),
+ },
+ };
+ int err;
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ netdev_dbg(netdev, "rx_filter add VLAN %d (id %d)\n", vid,
+ ctx.comp.rx_filter_add.filter_id);
+
+ return ionic_rx_filter_save(lif, 0, IONIC_RXQ_INDEX_ANY, 0, &ctx);
+}
+
+static int ionic_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto,
+ u16 vid)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.rx_filter_del = {
+ .opcode = IONIC_CMD_RX_FILTER_DEL,
+ .lif_index = cpu_to_le16(lif->index),
+ },
+ };
+ struct ionic_rx_filter *f;
+
+ spin_lock_bh(&lif->rx_filters.lock);
+
+ f = ionic_rx_filter_by_vlan(lif, vid);
+ if (!f) {
+ spin_unlock_bh(&lif->rx_filters.lock);
+ return -ENOENT;
+ }
+
+ netdev_dbg(netdev, "rx_filter del VLAN %d (id %d)\n", vid,
+ le32_to_cpu(ctx.cmd.rx_filter_del.filter_id));
+
+ ctx.cmd.rx_filter_del.filter_id = cpu_to_le32(f->filter_id);
+ ionic_rx_filter_free(lif, f);
+ spin_unlock_bh(&lif->rx_filters.lock);
+
+ return ionic_adminq_post_wait(lif, &ctx);
+}
+
+int ionic_lif_rss_config(struct ionic_lif *lif, const u16 types,
+ const u8 *key, const u32 *indir)
+{
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.lif_setattr = {
+ .opcode = IONIC_CMD_LIF_SETATTR,
+ .attr = IONIC_LIF_ATTR_RSS,
+ .rss.types = cpu_to_le16(types),
+ .rss.addr = cpu_to_le64(lif->rss_ind_tbl_pa),
+ },
+ };
+ unsigned int i, tbl_sz;
+
+ lif->rss_types = types;
+
+ if (key)
+ memcpy(lif->rss_hash_key, key, IONIC_RSS_HASH_KEY_SIZE);
+
+ if (indir) {
+ tbl_sz = le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+ for (i = 0; i < tbl_sz; i++)
+ lif->rss_ind_tbl[i] = indir[i];
+ }
+
+ memcpy(ctx.cmd.lif_setattr.rss.key, lif->rss_hash_key,
+ IONIC_RSS_HASH_KEY_SIZE);
+
+ return ionic_adminq_post_wait(lif, &ctx);
+}
+
+static int ionic_lif_rss_init(struct ionic_lif *lif)
+{
+ u8 rss_key[IONIC_RSS_HASH_KEY_SIZE];
+ unsigned int tbl_sz;
+ unsigned int i;
+
+ netdev_rss_key_fill(rss_key, IONIC_RSS_HASH_KEY_SIZE);
+
+ lif->rss_types = IONIC_RSS_TYPE_IPV4 |
+ IONIC_RSS_TYPE_IPV4_TCP |
+ IONIC_RSS_TYPE_IPV4_UDP |
+ IONIC_RSS_TYPE_IPV6 |
+ IONIC_RSS_TYPE_IPV6_TCP |
+ IONIC_RSS_TYPE_IPV6_UDP;
+
+ /* Fill indirection table with 'default' values */
+ tbl_sz = le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+ for (i = 0; i < tbl_sz; i++)
+ lif->rss_ind_tbl[i] = ethtool_rxfh_indir_default(i, lif->nxqs);
+
+ return ionic_lif_rss_config(lif, lif->rss_types, rss_key, NULL);
+}
+
+static int ionic_lif_rss_deinit(struct ionic_lif *lif)
+{
+ return ionic_lif_rss_config(lif, 0x0, NULL, NULL);
+}
+
+static void ionic_txrx_disable(struct ionic_lif *lif)
+{
+ unsigned int i;
+
+ for (i = 0; i < lif->nxqs; i++) {
+ ionic_qcq_disable(lif->txqcqs[i].qcq);
+ ionic_qcq_disable(lif->rxqcqs[i].qcq);
+ }
+}
+
+static void ionic_txrx_deinit(struct ionic_lif *lif)
+{
+ unsigned int i;
+
+ for (i = 0; i < lif->nxqs; i++) {
+ ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
+ ionic_tx_flush(&lif->txqcqs[i].qcq->cq);
+
+ ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
+ ionic_rx_flush(&lif->rxqcqs[i].qcq->cq);
+ ionic_rx_empty(&lif->rxqcqs[i].qcq->q);
+ }
+}
+
+static void ionic_txrx_free(struct ionic_lif *lif)
+{
+ unsigned int i;
+
+ for (i = 0; i < lif->nxqs; i++) {
+ ionic_qcq_free(lif, lif->txqcqs[i].qcq);
+ lif->txqcqs[i].qcq = NULL;
+
+ ionic_qcq_free(lif, lif->rxqcqs[i].qcq);
+ lif->rxqcqs[i].qcq = NULL;
+ }
+}
+
+static int ionic_txrx_alloc(struct ionic_lif *lif)
+{
+ unsigned int flags;
+ unsigned int i;
+ int err = 0;
+ u32 coal;
+
+ flags = IONIC_QCQ_F_TX_STATS | IONIC_QCQ_F_SG;
+ for (i = 0; i < lif->nxqs; i++) {
+ err = ionic_qcq_alloc(lif, IONIC_QTYPE_TXQ, i, "tx", flags,
+ lif->ntxq_descs,
+ sizeof(struct ionic_txq_desc),
+ sizeof(struct ionic_txq_comp),
+ sizeof(struct ionic_txq_sg_desc),
+ lif->kern_pid, &lif->txqcqs[i].qcq);
+ if (err)
+ goto err_out;
+
+ lif->txqcqs[i].qcq->stats = lif->txqcqs[i].stats;
+ }
+
+ flags = IONIC_QCQ_F_RX_STATS | IONIC_QCQ_F_INTR;
+ coal = ionic_coal_usec_to_hw(lif->ionic, lif->rx_coalesce_usecs);
+ for (i = 0; i < lif->nxqs; i++) {
+ err = ionic_qcq_alloc(lif, IONIC_QTYPE_RXQ, i, "rx", flags,
+ lif->nrxq_descs,
+ sizeof(struct ionic_rxq_desc),
+ sizeof(struct ionic_rxq_comp),
+ 0, lif->kern_pid, &lif->rxqcqs[i].qcq);
+ if (err)
+ goto err_out;
+
+ lif->rxqcqs[i].qcq->stats = lif->rxqcqs[i].stats;
+
+ ionic_intr_coal_init(lif->ionic->idev.intr_ctrl,
+ lif->rxqcqs[i].qcq->intr.index, coal);
+ ionic_link_qcq_interrupts(lif->rxqcqs[i].qcq,
+ lif->txqcqs[i].qcq);
+ }
+
+ return 0;
+
+err_out:
+ ionic_txrx_free(lif);
+
+ return err;
+}
+
+static int ionic_txrx_init(struct ionic_lif *lif)
+{
+ unsigned int i;
+ int err;
+
+ for (i = 0; i < lif->nxqs; i++) {
+ err = ionic_lif_txq_init(lif, lif->txqcqs[i].qcq);
+ if (err)
+ goto err_out;
+
+ err = ionic_lif_rxq_init(lif, lif->rxqcqs[i].qcq);
+ if (err) {
+ ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
+ goto err_out;
+ }
+ }
+
+ if (lif->netdev->features & NETIF_F_RXHASH)
+ ionic_lif_rss_init(lif);
+
+ ionic_set_rx_mode(lif->netdev);
+
+ return 0;
+
+err_out:
+ while (i--) {
+ ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
+ ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
+ }
+
+ return err;
+}
+
+static int ionic_txrx_enable(struct ionic_lif *lif)
+{
+ int i, err;
+
+ for (i = 0; i < lif->nxqs; i++) {
+ err = ionic_qcq_enable(lif->txqcqs[i].qcq);
+ if (err)
+ goto err_out;
+
+ ionic_rx_fill(&lif->rxqcqs[i].qcq->q);
+ err = ionic_qcq_enable(lif->rxqcqs[i].qcq);
+ if (err) {
+ ionic_qcq_disable(lif->txqcqs[i].qcq);
+ goto err_out;
+ }
+ }
+
+ return 0;
+
+err_out:
+ while (i--) {
+ ionic_qcq_disable(lif->rxqcqs[i].qcq);
+ ionic_qcq_disable(lif->txqcqs[i].qcq);
+ }
+
+ return err;
+}
+
+int ionic_open(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ int err;
+
+ netif_carrier_off(netdev);
+
+ err = ionic_txrx_alloc(lif);
+ if (err)
+ return err;
+
+ err = ionic_txrx_init(lif);
+ if (err)
+ goto err_txrx_free;
+
+ err = ionic_txrx_enable(lif);
+ if (err)
+ goto err_txrx_deinit;
+
+ netif_set_real_num_tx_queues(netdev, lif->nxqs);
+ netif_set_real_num_rx_queues(netdev, lif->nxqs);
+
+ set_bit(IONIC_LIF_UP, lif->state);
+
+ ionic_link_status_check_request(lif);
+ if (netif_carrier_ok(netdev))
+ netif_tx_wake_all_queues(netdev);
+
+ return 0;
+
+err_txrx_deinit:
+ ionic_txrx_deinit(lif);
+err_txrx_free:
+ ionic_txrx_free(lif);
+ return err;
+}
+
+int ionic_stop(struct net_device *netdev)
+{
+ struct ionic_lif *lif = netdev_priv(netdev);
+ int err = 0;
+
+ if (!test_bit(IONIC_LIF_UP, lif->state)) {
+ dev_dbg(lif->ionic->dev, "%s: %s state=DOWN\n",
+ __func__, lif->name);
+ return 0;
+ }
+ dev_dbg(lif->ionic->dev, "%s: %s state=UP\n", __func__, lif->name);
+ clear_bit(IONIC_LIF_UP, lif->state);
+
+ /* carrier off before disabling queues to avoid watchdog timeout */
+ netif_carrier_off(netdev);
+ netif_tx_stop_all_queues(netdev);
+ netif_tx_disable(netdev);
+
+ ionic_txrx_disable(lif);
+ ionic_txrx_deinit(lif);
+ ionic_txrx_free(lif);
+
+ return err;
+}
+
+static const struct net_device_ops ionic_netdev_ops = {
+ .ndo_open = ionic_open,
+ .ndo_stop = ionic_stop,
+ .ndo_start_xmit = ionic_start_xmit,
+ .ndo_get_stats64 = ionic_get_stats64,
+ .ndo_set_rx_mode = ionic_set_rx_mode,
+ .ndo_set_features = ionic_set_features,
+ .ndo_set_mac_address = ionic_set_mac_address,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_tx_timeout = ionic_tx_timeout,
+ .ndo_change_mtu = ionic_change_mtu,
+ .ndo_vlan_rx_add_vid = ionic_vlan_rx_add_vid,
+ .ndo_vlan_rx_kill_vid = ionic_vlan_rx_kill_vid,
+};
+
+int ionic_reset_queues(struct ionic_lif *lif)
+{
+ bool running;
+ int err = 0;
+
+ /* Put off the next watchdog timeout */
+ netif_trans_update(lif->netdev);
+
+ if (!ionic_wait_for_bit(lif, IONIC_LIF_QUEUE_RESET))
+ return -EBUSY;
+
+ running = netif_running(lif->netdev);
+ if (running)
+ err = ionic_stop(lif->netdev);
+ if (!err && running)
+ ionic_open(lif->netdev);
+
+ clear_bit(IONIC_LIF_QUEUE_RESET, lif->state);
+
+ return err;
+}
+
+static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index)
+{
+ struct device *dev = ionic->dev;
+ struct net_device *netdev;
+ struct ionic_lif *lif;
+ int tbl_sz;
+ u32 coal;
+ int err;
+
+ netdev = alloc_etherdev_mqs(sizeof(*lif),
+ ionic->ntxqs_per_lif, ionic->ntxqs_per_lif);
+ if (!netdev) {
+ dev_err(dev, "Cannot allocate netdev, aborting\n");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ SET_NETDEV_DEV(netdev, dev);
+
+ lif = netdev_priv(netdev);
+ lif->netdev = netdev;
+ ionic->master_lif = lif;
+ netdev->netdev_ops = &ionic_netdev_ops;
+ ionic_ethtool_set_ops(netdev);
+
+ netdev->watchdog_timeo = 2 * HZ;
+ netdev->min_mtu = IONIC_MIN_MTU;
+ netdev->max_mtu = IONIC_MAX_MTU;
+
+ lif->neqs = ionic->neqs_per_lif;
+ lif->nxqs = ionic->ntxqs_per_lif;
+
+ lif->ionic = ionic;
+ lif->index = index;
+ lif->ntxq_descs = IONIC_DEF_TXRX_DESC;
+ lif->nrxq_descs = IONIC_DEF_TXRX_DESC;
+
+ /* Convert the default coalesce value to actual hw resolution */
+ coal = ionic_coal_usec_to_hw(lif->ionic, IONIC_ITR_COAL_USEC_DEFAULT);
+ lif->rx_coalesce_usecs = ionic_coal_hw_to_usec(lif->ionic, coal);
+
+ snprintf(lif->name, sizeof(lif->name), "lif%u", index);
+
+ spin_lock_init(&lif->adminq_lock);
+
+ spin_lock_init(&lif->deferred.lock);
+ INIT_LIST_HEAD(&lif->deferred.list);
+ INIT_WORK(&lif->deferred.work, ionic_lif_deferred_work);
+
+ /* allocate lif info */
+ lif->info_sz = ALIGN(sizeof(*lif->info), PAGE_SIZE);
+ lif->info = dma_alloc_coherent(dev, lif->info_sz,
+ &lif->info_pa, GFP_KERNEL);
+ if (!lif->info) {
+ dev_err(dev, "Failed to allocate lif info, aborting\n");
+ err = -ENOMEM;
+ goto err_out_free_netdev;
+ }
+
+ /* allocate queues */
+ err = ionic_qcqs_alloc(lif);
+ if (err)
+ goto err_out_free_lif_info;
+
+ /* allocate rss indirection table */
+ tbl_sz = le16_to_cpu(lif->ionic->ident.lif.eth.rss_ind_tbl_sz);
+ lif->rss_ind_tbl_sz = sizeof(*lif->rss_ind_tbl) * tbl_sz;
+ lif->rss_ind_tbl = dma_alloc_coherent(dev, lif->rss_ind_tbl_sz,
+ &lif->rss_ind_tbl_pa,
+ GFP_KERNEL);
+
+ if (!lif->rss_ind_tbl) {
+ dev_err(dev, "Failed to allocate rss indirection table, aborting\n");
+ goto err_out_free_qcqs;
+ }
+
+ list_add_tail(&lif->list, &ionic->lifs);
+
+ return lif;
+
+err_out_free_qcqs:
+ ionic_qcqs_free(lif);
+err_out_free_lif_info:
+ dma_free_coherent(dev, lif->info_sz, lif->info, lif->info_pa);
+ lif->info = NULL;
+ lif->info_pa = 0;
+err_out_free_netdev:
+ free_netdev(lif->netdev);
+ lif = NULL;
+
+ return ERR_PTR(err);
+}
+
+int ionic_lifs_alloc(struct ionic *ionic)
+{
+ struct ionic_lif *lif;
+
+ INIT_LIST_HEAD(&ionic->lifs);
+
+ /* only build the first lif, others are for later features */
+ set_bit(0, ionic->lifbits);
+ lif = ionic_lif_alloc(ionic, 0);
+
+ return PTR_ERR_OR_ZERO(lif);
+}
+
+static void ionic_lif_reset(struct ionic_lif *lif)
+{
+ struct ionic_dev *idev = &lif->ionic->idev;
+
+ mutex_lock(&lif->ionic->dev_cmd_lock);
+ ionic_dev_cmd_lif_reset(idev, lif->index);
+ ionic_dev_cmd_wait(lif->ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&lif->ionic->dev_cmd_lock);
+}
+
+static void ionic_lif_free(struct ionic_lif *lif)
+{
+ struct device *dev = lif->ionic->dev;
+
+ /* free rss indirection table */
+ dma_free_coherent(dev, lif->rss_ind_tbl_sz, lif->rss_ind_tbl,
+ lif->rss_ind_tbl_pa);
+ lif->rss_ind_tbl = NULL;
+ lif->rss_ind_tbl_pa = 0;
+
+ /* free queues */
+ ionic_qcqs_free(lif);
+ ionic_lif_reset(lif);
+
+ /* free lif info */
+ dma_free_coherent(dev, lif->info_sz, lif->info, lif->info_pa);
+ lif->info = NULL;
+ lif->info_pa = 0;
+
+ /* unmap doorbell page */
+ ionic_bus_unmap_dbpage(lif->ionic, lif->kern_dbpage);
+ lif->kern_dbpage = NULL;
+ kfree(lif->dbid_inuse);
+ lif->dbid_inuse = NULL;
+
+ /* free netdev & lif */
+ ionic_debugfs_del_lif(lif);
+ list_del(&lif->list);
+ free_netdev(lif->netdev);
+}
+
+void ionic_lifs_free(struct ionic *ionic)
+{
+ struct list_head *cur, *tmp;
+ struct ionic_lif *lif;
+
+ list_for_each_safe(cur, tmp, &ionic->lifs) {
+ lif = list_entry(cur, struct ionic_lif, list);
+
+ ionic_lif_free(lif);
+ }
+}
+
+static void ionic_lif_deinit(struct ionic_lif *lif)
+{
+ if (!test_bit(IONIC_LIF_INITED, lif->state))
+ return;
+
+ clear_bit(IONIC_LIF_INITED, lif->state);
+
+ ionic_rx_filters_deinit(lif);
+ ionic_lif_rss_deinit(lif);
+
+ napi_disable(&lif->adminqcq->napi);
+ ionic_lif_qcq_deinit(lif, lif->notifyqcq);
+ ionic_lif_qcq_deinit(lif, lif->adminqcq);
+
+ ionic_lif_reset(lif);
+}
+
+void ionic_lifs_deinit(struct ionic *ionic)
+{
+ struct list_head *cur, *tmp;
+ struct ionic_lif *lif;
+
+ list_for_each_safe(cur, tmp, &ionic->lifs) {
+ lif = list_entry(cur, struct ionic_lif, list);
+ ionic_lif_deinit(lif);
+ }
+}
+
+static int ionic_lif_adminq_init(struct ionic_lif *lif)
+{
+ struct device *dev = lif->ionic->dev;
+ struct ionic_q_init_comp comp;
+ struct ionic_dev *idev;
+ struct ionic_qcq *qcq;
+ struct ionic_queue *q;
+ int err;
+
+ idev = &lif->ionic->idev;
+ qcq = lif->adminqcq;
+ q = &qcq->q;
+
+ mutex_lock(&lif->ionic->dev_cmd_lock);
+ ionic_dev_cmd_adminq_init(idev, qcq, lif->index, qcq->intr.index);
+ err = ionic_dev_cmd_wait(lif->ionic, DEVCMD_TIMEOUT);
+ ionic_dev_cmd_comp(idev, (union ionic_dev_cmd_comp *)&comp);
+ mutex_unlock(&lif->ionic->dev_cmd_lock);
+ if (err) {
+ netdev_err(lif->netdev, "adminq init failed %d\n", err);
+ return err;
+ }
+
+ q->hw_type = comp.hw_type;
+ q->hw_index = le32_to_cpu(comp.hw_index);
+ q->dbval = IONIC_DBELL_QID(q->hw_index);
+
+ dev_dbg(dev, "adminq->hw_type %d\n", q->hw_type);
+ dev_dbg(dev, "adminq->hw_index %d\n", q->hw_index);
+
+ netif_napi_add(lif->netdev, &qcq->napi, ionic_adminq_napi,
+ NAPI_POLL_WEIGHT);
+
+ err = ionic_request_irq(lif, qcq);
+ if (err) {
+ netdev_warn(lif->netdev, "adminq irq request failed %d\n", err);
+ netif_napi_del(&qcq->napi);
+ return err;
+ }
+
+ napi_enable(&qcq->napi);
+
+ if (qcq->flags & IONIC_QCQ_F_INTR)
+ ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
+ IONIC_INTR_MASK_CLEAR);
+
+ qcq->flags |= IONIC_QCQ_F_INITED;
+
+ ionic_debugfs_add_qcq(lif, qcq);
+
+ return 0;
+}
+
+static int ionic_lif_notifyq_init(struct ionic_lif *lif)
+{
+ struct ionic_qcq *qcq = lif->notifyqcq;
+ struct device *dev = lif->ionic->dev;
+ struct ionic_queue *q = &qcq->q;
+ int err;
+
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.q_init = {
+ .opcode = IONIC_CMD_Q_INIT,
+ .lif_index = cpu_to_le16(lif->index),
+ .type = q->type,
+ .index = cpu_to_le32(q->index),
+ .flags = cpu_to_le16(IONIC_QINIT_F_IRQ |
+ IONIC_QINIT_F_ENA),
+ .intr_index = cpu_to_le16(lif->adminqcq->intr.index),
+ .pid = cpu_to_le16(q->pid),
+ .ring_size = ilog2(q->num_descs),
+ .ring_base = cpu_to_le64(q->base_pa),
+ }
+ };
+
+ dev_dbg(dev, "notifyq_init.pid %d\n", ctx.cmd.q_init.pid);
+ dev_dbg(dev, "notifyq_init.index %d\n", ctx.cmd.q_init.index);
+ dev_dbg(dev, "notifyq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
+ dev_dbg(dev, "notifyq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ q->hw_type = ctx.comp.q_init.hw_type;
+ q->hw_index = le32_to_cpu(ctx.comp.q_init.hw_index);
+ q->dbval = IONIC_DBELL_QID(q->hw_index);
+
+ dev_dbg(dev, "notifyq->hw_type %d\n", q->hw_type);
+ dev_dbg(dev, "notifyq->hw_index %d\n", q->hw_index);
+
+ /* preset the callback info */
+ q->info[0].cb_arg = lif;
+
+ qcq->flags |= IONIC_QCQ_F_INITED;
+
+ ionic_debugfs_add_qcq(lif, qcq);
+
+ return 0;
+}
+
+static int ionic_station_set(struct ionic_lif *lif)
+{
+ struct net_device *netdev = lif->netdev;
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.lif_getattr = {
+ .opcode = IONIC_CMD_LIF_GETATTR,
+ .index = cpu_to_le16(lif->index),
+ .attr = IONIC_LIF_ATTR_MAC,
+ },
+ };
+ struct sockaddr addr;
+ int err;
+
+ err = ionic_adminq_post_wait(lif, &ctx);
+ if (err)
+ return err;
+
+ memcpy(addr.sa_data, ctx.comp.lif_getattr.mac, netdev->addr_len);
+ addr.sa_family = AF_INET;
+ err = eth_prepare_mac_addr_change(netdev, &addr);
+ if (err)
+ return err;
+
+ if (!is_zero_ether_addr(netdev->dev_addr)) {
+ netdev_dbg(lif->netdev, "deleting station MAC addr %pM\n",
+ netdev->dev_addr);
+ ionic_lif_addr(lif, netdev->dev_addr, false);
+ }
+
+ eth_commit_mac_addr_change(netdev, &addr);
+ netdev_dbg(lif->netdev, "adding station MAC addr %pM\n",
+ netdev->dev_addr);
+ ionic_lif_addr(lif, netdev->dev_addr, true);
+
+ return 0;
+}
+
+static int ionic_lif_init(struct ionic_lif *lif)
+{
+ struct ionic_dev *idev = &lif->ionic->idev;
+ struct device *dev = lif->ionic->dev;
+ struct ionic_lif_init_comp comp;
+ int dbpage_num;
+ int err;
+
+ ionic_debugfs_add_lif(lif);
+
+ mutex_lock(&lif->ionic->dev_cmd_lock);
+ ionic_dev_cmd_lif_init(idev, lif->index, lif->info_pa);
+ err = ionic_dev_cmd_wait(lif->ionic, DEVCMD_TIMEOUT);
+ ionic_dev_cmd_comp(idev, (union ionic_dev_cmd_comp *)&comp);
+ mutex_unlock(&lif->ionic->dev_cmd_lock);
+ if (err)
+ return err;
+
+ lif->hw_index = le16_to_cpu(comp.hw_index);
+
+ /* now that we have the hw_index we can figure out our doorbell page */
+ lif->dbid_count = le32_to_cpu(lif->ionic->ident.dev.ndbpgs_per_lif);
+ if (!lif->dbid_count) {
+ dev_err(dev, "No doorbell pages, aborting\n");
+ return -EINVAL;
+ }
+
+ lif->dbid_inuse = bitmap_alloc(lif->dbid_count, GFP_KERNEL);
+ if (!lif->dbid_inuse) {
+ dev_err(dev, "Failed alloc doorbell id bitmap, aborting\n");
+ return -ENOMEM;
+ }
+
+ /* first doorbell id reserved for kernel (dbid aka pid == zero) */
+ set_bit(0, lif->dbid_inuse);
+ lif->kern_pid = 0;
+
+ dbpage_num = ionic_db_page_num(lif, lif->kern_pid);
+ lif->kern_dbpage = ionic_bus_map_dbpage(lif->ionic, dbpage_num);
+ if (!lif->kern_dbpage) {
+ dev_err(dev, "Cannot map dbpage, aborting\n");
+ err = -ENOMEM;
+ goto err_out_free_dbid;
+ }
+
+ err = ionic_lif_adminq_init(lif);
+ if (err)
+ goto err_out_adminq_deinit;
+
+ if (lif->ionic->nnqs_per_lif) {
+ err = ionic_lif_notifyq_init(lif);
+ if (err)
+ goto err_out_notifyq_deinit;
+ }
+
+ err = ionic_init_nic_features(lif);
+ if (err)
+ goto err_out_notifyq_deinit;
+
+ err = ionic_rx_filters_init(lif);
+ if (err)
+ goto err_out_notifyq_deinit;
+
+ err = ionic_station_set(lif);
+ if (err)
+ goto err_out_notifyq_deinit;
+
+ lif->rx_copybreak = IONIC_RX_COPYBREAK_DEFAULT;
+
+ set_bit(IONIC_LIF_INITED, lif->state);
+
+ INIT_WORK(&lif->tx_timeout_work, ionic_tx_timeout_work);
+
+ return 0;
+
+err_out_notifyq_deinit:
+ ionic_lif_qcq_deinit(lif, lif->notifyqcq);
+err_out_adminq_deinit:
+ ionic_lif_qcq_deinit(lif, lif->adminqcq);
+ ionic_lif_reset(lif);
+ ionic_bus_unmap_dbpage(lif->ionic, lif->kern_dbpage);
+ lif->kern_dbpage = NULL;
+err_out_free_dbid:
+ kfree(lif->dbid_inuse);
+ lif->dbid_inuse = NULL;
+
+ return err;
+}
+
+int ionic_lifs_init(struct ionic *ionic)
+{
+ struct list_head *cur, *tmp;
+ struct ionic_lif *lif;
+ int err;
+
+ list_for_each_safe(cur, tmp, &ionic->lifs) {
+ lif = list_entry(cur, struct ionic_lif, list);
+ err = ionic_lif_init(lif);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static void ionic_lif_notify_work(struct work_struct *ws)
+{
+}
+
+static void ionic_lif_set_netdev_info(struct ionic_lif *lif)
+{
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.lif_setattr = {
+ .opcode = IONIC_CMD_LIF_SETATTR,
+ .index = cpu_to_le16(lif->index),
+ .attr = IONIC_LIF_ATTR_NAME,
+ },
+ };
+
+ strlcpy(ctx.cmd.lif_setattr.name, lif->netdev->name,
+ sizeof(ctx.cmd.lif_setattr.name));
+
+ ionic_adminq_post_wait(lif, &ctx);
+}
+
+static struct ionic_lif *ionic_netdev_lif(struct net_device *netdev)
+{
+ if (!netdev || netdev->netdev_ops->ndo_start_xmit != ionic_start_xmit)
+ return NULL;
+
+ return netdev_priv(netdev);
+}
+
+static int ionic_lif_notify(struct notifier_block *nb,
+ unsigned long event, void *info)
+{
+ struct net_device *ndev = netdev_notifier_info_to_dev(info);
+ struct ionic *ionic = container_of(nb, struct ionic, nb);
+ struct ionic_lif *lif = ionic_netdev_lif(ndev);
+
+ if (!lif || lif->ionic != ionic)
+ return NOTIFY_DONE;
+
+ switch (event) {
+ case NETDEV_CHANGENAME:
+ ionic_lif_set_netdev_info(lif);
+ break;
+ }
+
+ return NOTIFY_DONE;
+}
+
+int ionic_lifs_register(struct ionic *ionic)
+{
+ int err;
+
+ INIT_WORK(&ionic->nb_work, ionic_lif_notify_work);
+
+ ionic->nb.notifier_call = ionic_lif_notify;
+
+ err = register_netdevice_notifier(&ionic->nb);
+ if (err)
+ ionic->nb.notifier_call = NULL;
+
+ /* only register LIF0 for now */
+ err = register_netdev(ionic->master_lif->netdev);
+ if (err) {
+ dev_err(ionic->dev, "Cannot register net device, aborting\n");
+ return err;
+ }
+
+ ionic_link_status_check_request(ionic->master_lif);
+ ionic->master_lif->registered = true;
+
+ return 0;
+}
+
+void ionic_lifs_unregister(struct ionic *ionic)
+{
+ if (ionic->nb.notifier_call) {
+ unregister_netdevice_notifier(&ionic->nb);
+ cancel_work_sync(&ionic->nb_work);
+ ionic->nb.notifier_call = NULL;
+ }
+
+ /* There is only one lif ever registered in the
+ * current model, so don't bother searching the
+ * ionic->lif for candidates to unregister
+ */
+ cancel_work_sync(&ionic->master_lif->deferred.work);
+ cancel_work_sync(&ionic->master_lif->tx_timeout_work);
+ if (ionic->master_lif->netdev->reg_state == NETREG_REGISTERED)
+ unregister_netdev(ionic->master_lif->netdev);
+}
+
+int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
+ union ionic_lif_identity *lid)
+{
+ struct ionic_dev *idev = &ionic->idev;
+ size_t sz;
+ int err;
+
+ sz = min(sizeof(*lid), sizeof(idev->dev_cmd_regs->data));
+
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_lif_identify(idev, lif_type, IONIC_IDENTITY_VERSION_1);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ memcpy_fromio(lid, &idev->dev_cmd_regs->data, sz);
+ mutex_unlock(&ionic->dev_cmd_lock);
+ if (err)
+ return (err);
+
+ dev_dbg(ionic->dev, "capabilities 0x%llx\n",
+ le64_to_cpu(lid->capabilities));
+
+ dev_dbg(ionic->dev, "eth.max_ucast_filters %d\n",
+ le32_to_cpu(lid->eth.max_ucast_filters));
+ dev_dbg(ionic->dev, "eth.max_mcast_filters %d\n",
+ le32_to_cpu(lid->eth.max_mcast_filters));
+ dev_dbg(ionic->dev, "eth.features 0x%llx\n",
+ le64_to_cpu(lid->eth.config.features));
+ dev_dbg(ionic->dev, "eth.queue_count[IONIC_QTYPE_ADMINQ] %d\n",
+ le32_to_cpu(lid->eth.config.queue_count[IONIC_QTYPE_ADMINQ]));
+ dev_dbg(ionic->dev, "eth.queue_count[IONIC_QTYPE_NOTIFYQ] %d\n",
+ le32_to_cpu(lid->eth.config.queue_count[IONIC_QTYPE_NOTIFYQ]));
+ dev_dbg(ionic->dev, "eth.queue_count[IONIC_QTYPE_RXQ] %d\n",
+ le32_to_cpu(lid->eth.config.queue_count[IONIC_QTYPE_RXQ]));
+ dev_dbg(ionic->dev, "eth.queue_count[IONIC_QTYPE_TXQ] %d\n",
+ le32_to_cpu(lid->eth.config.queue_count[IONIC_QTYPE_TXQ]));
+ dev_dbg(ionic->dev, "eth.config.name %s\n", lid->eth.config.name);
+ dev_dbg(ionic->dev, "eth.config.mac %pM\n", lid->eth.config.mac);
+ dev_dbg(ionic->dev, "eth.config.mtu %d\n",
+ le32_to_cpu(lid->eth.config.mtu));
+
+ return 0;
+}
+
+int ionic_lifs_size(struct ionic *ionic)
+{
+ struct ionic_identity *ident = &ionic->ident;
+ unsigned int nintrs, dev_nintrs;
+ union ionic_lif_config *lc;
+ unsigned int ntxqs_per_lif;
+ unsigned int nrxqs_per_lif;
+ unsigned int neqs_per_lif;
+ unsigned int nnqs_per_lif;
+ unsigned int nxqs, neqs;
+ unsigned int min_intrs;
+ int err;
+
+ lc = &ident->lif.eth.config;
+ dev_nintrs = le32_to_cpu(ident->dev.nintrs);
+ neqs_per_lif = le32_to_cpu(ident->lif.rdma.eq_qtype.qid_count);
+ nnqs_per_lif = le32_to_cpu(lc->queue_count[IONIC_QTYPE_NOTIFYQ]);
+ ntxqs_per_lif = le32_to_cpu(lc->queue_count[IONIC_QTYPE_TXQ]);
+ nrxqs_per_lif = le32_to_cpu(lc->queue_count[IONIC_QTYPE_RXQ]);
+
+ nxqs = min(ntxqs_per_lif, nrxqs_per_lif);
+ nxqs = min(nxqs, num_online_cpus());
+ neqs = min(neqs_per_lif, num_online_cpus());
+
+try_again:
+ /* interrupt usage:
+ * 1 for master lif adminq/notifyq
+ * 1 for each CPU for master lif TxRx queue pairs
+ * whatever's left is for RDMA queues
+ */
+ nintrs = 1 + nxqs + neqs;
+ min_intrs = 2; /* adminq + 1 TxRx queue pair */
+
+ if (nintrs > dev_nintrs)
+ goto try_fewer;
+
+ err = ionic_bus_alloc_irq_vectors(ionic, nintrs);
+ if (err < 0 && err != -ENOSPC) {
+ dev_err(ionic->dev, "Can't get intrs from OS: %d\n", err);
+ return err;
+ }
+ if (err == -ENOSPC)
+ goto try_fewer;
+
+ if (err != nintrs) {
+ ionic_bus_free_irq_vectors(ionic);
+ goto try_fewer;
+ }
+
+ ionic->nnqs_per_lif = nnqs_per_lif;
+ ionic->neqs_per_lif = neqs;
+ ionic->ntxqs_per_lif = nxqs;
+ ionic->nrxqs_per_lif = nxqs;
+ ionic->nintrs = nintrs;
+
+ ionic_debugfs_add_sizes(ionic);
+
+ return 0;
+
+try_fewer:
+ if (nnqs_per_lif > 1) {
+ nnqs_per_lif >>= 1;
+ goto try_again;
+ }
+ if (neqs > 1) {
+ neqs >>= 1;
+ goto try_again;
+ }
+ if (nxqs > 1) {
+ nxqs >>= 1;
+ goto try_again;
+ }
+ dev_err(ionic->dev, "Can't get minimum %d intrs from OS\n", min_intrs);
+ return -ENOSPC;
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h
new file mode 100644
index 0000000..812190e
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h
@@ -0,0 +1,277 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_LIF_H_
+#define _IONIC_LIF_H_
+
+#include <linux/pci.h>
+#include "ionic_rx_filter.h"
+
+#define IONIC_ADMINQ_LENGTH 16 /* must be a power of two */
+#define IONIC_NOTIFYQ_LENGTH 64 /* must be a power of two */
+
+#define IONIC_MAX_NUM_NAPI_CNTR (NAPI_POLL_WEIGHT + 1)
+#define IONIC_MAX_NUM_SG_CNTR (IONIC_TX_MAX_SG_ELEMS + 1)
+#define IONIC_RX_COPYBREAK_DEFAULT 256
+
+struct ionic_tx_stats {
+ u64 dma_map_err;
+ u64 pkts;
+ u64 bytes;
+ u64 clean;
+ u64 linearize;
+ u64 no_csum;
+ u64 csum;
+ u64 crc32_csum;
+ u64 tso;
+ u64 frags;
+ u64 sg_cntr[IONIC_MAX_NUM_SG_CNTR];
+};
+
+struct ionic_rx_stats {
+ u64 dma_map_err;
+ u64 alloc_err;
+ u64 pkts;
+ u64 bytes;
+ u64 csum_none;
+ u64 csum_complete;
+ u64 csum_error;
+ u64 buffers_posted;
+};
+
+#define IONIC_QCQ_F_INITED BIT(0)
+#define IONIC_QCQ_F_SG BIT(1)
+#define IONIC_QCQ_F_INTR BIT(2)
+#define IONIC_QCQ_F_TX_STATS BIT(3)
+#define IONIC_QCQ_F_RX_STATS BIT(4)
+#define IONIC_QCQ_F_NOTIFYQ BIT(5)
+
+struct ionic_napi_stats {
+ u64 poll_count;
+ u64 work_done_cntr[IONIC_MAX_NUM_NAPI_CNTR];
+};
+
+struct ionic_q_stats {
+ union {
+ struct ionic_tx_stats tx;
+ struct ionic_rx_stats rx;
+ };
+};
+
+struct ionic_qcq {
+ void *base;
+ dma_addr_t base_pa;
+ unsigned int total_size;
+ struct ionic_queue q;
+ struct ionic_cq cq;
+ struct ionic_intr_info intr;
+ struct napi_struct napi;
+ struct ionic_napi_stats napi_stats;
+ struct ionic_q_stats *stats;
+ unsigned int flags;
+ struct dentry *dentry;
+};
+
+struct ionic_qcqst {
+ struct ionic_qcq *qcq;
+ struct ionic_q_stats *stats;
+};
+
+#define q_to_qcq(q) container_of(q, struct ionic_qcq, q)
+#define q_to_tx_stats(q) (&q_to_qcq(q)->stats->tx)
+#define q_to_rx_stats(q) (&q_to_qcq(q)->stats->rx)
+#define napi_to_qcq(napi) container_of(napi, struct ionic_qcq, napi)
+#define napi_to_cq(napi) (&napi_to_qcq(napi)->cq)
+
+enum ionic_deferred_work_type {
+ IONIC_DW_TYPE_RX_MODE,
+ IONIC_DW_TYPE_RX_ADDR_ADD,
+ IONIC_DW_TYPE_RX_ADDR_DEL,
+ IONIC_DW_TYPE_LINK_STATUS,
+ IONIC_DW_TYPE_LIF_RESET,
+};
+
+struct ionic_deferred_work {
+ struct list_head list;
+ enum ionic_deferred_work_type type;
+ union {
+ unsigned int rx_mode;
+ u8 addr[ETH_ALEN];
+ };
+};
+
+struct ionic_deferred {
+ spinlock_t lock; /* lock for deferred work list */
+ struct list_head list;
+ struct work_struct work;
+};
+
+struct ionic_lif_sw_stats {
+ u64 tx_packets;
+ u64 tx_bytes;
+ u64 rx_packets;
+ u64 rx_bytes;
+ u64 tx_tso;
+ u64 tx_no_csum;
+ u64 tx_csum;
+ u64 rx_csum_none;
+ u64 rx_csum_complete;
+ u64 rx_csum_error;
+};
+
+enum ionic_lif_state_flags {
+ IONIC_LIF_INITED,
+ IONIC_LIF_SW_DEBUG_STATS,
+ IONIC_LIF_UP,
+ IONIC_LIF_LINK_CHECK_REQUESTED,
+ IONIC_LIF_QUEUE_RESET,
+
+ /* leave this as last */
+ IONIC_LIF_STATE_SIZE
+};
+
+#define IONIC_LIF_NAME_MAX_SZ 32
+struct ionic_lif {
+ char name[IONIC_LIF_NAME_MAX_SZ];
+ struct list_head list;
+ struct net_device *netdev;
+ DECLARE_BITMAP(state, IONIC_LIF_STATE_SIZE);
+ struct ionic *ionic;
+ bool registered;
+ unsigned int index;
+ unsigned int hw_index;
+ unsigned int kern_pid;
+ u64 __iomem *kern_dbpage;
+ spinlock_t adminq_lock; /* lock for AdminQ operations */
+ struct ionic_qcq *adminqcq;
+ struct ionic_qcq *notifyqcq;
+ struct ionic_qcqst *txqcqs;
+ struct ionic_qcqst *rxqcqs;
+ u64 last_eid;
+ unsigned int neqs;
+ unsigned int nxqs;
+ unsigned int ntxq_descs;
+ unsigned int nrxq_descs;
+ u32 rx_copybreak;
+ unsigned int rx_mode;
+ u64 hw_features;
+ bool mc_overflow;
+ unsigned int nmcast;
+ bool uc_overflow;
+ unsigned int nucast;
+
+ struct ionic_lif_info *info;
+ dma_addr_t info_pa;
+ u32 info_sz;
+
+ u16 rss_types;
+ u8 rss_hash_key[IONIC_RSS_HASH_KEY_SIZE];
+ u8 *rss_ind_tbl;
+ dma_addr_t rss_ind_tbl_pa;
+ u32 rss_ind_tbl_sz;
+
+ struct ionic_rx_filters rx_filters;
+ struct ionic_deferred deferred;
+ unsigned long *dbid_inuse;
+ unsigned int dbid_count;
+ struct dentry *dentry;
+ u32 rx_coalesce_usecs;
+ u32 flags;
+ struct work_struct tx_timeout_work;
+};
+
+#define lif_to_txqcq(lif, i) ((lif)->txqcqs[i].qcq)
+#define lif_to_rxqcq(lif, i) ((lif)->rxqcqs[i].qcq)
+#define lif_to_txq(lif, i) (&lif_to_txqcq((lif), i)->q)
+#define lif_to_rxq(lif, i) (&lif_to_txqcq((lif), i)->q)
+
+static inline int ionic_wait_for_bit(struct ionic_lif *lif, int bitname)
+{
+ unsigned long tlimit = jiffies + HZ;
+
+ while (test_and_set_bit(bitname, lif->state) &&
+ time_before(jiffies, tlimit))
+ usleep_range(100, 200);
+
+ return test_bit(bitname, lif->state);
+}
+
+static inline u32 ionic_coal_usec_to_hw(struct ionic *ionic, u32 usecs)
+{
+ u32 mult = le32_to_cpu(ionic->ident.dev.intr_coal_mult);
+ u32 div = le32_to_cpu(ionic->ident.dev.intr_coal_div);
+
+ /* Div-by-zero should never be an issue, but check anyway */
+ if (!div || !mult)
+ return 0;
+
+ /* Round up in case usecs is close to the next hw unit */
+ usecs += (div / mult) >> 1;
+
+ /* Convert from usecs to device units */
+ return (usecs * mult) / div;
+}
+
+static inline u32 ionic_coal_hw_to_usec(struct ionic *ionic, u32 units)
+{
+ u32 mult = le32_to_cpu(ionic->ident.dev.intr_coal_mult);
+ u32 div = le32_to_cpu(ionic->ident.dev.intr_coal_div);
+
+ /* Div-by-zero should never be an issue, but check anyway */
+ if (!div || !mult)
+ return 0;
+
+ /* Convert from device units to usec */
+ return (units * div) / mult;
+}
+
+int ionic_lifs_alloc(struct ionic *ionic);
+void ionic_lifs_free(struct ionic *ionic);
+void ionic_lifs_deinit(struct ionic *ionic);
+int ionic_lifs_init(struct ionic *ionic);
+int ionic_lifs_register(struct ionic *ionic);
+void ionic_lifs_unregister(struct ionic *ionic);
+int ionic_lif_identify(struct ionic *ionic, u8 lif_type,
+ union ionic_lif_identity *lif_ident);
+int ionic_lifs_size(struct ionic *ionic);
+int ionic_lif_rss_config(struct ionic_lif *lif, u16 types,
+ const u8 *key, const u32 *indir);
+
+int ionic_open(struct net_device *netdev);
+int ionic_stop(struct net_device *netdev);
+int ionic_reset_queues(struct ionic_lif *lif);
+
+static inline void debug_stats_txq_post(struct ionic_qcq *qcq,
+ struct ionic_txq_desc *desc, bool dbell)
+{
+ u8 num_sg_elems = ((le64_to_cpu(desc->cmd) >> IONIC_TXQ_DESC_NSGE_SHIFT)
+ & IONIC_TXQ_DESC_NSGE_MASK);
+
+ qcq->q.dbell_count += dbell;
+
+ if (num_sg_elems > (IONIC_MAX_NUM_SG_CNTR - 1))
+ num_sg_elems = IONIC_MAX_NUM_SG_CNTR - 1;
+
+ qcq->stats->tx.sg_cntr[num_sg_elems]++;
+}
+
+static inline void debug_stats_napi_poll(struct ionic_qcq *qcq,
+ unsigned int work_done)
+{
+ qcq->napi_stats.poll_count++;
+
+ if (work_done > (IONIC_MAX_NUM_NAPI_CNTR - 1))
+ work_done = IONIC_MAX_NUM_NAPI_CNTR - 1;
+
+ qcq->napi_stats.work_done_cntr[work_done]++;
+}
+
+#define DEBUG_STATS_CQE_CNT(cq) ((cq)->compl_count++)
+#define DEBUG_STATS_RX_BUFF_CNT(qcq) ((qcq)->stats->rx.buffers_posted++)
+#define DEBUG_STATS_INTR_REARM(intr) ((intr)->rearm_count++)
+#define DEBUG_STATS_TXQ_POST(qcq, txdesc, dbell) \
+ debug_stats_txq_post(qcq, txdesc, dbell)
+#define DEBUG_STATS_NAPI_POLL(qcq, work_done) \
+ debug_stats_napi_poll(qcq, work_done)
+
+#endif /* _IONIC_LIF_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_main.c b/drivers/net/ethernet/pensando/ionic/ionic_main.c
new file mode 100644
index 0000000..15e43238
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_main.c
@@ -0,0 +1,548 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/utsname.h>
+
+#include "ionic.h"
+#include "ionic_bus.h"
+#include "ionic_lif.h"
+#include "ionic_debugfs.h"
+
+MODULE_DESCRIPTION(IONIC_DRV_DESCRIPTION);
+MODULE_AUTHOR("Pensando Systems, Inc");
+MODULE_LICENSE("GPL");
+MODULE_VERSION(IONIC_DRV_VERSION);
+
+static const char *ionic_error_to_str(enum ionic_status_code code)
+{
+ switch (code) {
+ case IONIC_RC_SUCCESS:
+ return "IONIC_RC_SUCCESS";
+ case IONIC_RC_EVERSION:
+ return "IONIC_RC_EVERSION";
+ case IONIC_RC_EOPCODE:
+ return "IONIC_RC_EOPCODE";
+ case IONIC_RC_EIO:
+ return "IONIC_RC_EIO";
+ case IONIC_RC_EPERM:
+ return "IONIC_RC_EPERM";
+ case IONIC_RC_EQID:
+ return "IONIC_RC_EQID";
+ case IONIC_RC_EQTYPE:
+ return "IONIC_RC_EQTYPE";
+ case IONIC_RC_ENOENT:
+ return "IONIC_RC_ENOENT";
+ case IONIC_RC_EINTR:
+ return "IONIC_RC_EINTR";
+ case IONIC_RC_EAGAIN:
+ return "IONIC_RC_EAGAIN";
+ case IONIC_RC_ENOMEM:
+ return "IONIC_RC_ENOMEM";
+ case IONIC_RC_EFAULT:
+ return "IONIC_RC_EFAULT";
+ case IONIC_RC_EBUSY:
+ return "IONIC_RC_EBUSY";
+ case IONIC_RC_EEXIST:
+ return "IONIC_RC_EEXIST";
+ case IONIC_RC_EINVAL:
+ return "IONIC_RC_EINVAL";
+ case IONIC_RC_ENOSPC:
+ return "IONIC_RC_ENOSPC";
+ case IONIC_RC_ERANGE:
+ return "IONIC_RC_ERANGE";
+ case IONIC_RC_BAD_ADDR:
+ return "IONIC_RC_BAD_ADDR";
+ case IONIC_RC_DEV_CMD:
+ return "IONIC_RC_DEV_CMD";
+ case IONIC_RC_ERROR:
+ return "IONIC_RC_ERROR";
+ case IONIC_RC_ERDMA:
+ return "IONIC_RC_ERDMA";
+ default:
+ return "IONIC_RC_UNKNOWN";
+ }
+}
+
+static int ionic_error_to_errno(enum ionic_status_code code)
+{
+ switch (code) {
+ case IONIC_RC_SUCCESS:
+ return 0;
+ case IONIC_RC_EVERSION:
+ case IONIC_RC_EQTYPE:
+ case IONIC_RC_EQID:
+ case IONIC_RC_EINVAL:
+ return -EINVAL;
+ case IONIC_RC_EPERM:
+ return -EPERM;
+ case IONIC_RC_ENOENT:
+ return -ENOENT;
+ case IONIC_RC_EAGAIN:
+ return -EAGAIN;
+ case IONIC_RC_ENOMEM:
+ return -ENOMEM;
+ case IONIC_RC_EFAULT:
+ return -EFAULT;
+ case IONIC_RC_EBUSY:
+ return -EBUSY;
+ case IONIC_RC_EEXIST:
+ return -EEXIST;
+ case IONIC_RC_ENOSPC:
+ return -ENOSPC;
+ case IONIC_RC_ERANGE:
+ return -ERANGE;
+ case IONIC_RC_BAD_ADDR:
+ return -EFAULT;
+ case IONIC_RC_EOPCODE:
+ case IONIC_RC_EINTR:
+ case IONIC_RC_DEV_CMD:
+ case IONIC_RC_ERROR:
+ case IONIC_RC_ERDMA:
+ case IONIC_RC_EIO:
+ default:
+ return -EIO;
+ }
+}
+
+static const char *ionic_opcode_to_str(enum ionic_cmd_opcode opcode)
+{
+ switch (opcode) {
+ case IONIC_CMD_NOP:
+ return "IONIC_CMD_NOP";
+ case IONIC_CMD_INIT:
+ return "IONIC_CMD_INIT";
+ case IONIC_CMD_RESET:
+ return "IONIC_CMD_RESET";
+ case IONIC_CMD_IDENTIFY:
+ return "IONIC_CMD_IDENTIFY";
+ case IONIC_CMD_GETATTR:
+ return "IONIC_CMD_GETATTR";
+ case IONIC_CMD_SETATTR:
+ return "IONIC_CMD_SETATTR";
+ case IONIC_CMD_PORT_IDENTIFY:
+ return "IONIC_CMD_PORT_IDENTIFY";
+ case IONIC_CMD_PORT_INIT:
+ return "IONIC_CMD_PORT_INIT";
+ case IONIC_CMD_PORT_RESET:
+ return "IONIC_CMD_PORT_RESET";
+ case IONIC_CMD_PORT_GETATTR:
+ return "IONIC_CMD_PORT_GETATTR";
+ case IONIC_CMD_PORT_SETATTR:
+ return "IONIC_CMD_PORT_SETATTR";
+ case IONIC_CMD_LIF_INIT:
+ return "IONIC_CMD_LIF_INIT";
+ case IONIC_CMD_LIF_RESET:
+ return "IONIC_CMD_LIF_RESET";
+ case IONIC_CMD_LIF_IDENTIFY:
+ return "IONIC_CMD_LIF_IDENTIFY";
+ case IONIC_CMD_LIF_SETATTR:
+ return "IONIC_CMD_LIF_SETATTR";
+ case IONIC_CMD_LIF_GETATTR:
+ return "IONIC_CMD_LIF_GETATTR";
+ case IONIC_CMD_RX_MODE_SET:
+ return "IONIC_CMD_RX_MODE_SET";
+ case IONIC_CMD_RX_FILTER_ADD:
+ return "IONIC_CMD_RX_FILTER_ADD";
+ case IONIC_CMD_RX_FILTER_DEL:
+ return "IONIC_CMD_RX_FILTER_DEL";
+ case IONIC_CMD_Q_INIT:
+ return "IONIC_CMD_Q_INIT";
+ case IONIC_CMD_Q_CONTROL:
+ return "IONIC_CMD_Q_CONTROL";
+ case IONIC_CMD_RDMA_RESET_LIF:
+ return "IONIC_CMD_RDMA_RESET_LIF";
+ case IONIC_CMD_RDMA_CREATE_EQ:
+ return "IONIC_CMD_RDMA_CREATE_EQ";
+ case IONIC_CMD_RDMA_CREATE_CQ:
+ return "IONIC_CMD_RDMA_CREATE_CQ";
+ case IONIC_CMD_RDMA_CREATE_ADMINQ:
+ return "IONIC_CMD_RDMA_CREATE_ADMINQ";
+ case IONIC_CMD_FW_DOWNLOAD:
+ return "IONIC_CMD_FW_DOWNLOAD";
+ case IONIC_CMD_FW_CONTROL:
+ return "IONIC_CMD_FW_CONTROL";
+ default:
+ return "DEVCMD_UNKNOWN";
+ }
+}
+
+static void ionic_adminq_flush(struct ionic_lif *lif)
+{
+ struct ionic_queue *adminq = &lif->adminqcq->q;
+
+ spin_lock(&lif->adminq_lock);
+
+ while (adminq->tail != adminq->head) {
+ memset(adminq->tail->desc, 0, sizeof(union ionic_adminq_cmd));
+ adminq->tail->cb = NULL;
+ adminq->tail->cb_arg = NULL;
+ adminq->tail = adminq->tail->next;
+ }
+ spin_unlock(&lif->adminq_lock);
+}
+
+static int ionic_adminq_check_err(struct ionic_lif *lif,
+ struct ionic_admin_ctx *ctx,
+ bool timeout)
+{
+ struct net_device *netdev = lif->netdev;
+ const char *opcode_str;
+ const char *status_str;
+ int err = 0;
+
+ if (ctx->comp.comp.status || timeout) {
+ opcode_str = ionic_opcode_to_str(ctx->cmd.cmd.opcode);
+ status_str = ionic_error_to_str(ctx->comp.comp.status);
+ err = timeout ? -ETIMEDOUT :
+ ionic_error_to_errno(ctx->comp.comp.status);
+
+ netdev_err(netdev, "%s (%d) failed: %s (%d)\n",
+ opcode_str, ctx->cmd.cmd.opcode,
+ timeout ? "TIMEOUT" : status_str, err);
+
+ if (timeout)
+ ionic_adminq_flush(lif);
+ }
+
+ return err;
+}
+
+static void ionic_adminq_cb(struct ionic_queue *q,
+ struct ionic_desc_info *desc_info,
+ struct ionic_cq_info *cq_info, void *cb_arg)
+{
+ struct ionic_admin_ctx *ctx = cb_arg;
+ struct ionic_admin_comp *comp;
+ struct device *dev;
+
+ if (!ctx)
+ return;
+
+ comp = cq_info->cq_desc;
+ dev = &q->lif->netdev->dev;
+
+ memcpy(&ctx->comp, comp, sizeof(*comp));
+
+ dev_dbg(dev, "comp admin queue command:\n");
+ dynamic_hex_dump("comp ", DUMP_PREFIX_OFFSET, 16, 1,
+ &ctx->comp, sizeof(ctx->comp), true);
+
+ complete_all(&ctx->work);
+}
+
+static int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
+{
+ struct ionic_queue *adminq = &lif->adminqcq->q;
+ int err = 0;
+
+ WARN_ON(in_interrupt());
+
+ spin_lock(&lif->adminq_lock);
+ if (!ionic_q_has_space(adminq, 1)) {
+ err = -ENOSPC;
+ goto err_out;
+ }
+
+ memcpy(adminq->head->desc, &ctx->cmd, sizeof(ctx->cmd));
+
+ dev_dbg(&lif->netdev->dev, "post admin queue command:\n");
+ dynamic_hex_dump("cmd ", DUMP_PREFIX_OFFSET, 16, 1,
+ &ctx->cmd, sizeof(ctx->cmd), true);
+
+ ionic_q_post(adminq, true, ionic_adminq_cb, ctx);
+
+err_out:
+ spin_unlock(&lif->adminq_lock);
+
+ return err;
+}
+
+int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
+{
+ struct net_device *netdev = lif->netdev;
+ unsigned long remaining;
+ const char *name;
+ int err;
+
+ err = ionic_adminq_post(lif, ctx);
+ if (err) {
+ name = ionic_opcode_to_str(ctx->cmd.cmd.opcode);
+ netdev_err(netdev, "Posting of %s (%d) failed: %d\n",
+ name, ctx->cmd.cmd.opcode, err);
+ return err;
+ }
+
+ remaining = wait_for_completion_timeout(&ctx->work,
+ HZ * (ulong)DEVCMD_TIMEOUT);
+ return ionic_adminq_check_err(lif, ctx, (remaining == 0));
+}
+
+int ionic_napi(struct napi_struct *napi, int budget, ionic_cq_cb cb,
+ ionic_cq_done_cb done_cb, void *done_arg)
+{
+ struct ionic_qcq *qcq = napi_to_qcq(napi);
+ struct ionic_cq *cq = &qcq->cq;
+ u32 work_done, flags = 0;
+
+ work_done = ionic_cq_service(cq, budget, cb, done_cb, done_arg);
+
+ if (work_done < budget && napi_complete_done(napi, work_done)) {
+ flags |= IONIC_INTR_CRED_UNMASK;
+ DEBUG_STATS_INTR_REARM(cq->bound_intr);
+ }
+
+ if (work_done || flags) {
+ flags |= IONIC_INTR_CRED_RESET_COALESCE;
+ ionic_intr_credits(cq->lif->ionic->idev.intr_ctrl,
+ cq->bound_intr->index,
+ work_done, flags);
+ }
+
+ DEBUG_STATS_NAPI_POLL(qcq, work_done);
+
+ return work_done;
+}
+
+int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_seconds)
+{
+ struct ionic_dev *idev = &ionic->idev;
+ unsigned long start_time;
+ unsigned long max_wait;
+ unsigned long duration;
+ int opcode;
+ int done;
+ int err;
+
+ WARN_ON(in_interrupt());
+
+ /* Wait for dev cmd to complete, retrying if we get EAGAIN,
+ * but don't wait any longer than max_seconds.
+ */
+ max_wait = jiffies + (max_seconds * HZ);
+try_again:
+ start_time = jiffies;
+ do {
+ done = ionic_dev_cmd_done(idev);
+ if (done)
+ break;
+ msleep(20);
+ } while (!done && time_before(jiffies, max_wait));
+ duration = jiffies - start_time;
+
+ opcode = idev->dev_cmd_regs->cmd.cmd.opcode;
+ dev_dbg(ionic->dev, "DEVCMD %s (%d) done=%d took %ld secs (%ld jiffies)\n",
+ ionic_opcode_to_str(opcode), opcode,
+ done, duration / HZ, duration);
+
+ if (!done && !time_before(jiffies, max_wait)) {
+ dev_warn(ionic->dev, "DEVCMD %s (%d) timeout after %ld secs\n",
+ ionic_opcode_to_str(opcode), opcode, max_seconds);
+ return -ETIMEDOUT;
+ }
+
+ err = ionic_dev_cmd_status(&ionic->idev);
+ if (err) {
+ if (err == IONIC_RC_EAGAIN && !time_after(jiffies, max_wait)) {
+ dev_err(ionic->dev, "DEV_CMD %s (%d) error, %s (%d) retrying...\n",
+ ionic_opcode_to_str(opcode), opcode,
+ ionic_error_to_str(err), err);
+
+ msleep(1000);
+ iowrite32(0, &idev->dev_cmd_regs->done);
+ iowrite32(1, &idev->dev_cmd_regs->doorbell);
+ goto try_again;
+ }
+
+ dev_err(ionic->dev, "DEV_CMD %s (%d) error, %s (%d) failed\n",
+ ionic_opcode_to_str(opcode), opcode,
+ ionic_error_to_str(err), err);
+
+ return ionic_error_to_errno(err);
+ }
+
+ return 0;
+}
+
+int ionic_setup(struct ionic *ionic)
+{
+ int err;
+
+ err = ionic_dev_setup(ionic);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+int ionic_identify(struct ionic *ionic)
+{
+ struct ionic_identity *ident = &ionic->ident;
+ struct ionic_dev *idev = &ionic->idev;
+ size_t sz;
+ int err;
+
+ memset(ident, 0, sizeof(*ident));
+
+ ident->drv.os_type = cpu_to_le32(IONIC_OS_TYPE_LINUX);
+ strncpy(ident->drv.driver_ver_str, IONIC_DRV_VERSION,
+ sizeof(ident->drv.driver_ver_str) - 1);
+
+ mutex_lock(&ionic->dev_cmd_lock);
+
+ sz = min(sizeof(ident->drv), sizeof(idev->dev_cmd_regs->data));
+ memcpy_toio(&idev->dev_cmd_regs->data, &ident->drv, sz);
+
+ ionic_dev_cmd_identify(idev, IONIC_IDENTITY_VERSION_1);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ if (!err) {
+ sz = min(sizeof(ident->dev), sizeof(idev->dev_cmd_regs->data));
+ memcpy_fromio(&ident->dev, &idev->dev_cmd_regs->data, sz);
+ }
+
+ mutex_unlock(&ionic->dev_cmd_lock);
+
+ if (err)
+ goto err_out_unmap;
+
+ ionic_debugfs_add_ident(ionic);
+
+ return 0;
+
+err_out_unmap:
+ return err;
+}
+
+int ionic_init(struct ionic *ionic)
+{
+ struct ionic_dev *idev = &ionic->idev;
+ int err;
+
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_init(idev);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+
+ return err;
+}
+
+int ionic_reset(struct ionic *ionic)
+{
+ struct ionic_dev *idev = &ionic->idev;
+ int err;
+
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_reset(idev);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+
+ return err;
+}
+
+int ionic_port_identify(struct ionic *ionic)
+{
+ struct ionic_identity *ident = &ionic->ident;
+ struct ionic_dev *idev = &ionic->idev;
+ size_t sz;
+ int err;
+
+ mutex_lock(&ionic->dev_cmd_lock);
+
+ ionic_dev_cmd_port_identify(idev);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ if (!err) {
+ sz = min(sizeof(ident->port), sizeof(idev->dev_cmd_regs->data));
+ memcpy_fromio(&ident->port, &idev->dev_cmd_regs->data, sz);
+ }
+
+ mutex_unlock(&ionic->dev_cmd_lock);
+
+ return err;
+}
+
+int ionic_port_init(struct ionic *ionic)
+{
+ struct ionic_identity *ident = &ionic->ident;
+ struct ionic_dev *idev = &ionic->idev;
+ size_t sz;
+ int err;
+
+ if (idev->port_info)
+ return 0;
+
+ idev->port_info_sz = ALIGN(sizeof(*idev->port_info), PAGE_SIZE);
+ idev->port_info = dma_alloc_coherent(ionic->dev, idev->port_info_sz,
+ &idev->port_info_pa,
+ GFP_KERNEL);
+ if (!idev->port_info) {
+ dev_err(ionic->dev, "Failed to allocate port info, aborting\n");
+ return -ENOMEM;
+ }
+
+ sz = min(sizeof(ident->port.config), sizeof(idev->dev_cmd_regs->data));
+
+ mutex_lock(&ionic->dev_cmd_lock);
+
+ memcpy_toio(&idev->dev_cmd_regs->data, &ident->port.config, sz);
+ ionic_dev_cmd_port_init(idev);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+
+ ionic_dev_cmd_port_state(&ionic->idev, IONIC_PORT_ADMIN_STATE_UP);
+ (void)ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+
+ mutex_unlock(&ionic->dev_cmd_lock);
+ if (err) {
+ dev_err(ionic->dev, "Failed to init port\n");
+ dma_free_coherent(ionic->dev, idev->port_info_sz,
+ idev->port_info, idev->port_info_pa);
+ idev->port_info = NULL;
+ idev->port_info_pa = 0;
+ }
+
+ return err;
+}
+
+int ionic_port_reset(struct ionic *ionic)
+{
+ struct ionic_dev *idev = &ionic->idev;
+ int err;
+
+ if (!idev->port_info)
+ return 0;
+
+ mutex_lock(&ionic->dev_cmd_lock);
+ ionic_dev_cmd_port_reset(idev);
+ err = ionic_dev_cmd_wait(ionic, DEVCMD_TIMEOUT);
+ mutex_unlock(&ionic->dev_cmd_lock);
+
+ dma_free_coherent(ionic->dev, idev->port_info_sz,
+ idev->port_info, idev->port_info_pa);
+
+ idev->port_info = NULL;
+ idev->port_info_pa = 0;
+
+ if (err)
+ dev_err(ionic->dev, "Failed to reset port\n");
+
+ return err;
+}
+
+static int __init ionic_init_module(void)
+{
+ pr_info("%s %s, ver %s\n",
+ IONIC_DRV_NAME, IONIC_DRV_DESCRIPTION, IONIC_DRV_VERSION);
+ ionic_debugfs_create();
+ return ionic_bus_register_driver();
+}
+
+static void __exit ionic_cleanup_module(void)
+{
+ ionic_bus_unregister_driver();
+ ionic_debugfs_destroy();
+
+ pr_info("%s removed\n", IONIC_DRV_NAME);
+}
+
+module_init(ionic_init_module);
+module_exit(ionic_cleanup_module);
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_regs.h b/drivers/net/ethernet/pensando/ionic/ionic_regs.h
new file mode 100644
index 0000000..03ee5a3
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_regs.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB OR BSD-2-Clause */
+/* Copyright (c) 2018-2019 Pensando Systems, Inc. All rights reserved. */
+
+#ifndef IONIC_REGS_H
+#define IONIC_REGS_H
+
+#include <linux/io.h>
+
+/** struct ionic_intr - interrupt control register set.
+ * @coal_init: coalesce timer initial value.
+ * @mask: interrupt mask value.
+ * @credits: interrupt credit count and return.
+ * @mask_assert: interrupt mask value on assert.
+ * @coal: coalesce timer time remaining.
+ */
+struct ionic_intr {
+ u32 coal_init;
+ u32 mask;
+ u32 credits;
+ u32 mask_assert;
+ u32 coal;
+ u32 rsvd[3];
+};
+
+#define IONIC_INTR_CTRL_REGS_MAX 2048
+#define IONIC_INTR_CTRL_COAL_MAX 0x3F
+
+/** enum ionic_intr_mask_vals - valid values for mask and mask_assert.
+ * @IONIC_INTR_MASK_CLEAR: unmask interrupt.
+ * @IONIC_INTR_MASK_SET: mask interrupt.
+ */
+enum ionic_intr_mask_vals {
+ IONIC_INTR_MASK_CLEAR = 0,
+ IONIC_INTR_MASK_SET = 1,
+};
+
+/** enum ionic_intr_credits_bits - bitwise composition of credits values.
+ * @IONIC_INTR_CRED_COUNT: bit mask of credit count, no shift needed.
+ * @IONIC_INTR_CRED_COUNT_SIGNED: bit mask of credit count, including sign bit.
+ * @IONIC_INTR_CRED_UNMASK: unmask the interrupt.
+ * @IONIC_INTR_CRED_RESET_COALESCE: reset the coalesce timer.
+ * @IONIC_INTR_CRED_REARM: unmask the and reset the timer.
+ */
+enum ionic_intr_credits_bits {
+ IONIC_INTR_CRED_COUNT = 0x7fffu,
+ IONIC_INTR_CRED_COUNT_SIGNED = 0xffffu,
+ IONIC_INTR_CRED_UNMASK = 0x10000u,
+ IONIC_INTR_CRED_RESET_COALESCE = 0x20000u,
+ IONIC_INTR_CRED_REARM = (IONIC_INTR_CRED_UNMASK |
+ IONIC_INTR_CRED_RESET_COALESCE),
+};
+
+static inline void ionic_intr_coal_init(struct ionic_intr __iomem *intr_ctrl,
+ int intr_idx, u32 coal)
+{
+ iowrite32(coal, &intr_ctrl[intr_idx].coal_init);
+}
+
+static inline void ionic_intr_mask(struct ionic_intr __iomem *intr_ctrl,
+ int intr_idx, u32 mask)
+{
+ iowrite32(mask, &intr_ctrl[intr_idx].mask);
+}
+
+static inline void ionic_intr_credits(struct ionic_intr __iomem *intr_ctrl,
+ int intr_idx, u32 cred, u32 flags)
+{
+ if (WARN_ON_ONCE(cred > IONIC_INTR_CRED_COUNT)) {
+ cred = ioread32(&intr_ctrl[intr_idx].credits);
+ cred &= IONIC_INTR_CRED_COUNT_SIGNED;
+ }
+
+ iowrite32(cred | flags, &intr_ctrl[intr_idx].credits);
+}
+
+static inline void ionic_intr_clean(struct ionic_intr __iomem *intr_ctrl,
+ int intr_idx)
+{
+ u32 cred;
+
+ cred = ioread32(&intr_ctrl[intr_idx].credits);
+ cred &= IONIC_INTR_CRED_COUNT_SIGNED;
+ cred |= IONIC_INTR_CRED_RESET_COALESCE;
+ iowrite32(cred, &intr_ctrl[intr_idx].credits);
+}
+
+static inline void ionic_intr_mask_assert(struct ionic_intr __iomem *intr_ctrl,
+ int intr_idx, u32 mask)
+{
+ iowrite32(mask, &intr_ctrl[intr_idx].mask_assert);
+}
+
+/** enum ionic_dbell_bits - bitwise composition of dbell values.
+ *
+ * @IONIC_DBELL_QID_MASK: unshifted mask of valid queue id bits.
+ * @IONIC_DBELL_QID_SHIFT: queue id shift amount in dbell value.
+ * @IONIC_DBELL_QID: macro to build QID component of dbell value.
+ *
+ * @IONIC_DBELL_RING_MASK: unshifted mask of valid ring bits.
+ * @IONIC_DBELL_RING_SHIFT: ring shift amount in dbell value.
+ * @IONIC_DBELL_RING: macro to build ring component of dbell value.
+ *
+ * @IONIC_DBELL_RING_0: ring zero dbell component value.
+ * @IONIC_DBELL_RING_1: ring one dbell component value.
+ * @IONIC_DBELL_RING_2: ring two dbell component value.
+ * @IONIC_DBELL_RING_3: ring three dbell component value.
+ *
+ * @IONIC_DBELL_INDEX_MASK: bit mask of valid index bits, no shift needed.
+ */
+enum ionic_dbell_bits {
+ IONIC_DBELL_QID_MASK = 0xffffff,
+ IONIC_DBELL_QID_SHIFT = 24,
+
+#define IONIC_DBELL_QID(n) \
+ (((u64)(n) & IONIC_DBELL_QID_MASK) << IONIC_DBELL_QID_SHIFT)
+
+ IONIC_DBELL_RING_MASK = 0x7,
+ IONIC_DBELL_RING_SHIFT = 16,
+
+#define IONIC_DBELL_RING(n) \
+ (((u64)(n) & IONIC_DBELL_RING_MASK) << IONIC_DBELL_RING_SHIFT)
+
+ IONIC_DBELL_RING_0 = 0,
+ IONIC_DBELL_RING_1 = IONIC_DBELL_RING(1),
+ IONIC_DBELL_RING_2 = IONIC_DBELL_RING(2),
+ IONIC_DBELL_RING_3 = IONIC_DBELL_RING(3),
+
+ IONIC_DBELL_INDEX_MASK = 0xffff,
+};
+
+static inline void ionic_dbell_ring(u64 __iomem *db_page, int qtype, u64 val)
+{
+ writeq(val, &db_page[qtype]);
+}
+
+#endif /* IONIC_REGS_H */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c b/drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c
new file mode 100644
index 0000000..7a093f1
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+
+#include "ionic.h"
+#include "ionic_lif.h"
+#include "ionic_rx_filter.h"
+
+void ionic_rx_filter_free(struct ionic_lif *lif, struct ionic_rx_filter *f)
+{
+ struct device *dev = lif->ionic->dev;
+
+ hlist_del(&f->by_id);
+ hlist_del(&f->by_hash);
+ devm_kfree(dev, f);
+}
+
+int ionic_rx_filter_del(struct ionic_lif *lif, struct ionic_rx_filter *f)
+{
+ struct ionic_admin_ctx ctx = {
+ .work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+ .cmd.rx_filter_del = {
+ .opcode = IONIC_CMD_RX_FILTER_DEL,
+ .filter_id = cpu_to_le32(f->filter_id),
+ },
+ };
+
+ return ionic_adminq_post_wait(lif, &ctx);
+}
+
+int ionic_rx_filters_init(struct ionic_lif *lif)
+{
+ unsigned int i;
+
+ spin_lock_init(&lif->rx_filters.lock);
+
+ for (i = 0; i < IONIC_RX_FILTER_HLISTS; i++) {
+ INIT_HLIST_HEAD(&lif->rx_filters.by_hash[i]);
+ INIT_HLIST_HEAD(&lif->rx_filters.by_id[i]);
+ }
+
+ return 0;
+}
+
+void ionic_rx_filters_deinit(struct ionic_lif *lif)
+{
+ struct ionic_rx_filter *f;
+ struct hlist_head *head;
+ struct hlist_node *tmp;
+ unsigned int i;
+
+ for (i = 0; i < IONIC_RX_FILTER_HLISTS; i++) {
+ head = &lif->rx_filters.by_id[i];
+ hlist_for_each_entry_safe(f, tmp, head, by_id)
+ ionic_rx_filter_free(lif, f);
+ }
+}
+
+int ionic_rx_filter_save(struct ionic_lif *lif, u32 flow_id, u16 rxq_index,
+ u32 hash, struct ionic_admin_ctx *ctx)
+{
+ struct device *dev = lif->ionic->dev;
+ struct ionic_rx_filter_add_cmd *ac;
+ struct ionic_rx_filter *f;
+ struct hlist_head *head;
+ unsigned int key;
+
+ ac = &ctx->cmd.rx_filter_add;
+
+ switch (le16_to_cpu(ac->match)) {
+ case IONIC_RX_FILTER_MATCH_VLAN:
+ key = le16_to_cpu(ac->vlan.vlan);
+ break;
+ case IONIC_RX_FILTER_MATCH_MAC:
+ key = *(u32 *)ac->mac.addr;
+ break;
+ case IONIC_RX_FILTER_MATCH_MAC_VLAN:
+ key = le16_to_cpu(ac->mac_vlan.vlan);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ f = devm_kzalloc(dev, sizeof(*f), GFP_KERNEL);
+ if (!f)
+ return -ENOMEM;
+
+ f->flow_id = flow_id;
+ f->filter_id = le32_to_cpu(ctx->comp.rx_filter_add.filter_id);
+ f->rxq_index = rxq_index;
+ memcpy(&f->cmd, ac, sizeof(f->cmd));
+
+ INIT_HLIST_NODE(&f->by_hash);
+ INIT_HLIST_NODE(&f->by_id);
+
+ spin_lock_bh(&lif->rx_filters.lock);
+
+ key = hash_32(key, IONIC_RX_FILTER_HASH_BITS);
+ head = &lif->rx_filters.by_hash[key];
+ hlist_add_head(&f->by_hash, head);
+
+ key = f->filter_id & IONIC_RX_FILTER_HLISTS_MASK;
+ head = &lif->rx_filters.by_id[key];
+ hlist_add_head(&f->by_id, head);
+
+ spin_unlock_bh(&lif->rx_filters.lock);
+
+ return 0;
+}
+
+struct ionic_rx_filter *ionic_rx_filter_by_vlan(struct ionic_lif *lif, u16 vid)
+{
+ struct ionic_rx_filter *f;
+ struct hlist_head *head;
+ unsigned int key;
+
+ key = hash_32(vid, IONIC_RX_FILTER_HASH_BITS);
+ head = &lif->rx_filters.by_hash[key];
+
+ hlist_for_each_entry(f, head, by_hash) {
+ if (le16_to_cpu(f->cmd.match) != IONIC_RX_FILTER_MATCH_VLAN)
+ continue;
+ if (le16_to_cpu(f->cmd.vlan.vlan) == vid)
+ return f;
+ }
+
+ return NULL;
+}
+
+struct ionic_rx_filter *ionic_rx_filter_by_addr(struct ionic_lif *lif,
+ const u8 *addr)
+{
+ struct ionic_rx_filter *f;
+ struct hlist_head *head;
+ unsigned int key;
+
+ key = hash_32(*(u32 *)addr, IONIC_RX_FILTER_HASH_BITS);
+ head = &lif->rx_filters.by_hash[key];
+
+ hlist_for_each_entry(f, head, by_hash) {
+ if (le16_to_cpu(f->cmd.match) != IONIC_RX_FILTER_MATCH_MAC)
+ continue;
+ if (memcmp(addr, f->cmd.mac.addr, ETH_ALEN) == 0)
+ return f;
+ }
+
+ return NULL;
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h b/drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h
new file mode 100644
index 0000000..b6aec9c
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_RX_FILTER_H_
+#define _IONIC_RX_FILTER_H_
+
+#define IONIC_RXQ_INDEX_ANY (0xFFFF)
+struct ionic_rx_filter {
+ u32 flow_id;
+ u32 filter_id;
+ u16 rxq_index;
+ struct ionic_rx_filter_add_cmd cmd;
+ struct hlist_node by_hash;
+ struct hlist_node by_id;
+};
+
+#define IONIC_RX_FILTER_HASH_BITS 10
+#define IONIC_RX_FILTER_HLISTS BIT(IONIC_RX_FILTER_HASH_BITS)
+#define IONIC_RX_FILTER_HLISTS_MASK (IONIC_RX_FILTER_HLISTS - 1)
+struct ionic_rx_filters {
+ spinlock_t lock; /* filter list lock */
+ struct hlist_head by_hash[IONIC_RX_FILTER_HLISTS]; /* by skb hash */
+ struct hlist_head by_id[IONIC_RX_FILTER_HLISTS]; /* by filter_id */
+};
+
+void ionic_rx_filter_free(struct ionic_lif *lif, struct ionic_rx_filter *f);
+int ionic_rx_filter_del(struct ionic_lif *lif, struct ionic_rx_filter *f);
+int ionic_rx_filters_init(struct ionic_lif *lif);
+void ionic_rx_filters_deinit(struct ionic_lif *lif);
+int ionic_rx_filter_save(struct ionic_lif *lif, u32 flow_id, u16 rxq_index,
+ u32 hash, struct ionic_admin_ctx *ctx);
+struct ionic_rx_filter *ionic_rx_filter_by_vlan(struct ionic_lif *lif, u16 vid);
+struct ionic_rx_filter *ionic_rx_filter_by_addr(struct ionic_lif *lif, const u8 *addr);
+
+#endif /* _IONIC_RX_FILTER_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c
new file mode 100644
index 0000000..e290788
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c
@@ -0,0 +1,310 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/kernel.h>
+#include <linux/mutex.h>
+#include <linux/netdevice.h>
+
+#include "ionic.h"
+#include "ionic_lif.h"
+#include "ionic_stats.h"
+
+static const struct ionic_stat_desc ionic_lif_stats_desc[] = {
+ IONIC_LIF_STAT_DESC(tx_packets),
+ IONIC_LIF_STAT_DESC(tx_bytes),
+ IONIC_LIF_STAT_DESC(rx_packets),
+ IONIC_LIF_STAT_DESC(rx_bytes),
+ IONIC_LIF_STAT_DESC(tx_tso),
+ IONIC_LIF_STAT_DESC(tx_no_csum),
+ IONIC_LIF_STAT_DESC(tx_csum),
+ IONIC_LIF_STAT_DESC(rx_csum_none),
+ IONIC_LIF_STAT_DESC(rx_csum_complete),
+ IONIC_LIF_STAT_DESC(rx_csum_error),
+};
+
+static const struct ionic_stat_desc ionic_tx_stats_desc[] = {
+ IONIC_TX_STAT_DESC(pkts),
+ IONIC_TX_STAT_DESC(bytes),
+ IONIC_TX_STAT_DESC(clean),
+ IONIC_TX_STAT_DESC(dma_map_err),
+ IONIC_TX_STAT_DESC(linearize),
+ IONIC_TX_STAT_DESC(frags),
+};
+
+static const struct ionic_stat_desc ionic_rx_stats_desc[] = {
+ IONIC_RX_STAT_DESC(pkts),
+ IONIC_RX_STAT_DESC(bytes),
+ IONIC_RX_STAT_DESC(dma_map_err),
+ IONIC_RX_STAT_DESC(alloc_err),
+ IONIC_RX_STAT_DESC(csum_none),
+ IONIC_RX_STAT_DESC(csum_complete),
+ IONIC_RX_STAT_DESC(csum_error),
+};
+
+static const struct ionic_stat_desc ionic_txq_stats_desc[] = {
+ IONIC_TX_Q_STAT_DESC(stop),
+ IONIC_TX_Q_STAT_DESC(wake),
+ IONIC_TX_Q_STAT_DESC(drop),
+ IONIC_TX_Q_STAT_DESC(dbell_count),
+};
+
+static const struct ionic_stat_desc ionic_dbg_cq_stats_desc[] = {
+ IONIC_CQ_STAT_DESC(compl_count),
+};
+
+static const struct ionic_stat_desc ionic_dbg_intr_stats_desc[] = {
+ IONIC_INTR_STAT_DESC(rearm_count),
+};
+
+static const struct ionic_stat_desc ionic_dbg_napi_stats_desc[] = {
+ IONIC_NAPI_STAT_DESC(poll_count),
+};
+
+#define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc)
+#define IONIC_NUM_TX_STATS ARRAY_SIZE(ionic_tx_stats_desc)
+#define IONIC_NUM_RX_STATS ARRAY_SIZE(ionic_rx_stats_desc)
+#define IONIC_NUM_TX_Q_STATS ARRAY_SIZE(ionic_txq_stats_desc)
+#define IONIC_NUM_DBG_CQ_STATS ARRAY_SIZE(ionic_dbg_cq_stats_desc)
+#define IONIC_NUM_DBG_INTR_STATS ARRAY_SIZE(ionic_dbg_intr_stats_desc)
+#define IONIC_NUM_DBG_NAPI_STATS ARRAY_SIZE(ionic_dbg_napi_stats_desc)
+
+#define MAX_Q(lif) ((lif)->netdev->real_num_tx_queues)
+
+static void ionic_get_lif_stats(struct ionic_lif *lif,
+ struct ionic_lif_sw_stats *stats)
+{
+ struct ionic_tx_stats *tstats;
+ struct ionic_rx_stats *rstats;
+ struct ionic_qcq *txqcq;
+ struct ionic_qcq *rxqcq;
+ int q_num;
+
+ memset(stats, 0, sizeof(*stats));
+
+ for (q_num = 0; q_num < MAX_Q(lif); q_num++) {
+ txqcq = lif_to_txqcq(lif, q_num);
+ if (txqcq && txqcq->stats) {
+ tstats = &txqcq->stats->tx;
+ stats->tx_packets += tstats->pkts;
+ stats->tx_bytes += tstats->bytes;
+ stats->tx_tso += tstats->tso;
+ stats->tx_no_csum += tstats->no_csum;
+ stats->tx_csum += tstats->csum;
+ }
+
+ rxqcq = lif_to_rxqcq(lif, q_num);
+ if (rxqcq && rxqcq->stats) {
+ rstats = &rxqcq->stats->rx;
+ stats->rx_packets += rstats->pkts;
+ stats->rx_bytes += rstats->bytes;
+ stats->rx_csum_none += rstats->csum_none;
+ stats->rx_csum_complete += rstats->csum_complete;
+ stats->rx_csum_error += rstats->csum_error;
+ }
+ }
+}
+
+static u64 ionic_sw_stats_get_count(struct ionic_lif *lif)
+{
+ u64 total = 0;
+
+ /* lif stats */
+ total += IONIC_NUM_LIF_STATS;
+
+ /* tx stats */
+ total += MAX_Q(lif) * IONIC_NUM_TX_STATS;
+
+ /* rx stats */
+ total += MAX_Q(lif) * IONIC_NUM_RX_STATS;
+
+ if (test_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state)) {
+ /* tx debug stats */
+ total += MAX_Q(lif) * (IONIC_NUM_DBG_CQ_STATS +
+ IONIC_NUM_TX_Q_STATS +
+ IONIC_NUM_DBG_INTR_STATS +
+ IONIC_MAX_NUM_SG_CNTR);
+
+ /* rx debug stats */
+ total += MAX_Q(lif) * (IONIC_NUM_DBG_CQ_STATS +
+ IONIC_NUM_DBG_INTR_STATS +
+ IONIC_NUM_DBG_NAPI_STATS +
+ IONIC_MAX_NUM_NAPI_CNTR);
+ }
+
+ return total;
+}
+
+static void ionic_sw_stats_get_strings(struct ionic_lif *lif, u8 **buf)
+{
+ int i, q_num;
+
+ for (i = 0; i < IONIC_NUM_LIF_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN, ionic_lif_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (q_num = 0; q_num < MAX_Q(lif); q_num++) {
+ for (i = 0; i < IONIC_NUM_TX_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN, "tx_%d_%s",
+ q_num, ionic_tx_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+
+ if (test_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state)) {
+ for (i = 0; i < IONIC_NUM_TX_Q_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "txq_%d_%s",
+ q_num,
+ ionic_txq_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_CQ_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "txq_%d_cq_%s",
+ q_num,
+ ionic_dbg_cq_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_INTR_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "txq_%d_intr_%s",
+ q_num,
+ ionic_dbg_intr_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (i = 0; i < IONIC_MAX_NUM_SG_CNTR; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "txq_%d_sg_cntr_%d",
+ q_num, i);
+ *buf += ETH_GSTRING_LEN;
+ }
+ }
+ }
+ for (q_num = 0; q_num < MAX_Q(lif); q_num++) {
+ for (i = 0; i < IONIC_NUM_RX_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "rx_%d_%s",
+ q_num, ionic_rx_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+
+ if (test_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state)) {
+ for (i = 0; i < IONIC_NUM_DBG_CQ_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "rxq_%d_cq_%s",
+ q_num,
+ ionic_dbg_cq_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_INTR_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "rxq_%d_intr_%s",
+ q_num,
+ ionic_dbg_intr_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_NAPI_STATS; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "rxq_%d_napi_%s",
+ q_num,
+ ionic_dbg_napi_stats_desc[i].name);
+ *buf += ETH_GSTRING_LEN;
+ }
+ for (i = 0; i < IONIC_MAX_NUM_NAPI_CNTR; i++) {
+ snprintf(*buf, ETH_GSTRING_LEN,
+ "rxq_%d_napi_work_done_%d",
+ q_num, i);
+ *buf += ETH_GSTRING_LEN;
+ }
+ }
+ }
+}
+
+static void ionic_sw_stats_get_values(struct ionic_lif *lif, u64 **buf)
+{
+ struct ionic_lif_sw_stats lif_stats;
+ struct ionic_qcq *txqcq, *rxqcq;
+ int i, q_num;
+
+ ionic_get_lif_stats(lif, &lif_stats);
+
+ for (i = 0; i < IONIC_NUM_LIF_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&lif_stats, &ionic_lif_stats_desc[i]);
+ (*buf)++;
+ }
+
+ for (q_num = 0; q_num < MAX_Q(lif); q_num++) {
+ txqcq = lif_to_txqcq(lif, q_num);
+
+ for (i = 0; i < IONIC_NUM_TX_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&txqcq->stats->tx,
+ &ionic_tx_stats_desc[i]);
+ (*buf)++;
+ }
+
+ if (test_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state)) {
+ for (i = 0; i < IONIC_NUM_TX_Q_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&txqcq->q,
+ &ionic_txq_stats_desc[i]);
+ (*buf)++;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_CQ_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&txqcq->cq,
+ &ionic_dbg_cq_stats_desc[i]);
+ (*buf)++;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_INTR_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&txqcq->intr,
+ &ionic_dbg_intr_stats_desc[i]);
+ (*buf)++;
+ }
+ for (i = 0; i < IONIC_MAX_NUM_SG_CNTR; i++) {
+ **buf = txqcq->stats->tx.sg_cntr[i];
+ (*buf)++;
+ }
+ }
+ }
+
+ for (q_num = 0; q_num < MAX_Q(lif); q_num++) {
+ rxqcq = lif_to_rxqcq(lif, q_num);
+
+ for (i = 0; i < IONIC_NUM_RX_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&rxqcq->stats->rx,
+ &ionic_rx_stats_desc[i]);
+ (*buf)++;
+ }
+
+ if (test_bit(IONIC_LIF_SW_DEBUG_STATS, lif->state)) {
+ for (i = 0; i < IONIC_NUM_DBG_CQ_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&rxqcq->cq,
+ &ionic_dbg_cq_stats_desc[i]);
+ (*buf)++;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_INTR_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&rxqcq->intr,
+ &ionic_dbg_intr_stats_desc[i]);
+ (*buf)++;
+ }
+ for (i = 0; i < IONIC_NUM_DBG_NAPI_STATS; i++) {
+ **buf = IONIC_READ_STAT64(&rxqcq->napi_stats,
+ &ionic_dbg_napi_stats_desc[i]);
+ (*buf)++;
+ }
+ for (i = 0; i < IONIC_MAX_NUM_NAPI_CNTR; i++) {
+ **buf = rxqcq->napi_stats.work_done_cntr[i];
+ (*buf)++;
+ }
+ }
+ }
+}
+
+const struct ionic_stats_group_intf ionic_stats_groups[] = {
+ /* SW Stats group */
+ {
+ .get_strings = ionic_sw_stats_get_strings,
+ .get_values = ionic_sw_stats_get_values,
+ .get_count = ionic_sw_stats_get_count,
+ },
+ /* Add more stat groups here */
+};
+
+const int ionic_num_stats_grps = ARRAY_SIZE(ionic_stats_groups);
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.h b/drivers/net/ethernet/pensando/ionic/ionic_stats.h
new file mode 100644
index 0000000..d2c1122
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_STATS_H_
+#define _IONIC_STATS_H_
+
+#define IONIC_STAT_TO_OFFSET(type, stat_name) (offsetof(type, stat_name))
+
+#define IONIC_STAT_DESC(type, stat_name) { \
+ .name = #stat_name, \
+ .offset = IONIC_STAT_TO_OFFSET(type, stat_name) \
+}
+
+#define IONIC_LIF_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_lif_sw_stats, stat_name)
+
+#define IONIC_TX_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_tx_stats, stat_name)
+
+#define IONIC_RX_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_rx_stats, stat_name)
+
+#define IONIC_TX_Q_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_queue, stat_name)
+
+#define IONIC_CQ_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_cq, stat_name)
+
+#define IONIC_INTR_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_intr_info, stat_name)
+
+#define IONIC_NAPI_STAT_DESC(stat_name) \
+ IONIC_STAT_DESC(struct ionic_napi_stats, stat_name)
+
+/* Interface structure for a particalar stats group */
+struct ionic_stats_group_intf {
+ void (*get_strings)(struct ionic_lif *lif, u8 **buf);
+ void (*get_values)(struct ionic_lif *lif, u64 **buf);
+ u64 (*get_count)(struct ionic_lif *lif);
+};
+
+extern const struct ionic_stats_group_intf ionic_stats_groups[];
+extern const int ionic_num_stats_grps;
+
+#define IONIC_READ_STAT64(base_ptr, desc_ptr) \
+ (*((u64 *)(((u8 *)(base_ptr)) + (desc_ptr)->offset)))
+
+struct ionic_stat_desc {
+ char name[ETH_GSTRING_LEN];
+ u64 offset;
+};
+
+#endif /* _IONIC_STATS_H_ */
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
new file mode 100644
index 0000000..ab6663d
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
@@ -0,0 +1,925 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <linux/if_vlan.h>
+#include <net/ip6_checksum.h>
+
+#include "ionic.h"
+#include "ionic_lif.h"
+#include "ionic_txrx.h"
+
+static void ionic_rx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+ struct ionic_cq_info *cq_info, void *cb_arg);
+
+static inline void ionic_txq_post(struct ionic_queue *q, bool ring_dbell,
+ ionic_desc_cb cb_func, void *cb_arg)
+{
+ DEBUG_STATS_TXQ_POST(q_to_qcq(q), q->head->desc, ring_dbell);
+
+ ionic_q_post(q, ring_dbell, cb_func, cb_arg);
+}
+
+static inline void ionic_rxq_post(struct ionic_queue *q, bool ring_dbell,
+ ionic_desc_cb cb_func, void *cb_arg)
+{
+ ionic_q_post(q, ring_dbell, cb_func, cb_arg);
+
+ DEBUG_STATS_RX_BUFF_CNT(q_to_qcq(q));
+}
+
+static inline struct netdev_queue *q_to_ndq(struct ionic_queue *q)
+{
+ return netdev_get_tx_queue(q->lif->netdev, q->index);
+}
+
+static void ionic_rx_recycle(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+ struct sk_buff *skb)
+{
+ struct ionic_rxq_desc *old = desc_info->desc;
+ struct ionic_rxq_desc *new = q->head->desc;
+
+ new->addr = old->addr;
+ new->len = old->len;
+
+ ionic_rxq_post(q, true, ionic_rx_clean, skb);
+}
+
+static bool ionic_rx_copybreak(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+ struct ionic_cq_info *cq_info, struct sk_buff **skb)
+{
+ struct ionic_rxq_comp *comp = cq_info->cq_desc;
+ struct ionic_rxq_desc *desc = desc_info->desc;
+ struct net_device *netdev = q->lif->netdev;
+ struct device *dev = q->lif->ionic->dev;
+ struct sk_buff *new_skb;
+ u16 clen, dlen;
+
+ clen = le16_to_cpu(comp->len);
+ dlen = le16_to_cpu(desc->len);
+ if (clen > q->lif->rx_copybreak) {
+ dma_unmap_single(dev, (dma_addr_t)le64_to_cpu(desc->addr),
+ dlen, DMA_FROM_DEVICE);
+ return false;
+ }
+
+ new_skb = netdev_alloc_skb_ip_align(netdev, clen);
+ if (!new_skb) {
+ dma_unmap_single(dev, (dma_addr_t)le64_to_cpu(desc->addr),
+ dlen, DMA_FROM_DEVICE);
+ return false;
+ }
+
+ dma_sync_single_for_cpu(dev, (dma_addr_t)le64_to_cpu(desc->addr),
+ clen, DMA_FROM_DEVICE);
+
+ memcpy(new_skb->data, (*skb)->data, clen);
+
+ ionic_rx_recycle(q, desc_info, *skb);
+ *skb = new_skb;
+
+ return true;
+}
+
+static void ionic_rx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+ struct ionic_cq_info *cq_info, void *cb_arg)
+{
+ struct ionic_rxq_comp *comp = cq_info->cq_desc;
+ struct ionic_qcq *qcq = q_to_qcq(q);
+ struct sk_buff *skb = cb_arg;
+ struct ionic_rx_stats *stats;
+ struct net_device *netdev;
+
+ stats = q_to_rx_stats(q);
+ netdev = q->lif->netdev;
+
+ if (comp->status) {
+ ionic_rx_recycle(q, desc_info, skb);
+ return;
+ }
+
+ if (unlikely(test_bit(IONIC_LIF_QUEUE_RESET, q->lif->state))) {
+ /* no packet processing while resetting */
+ ionic_rx_recycle(q, desc_info, skb);
+ return;
+ }
+
+ stats->pkts++;
+ stats->bytes += le16_to_cpu(comp->len);
+
+ ionic_rx_copybreak(q, desc_info, cq_info, &skb);
+
+ skb_put(skb, le16_to_cpu(comp->len));
+ skb->protocol = eth_type_trans(skb, netdev);
+
+ skb_record_rx_queue(skb, q->index);
+
+ if (netdev->features & NETIF_F_RXHASH) {
+ switch (comp->pkt_type_color & IONIC_RXQ_COMP_PKT_TYPE_MASK) {
+ case IONIC_PKT_TYPE_IPV4:
+ case IONIC_PKT_TYPE_IPV6:
+ skb_set_hash(skb, le32_to_cpu(comp->rss_hash),
+ PKT_HASH_TYPE_L3);
+ break;
+ case IONIC_PKT_TYPE_IPV4_TCP:
+ case IONIC_PKT_TYPE_IPV6_TCP:
+ case IONIC_PKT_TYPE_IPV4_UDP:
+ case IONIC_PKT_TYPE_IPV6_UDP:
+ skb_set_hash(skb, le32_to_cpu(comp->rss_hash),
+ PKT_HASH_TYPE_L4);
+ break;
+ }
+ }
+
+ if (netdev->features & NETIF_F_RXCSUM) {
+ if (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_CALC) {
+ skb->ip_summed = CHECKSUM_COMPLETE;
+ skb->csum = (__wsum)le16_to_cpu(comp->csum);
+ stats->csum_complete++;
+ }
+ } else {
+ stats->csum_none++;
+ }
+
+ if ((comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_TCP_BAD) ||
+ (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_UDP_BAD) ||
+ (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_IP_BAD))
+ stats->csum_error++;
+
+ if (netdev->features & NETIF_F_HW_VLAN_CTAG_RX) {
+ if (comp->csum_flags & IONIC_RXQ_COMP_CSUM_F_VLAN)
+ __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
+ le16_to_cpu(comp->vlan_tci));
+ }
+
+ napi_gro_receive(&qcq->napi, skb);
+}
+
+static bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info)
+{
+ struct ionic_rxq_comp *comp = cq_info->cq_desc;
+ struct ionic_queue *q = cq->bound_q;
+ struct ionic_desc_info *desc_info;
+
+ if (!color_match(comp->pkt_type_color, cq->done_color))
+ return false;
+
+ /* check for empty queue */
+ if (q->tail->index == q->head->index)
+ return false;
+
+ desc_info = q->tail;
+ if (desc_info->index != le16_to_cpu(comp->comp_index))
+ return false;
+
+ q->tail = desc_info->next;
+
+ /* clean the related q entry, only one per qc completion */
+ ionic_rx_clean(q, desc_info, cq_info, desc_info->cb_arg);
+
+ desc_info->cb = NULL;
+ desc_info->cb_arg = NULL;
+
+ return true;
+}
+
+static u32 ionic_rx_walk_cq(struct ionic_cq *rxcq, u32 limit)
+{
+ u32 work_done = 0;
+
+ while (ionic_rx_service(rxcq, rxcq->tail)) {
+ if (rxcq->tail->last)
+ rxcq->done_color = !rxcq->done_color;
+ rxcq->tail = rxcq->tail->next;
+ DEBUG_STATS_CQE_CNT(rxcq);
+
+ if (++work_done >= limit)
+ break;
+ }
+
+ return work_done;
+}
+
+void ionic_rx_flush(struct ionic_cq *cq)
+{
+ struct ionic_dev *idev = &cq->lif->ionic->idev;
+ u32 work_done;
+
+ work_done = ionic_rx_walk_cq(cq, cq->num_descs);
+
+ if (work_done)
+ ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index,
+ work_done, IONIC_INTR_CRED_RESET_COALESCE);
+}
+
+static struct sk_buff *ionic_rx_skb_alloc(struct ionic_queue *q, unsigned int len,
+ dma_addr_t *dma_addr)
+{
+ struct ionic_lif *lif = q->lif;
+ struct ionic_rx_stats *stats;
+ struct net_device *netdev;
+ struct sk_buff *skb;
+ struct device *dev;
+
+ netdev = lif->netdev;
+ dev = lif->ionic->dev;
+ stats = q_to_rx_stats(q);
+ skb = netdev_alloc_skb_ip_align(netdev, len);
+ if (!skb) {
+ net_warn_ratelimited("%s: SKB alloc failed on %s!\n",
+ netdev->name, q->name);
+ stats->alloc_err++;
+ return NULL;
+ }
+
+ *dma_addr = dma_map_single(dev, skb->data, len, DMA_FROM_DEVICE);
+ if (dma_mapping_error(dev, *dma_addr)) {
+ dev_kfree_skb(skb);
+ net_warn_ratelimited("%s: DMA single map failed on %s!\n",
+ netdev->name, q->name);
+ stats->dma_map_err++;
+ return NULL;
+ }
+
+ return skb;
+}
+
+#define IONIC_RX_RING_DOORBELL_STRIDE ((1 << 2) - 1)
+
+void ionic_rx_fill(struct ionic_queue *q)
+{
+ struct net_device *netdev = q->lif->netdev;
+ struct ionic_rxq_desc *desc;
+ struct sk_buff *skb;
+ dma_addr_t dma_addr;
+ bool ring_doorbell;
+ unsigned int len;
+ unsigned int i;
+
+ len = netdev->mtu + ETH_HLEN;
+
+ for (i = ionic_q_space_avail(q); i; i--) {
+ skb = ionic_rx_skb_alloc(q, len, &dma_addr);
+ if (!skb)
+ return;
+
+ desc = q->head->desc;
+ desc->addr = cpu_to_le64(dma_addr);
+ desc->len = cpu_to_le16(len);
+ desc->opcode = IONIC_RXQ_DESC_OPCODE_SIMPLE;
+
+ ring_doorbell = ((q->head->index + 1) &
+ IONIC_RX_RING_DOORBELL_STRIDE) == 0;
+
+ ionic_rxq_post(q, ring_doorbell, ionic_rx_clean, skb);
+ }
+}
+
+static void ionic_rx_fill_cb(void *arg)
+{
+ ionic_rx_fill(arg);
+}
+
+void ionic_rx_empty(struct ionic_queue *q)
+{
+ struct device *dev = q->lif->ionic->dev;
+ struct ionic_desc_info *cur;
+ struct ionic_rxq_desc *desc;
+
+ for (cur = q->tail; cur != q->head; cur = cur->next) {
+ desc = cur->desc;
+ dma_unmap_single(dev, le64_to_cpu(desc->addr),
+ le16_to_cpu(desc->len), DMA_FROM_DEVICE);
+ dev_kfree_skb(cur->cb_arg);
+ cur->cb_arg = NULL;
+ }
+}
+
+int ionic_rx_napi(struct napi_struct *napi, int budget)
+{
+ struct ionic_qcq *qcq = napi_to_qcq(napi);
+ struct ionic_cq *rxcq = napi_to_cq(napi);
+ unsigned int qi = rxcq->bound_q->index;
+ struct ionic_dev *idev;
+ struct ionic_lif *lif;
+ struct ionic_cq *txcq;
+ u32 work_done = 0;
+ u32 flags = 0;
+
+ lif = rxcq->bound_q->lif;
+ idev = &lif->ionic->idev;
+ txcq = &lif->txqcqs[qi].qcq->cq;
+
+ ionic_tx_flush(txcq);
+
+ work_done = ionic_rx_walk_cq(rxcq, budget);
+
+ if (work_done)
+ ionic_rx_fill_cb(rxcq->bound_q);
+
+ if (work_done < budget && napi_complete_done(napi, work_done)) {
+ flags |= IONIC_INTR_CRED_UNMASK;
+ DEBUG_STATS_INTR_REARM(rxcq->bound_intr);
+ }
+
+ if (work_done || flags) {
+ flags |= IONIC_INTR_CRED_RESET_COALESCE;
+ ionic_intr_credits(idev->intr_ctrl, rxcq->bound_intr->index,
+ work_done, flags);
+ }
+
+ DEBUG_STATS_NAPI_POLL(qcq, work_done);
+
+ return work_done;
+}
+
+static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, void *data, size_t len)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct device *dev = q->lif->ionic->dev;
+ dma_addr_t dma_addr;
+
+ dma_addr = dma_map_single(dev, data, len, DMA_TO_DEVICE);
+ if (dma_mapping_error(dev, dma_addr)) {
+ net_warn_ratelimited("%s: DMA single map failed on %s!\n",
+ q->lif->netdev->name, q->name);
+ stats->dma_map_err++;
+ return 0;
+ }
+ return dma_addr;
+}
+
+static dma_addr_t ionic_tx_map_frag(struct ionic_queue *q, const skb_frag_t *frag,
+ size_t offset, size_t len)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct device *dev = q->lif->ionic->dev;
+ dma_addr_t dma_addr;
+
+ dma_addr = skb_frag_dma_map(dev, frag, offset, len, DMA_TO_DEVICE);
+ if (dma_mapping_error(dev, dma_addr)) {
+ net_warn_ratelimited("%s: DMA frag map failed on %s!\n",
+ q->lif->netdev->name, q->name);
+ stats->dma_map_err++;
+ }
+ return dma_addr;
+}
+
+static void ionic_tx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info,
+ struct ionic_cq_info *cq_info, void *cb_arg)
+{
+ struct ionic_txq_sg_desc *sg_desc = desc_info->sg_desc;
+ struct ionic_txq_sg_elem *elem = sg_desc->elems;
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct ionic_txq_desc *desc = desc_info->desc;
+ struct device *dev = q->lif->ionic->dev;
+ u8 opcode, flags, nsge;
+ u16 queue_index;
+ unsigned int i;
+ u64 addr;
+
+ decode_txq_desc_cmd(le64_to_cpu(desc->cmd),
+ &opcode, &flags, &nsge, &addr);
+
+ /* use unmap_single only if either this is not TSO,
+ * or this is first descriptor of a TSO
+ */
+ if (opcode != IONIC_TXQ_DESC_OPCODE_TSO ||
+ flags & IONIC_TXQ_DESC_FLAG_TSO_SOT)
+ dma_unmap_single(dev, (dma_addr_t)addr,
+ le16_to_cpu(desc->len), DMA_TO_DEVICE);
+ else
+ dma_unmap_page(dev, (dma_addr_t)addr,
+ le16_to_cpu(desc->len), DMA_TO_DEVICE);
+
+ for (i = 0; i < nsge; i++, elem++)
+ dma_unmap_page(dev, (dma_addr_t)le64_to_cpu(elem->addr),
+ le16_to_cpu(elem->len), DMA_TO_DEVICE);
+
+ if (cb_arg) {
+ struct sk_buff *skb = cb_arg;
+ u32 len = skb->len;
+
+ queue_index = skb_get_queue_mapping(skb);
+ if (unlikely(__netif_subqueue_stopped(q->lif->netdev,
+ queue_index))) {
+ netif_wake_subqueue(q->lif->netdev, queue_index);
+ q->wake++;
+ }
+ dev_kfree_skb_any(skb);
+ stats->clean++;
+ netdev_tx_completed_queue(q_to_ndq(q), 1, len);
+ }
+}
+
+void ionic_tx_flush(struct ionic_cq *cq)
+{
+ struct ionic_txq_comp *comp = cq->tail->cq_desc;
+ struct ionic_dev *idev = &cq->lif->ionic->idev;
+ struct ionic_queue *q = cq->bound_q;
+ struct ionic_desc_info *desc_info;
+ unsigned int work_done = 0;
+
+ /* walk the completed cq entries */
+ while (work_done < cq->num_descs &&
+ color_match(comp->color, cq->done_color)) {
+
+ /* clean the related q entries, there could be
+ * several q entries completed for each cq completion
+ */
+ do {
+ desc_info = q->tail;
+ q->tail = desc_info->next;
+ ionic_tx_clean(q, desc_info, cq->tail,
+ desc_info->cb_arg);
+ desc_info->cb = NULL;
+ desc_info->cb_arg = NULL;
+ } while (desc_info->index != le16_to_cpu(comp->comp_index));
+
+ if (cq->tail->last)
+ cq->done_color = !cq->done_color;
+
+ cq->tail = cq->tail->next;
+ comp = cq->tail->cq_desc;
+ DEBUG_STATS_CQE_CNT(cq);
+
+ work_done++;
+ }
+
+ if (work_done)
+ ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index,
+ work_done, 0);
+}
+
+static int ionic_tx_tcp_inner_pseudo_csum(struct sk_buff *skb)
+{
+ int err;
+
+ err = skb_cow_head(skb, 0);
+ if (err)
+ return err;
+
+ if (skb->protocol == cpu_to_be16(ETH_P_IP)) {
+ inner_ip_hdr(skb)->check = 0;
+ inner_tcp_hdr(skb)->check =
+ ~csum_tcpudp_magic(inner_ip_hdr(skb)->saddr,
+ inner_ip_hdr(skb)->daddr,
+ 0, IPPROTO_TCP, 0);
+ } else if (skb->protocol == cpu_to_be16(ETH_P_IPV6)) {
+ inner_tcp_hdr(skb)->check =
+ ~csum_ipv6_magic(&inner_ipv6_hdr(skb)->saddr,
+ &inner_ipv6_hdr(skb)->daddr,
+ 0, IPPROTO_TCP, 0);
+ }
+
+ return 0;
+}
+
+static int ionic_tx_tcp_pseudo_csum(struct sk_buff *skb)
+{
+ int err;
+
+ err = skb_cow_head(skb, 0);
+ if (err)
+ return err;
+
+ if (skb->protocol == cpu_to_be16(ETH_P_IP)) {
+ ip_hdr(skb)->check = 0;
+ tcp_hdr(skb)->check =
+ ~csum_tcpudp_magic(ip_hdr(skb)->saddr,
+ ip_hdr(skb)->daddr,
+ 0, IPPROTO_TCP, 0);
+ } else if (skb->protocol == cpu_to_be16(ETH_P_IPV6)) {
+ tcp_hdr(skb)->check =
+ ~csum_ipv6_magic(&ipv6_hdr(skb)->saddr,
+ &ipv6_hdr(skb)->daddr,
+ 0, IPPROTO_TCP, 0);
+ }
+
+ return 0;
+}
+
+static void ionic_tx_tso_post(struct ionic_queue *q, struct ionic_txq_desc *desc,
+ struct sk_buff *skb,
+ dma_addr_t addr, u8 nsge, u16 len,
+ unsigned int hdrlen, unsigned int mss,
+ bool outer_csum,
+ u16 vlan_tci, bool has_vlan,
+ bool start, bool done)
+{
+ u8 flags = 0;
+ u64 cmd;
+
+ flags |= has_vlan ? IONIC_TXQ_DESC_FLAG_VLAN : 0;
+ flags |= outer_csum ? IONIC_TXQ_DESC_FLAG_ENCAP : 0;
+ flags |= start ? IONIC_TXQ_DESC_FLAG_TSO_SOT : 0;
+ flags |= done ? IONIC_TXQ_DESC_FLAG_TSO_EOT : 0;
+
+ cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_TSO, flags, nsge, addr);
+ desc->cmd = cpu_to_le64(cmd);
+ desc->len = cpu_to_le16(len);
+ desc->vlan_tci = cpu_to_le16(vlan_tci);
+ desc->hdr_len = cpu_to_le16(hdrlen);
+ desc->mss = cpu_to_le16(mss);
+
+ if (done) {
+ skb_tx_timestamp(skb);
+ netdev_tx_sent_queue(q_to_ndq(q), skb->len);
+ ionic_txq_post(q, !netdev_xmit_more(), ionic_tx_clean, skb);
+ } else {
+ ionic_txq_post(q, false, ionic_tx_clean, NULL);
+ }
+}
+
+static struct ionic_txq_desc *ionic_tx_tso_next(struct ionic_queue *q,
+ struct ionic_txq_sg_elem **elem)
+{
+ struct ionic_txq_sg_desc *sg_desc = q->head->sg_desc;
+ struct ionic_txq_desc *desc = q->head->desc;
+
+ *elem = sg_desc->elems;
+ return desc;
+}
+
+static int ionic_tx_tso(struct ionic_queue *q, struct sk_buff *skb)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct ionic_desc_info *abort = q->head;
+ struct device *dev = q->lif->ionic->dev;
+ struct ionic_desc_info *rewind = abort;
+ struct ionic_txq_sg_elem *elem;
+ struct ionic_txq_desc *desc;
+ unsigned int frag_left = 0;
+ unsigned int offset = 0;
+ unsigned int len_left;
+ dma_addr_t desc_addr;
+ unsigned int hdrlen;
+ unsigned int nfrags;
+ unsigned int seglen;
+ u64 total_bytes = 0;
+ u64 total_pkts = 0;
+ unsigned int left;
+ unsigned int len;
+ unsigned int mss;
+ skb_frag_t *frag;
+ bool start, done;
+ bool outer_csum;
+ bool has_vlan;
+ u16 desc_len;
+ u8 desc_nsge;
+ u16 vlan_tci;
+ bool encap;
+ int err;
+
+ mss = skb_shinfo(skb)->gso_size;
+ nfrags = skb_shinfo(skb)->nr_frags;
+ len_left = skb->len - skb_headlen(skb);
+ outer_csum = (skb_shinfo(skb)->gso_type & SKB_GSO_GRE_CSUM) ||
+ (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_TUNNEL_CSUM);
+ has_vlan = !!skb_vlan_tag_present(skb);
+ vlan_tci = skb_vlan_tag_get(skb);
+ encap = skb->encapsulation;
+
+ /* Preload inner-most TCP csum field with IP pseudo hdr
+ * calculated with IP length set to zero. HW will later
+ * add in length to each TCP segment resulting from the TSO.
+ */
+
+ if (encap)
+ err = ionic_tx_tcp_inner_pseudo_csum(skb);
+ else
+ err = ionic_tx_tcp_pseudo_csum(skb);
+ if (err)
+ return err;
+
+ if (encap)
+ hdrlen = skb_inner_transport_header(skb) - skb->data +
+ inner_tcp_hdrlen(skb);
+ else
+ hdrlen = skb_transport_offset(skb) + tcp_hdrlen(skb);
+
+ seglen = hdrlen + mss;
+ left = skb_headlen(skb);
+
+ desc = ionic_tx_tso_next(q, &elem);
+ start = true;
+
+ /* Chop skb->data up into desc segments */
+
+ while (left > 0) {
+ len = min(seglen, left);
+ frag_left = seglen - len;
+ desc_addr = ionic_tx_map_single(q, skb->data + offset, len);
+ if (dma_mapping_error(dev, desc_addr))
+ goto err_out_abort;
+ desc_len = len;
+ desc_nsge = 0;
+ left -= len;
+ offset += len;
+ if (nfrags > 0 && frag_left > 0)
+ continue;
+ done = (nfrags == 0 && left == 0);
+ ionic_tx_tso_post(q, desc, skb,
+ desc_addr, desc_nsge, desc_len,
+ hdrlen, mss,
+ outer_csum,
+ vlan_tci, has_vlan,
+ start, done);
+ total_pkts++;
+ total_bytes += start ? len : len + hdrlen;
+ desc = ionic_tx_tso_next(q, &elem);
+ start = false;
+ seglen = mss;
+ }
+
+ /* Chop skb frags into desc segments */
+
+ for (frag = skb_shinfo(skb)->frags; len_left; frag++) {
+ offset = 0;
+ left = skb_frag_size(frag);
+ len_left -= left;
+ nfrags--;
+ stats->frags++;
+
+ while (left > 0) {
+ if (frag_left > 0) {
+ len = min(frag_left, left);
+ frag_left -= len;
+ elem->addr =
+ cpu_to_le64(ionic_tx_map_frag(q, frag,
+ offset, len));
+ if (dma_mapping_error(dev, elem->addr))
+ goto err_out_abort;
+ elem->len = cpu_to_le16(len);
+ elem++;
+ desc_nsge++;
+ left -= len;
+ offset += len;
+ if (nfrags > 0 && frag_left > 0)
+ continue;
+ done = (nfrags == 0 && left == 0);
+ ionic_tx_tso_post(q, desc, skb, desc_addr,
+ desc_nsge, desc_len,
+ hdrlen, mss, outer_csum,
+ vlan_tci, has_vlan,
+ start, done);
+ total_pkts++;
+ total_bytes += start ? len : len + hdrlen;
+ desc = ionic_tx_tso_next(q, &elem);
+ start = false;
+ } else {
+ len = min(mss, left);
+ frag_left = mss - len;
+ desc_addr = ionic_tx_map_frag(q, frag,
+ offset, len);
+ if (dma_mapping_error(dev, desc_addr))
+ goto err_out_abort;
+ desc_len = len;
+ desc_nsge = 0;
+ left -= len;
+ offset += len;
+ if (nfrags > 0 && frag_left > 0)
+ continue;
+ done = (nfrags == 0 && left == 0);
+ ionic_tx_tso_post(q, desc, skb, desc_addr,
+ desc_nsge, desc_len,
+ hdrlen, mss, outer_csum,
+ vlan_tci, has_vlan,
+ start, done);
+ total_pkts++;
+ total_bytes += start ? len : len + hdrlen;
+ desc = ionic_tx_tso_next(q, &elem);
+ start = false;
+ }
+ }
+ }
+
+ stats->pkts += total_pkts;
+ stats->bytes += total_bytes;
+ stats->tso++;
+
+ return 0;
+
+err_out_abort:
+ while (rewind->desc != q->head->desc) {
+ ionic_tx_clean(q, rewind, NULL, NULL);
+ rewind = rewind->next;
+ }
+ q->head = abort;
+
+ return -ENOMEM;
+}
+
+static int ionic_tx_calc_csum(struct ionic_queue *q, struct sk_buff *skb)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct ionic_txq_desc *desc = q->head->desc;
+ struct device *dev = q->lif->ionic->dev;
+ dma_addr_t dma_addr;
+ bool has_vlan;
+ u8 flags = 0;
+ bool encap;
+ u64 cmd;
+
+ has_vlan = !!skb_vlan_tag_present(skb);
+ encap = skb->encapsulation;
+
+ dma_addr = ionic_tx_map_single(q, skb->data, skb_headlen(skb));
+ if (dma_mapping_error(dev, dma_addr))
+ return -ENOMEM;
+
+ flags |= has_vlan ? IONIC_TXQ_DESC_FLAG_VLAN : 0;
+ flags |= encap ? IONIC_TXQ_DESC_FLAG_ENCAP : 0;
+
+ cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_PARTIAL,
+ flags, skb_shinfo(skb)->nr_frags, dma_addr);
+ desc->cmd = cpu_to_le64(cmd);
+ desc->len = cpu_to_le16(skb_headlen(skb));
+ desc->vlan_tci = cpu_to_le16(skb_vlan_tag_get(skb));
+ desc->csum_start = cpu_to_le16(skb_checksum_start_offset(skb));
+ desc->csum_offset = cpu_to_le16(skb->csum_offset);
+
+ if (skb->csum_not_inet)
+ stats->crc32_csum++;
+ else
+ stats->csum++;
+
+ return 0;
+}
+
+static int ionic_tx_calc_no_csum(struct ionic_queue *q, struct sk_buff *skb)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct ionic_txq_desc *desc = q->head->desc;
+ struct device *dev = q->lif->ionic->dev;
+ dma_addr_t dma_addr;
+ bool has_vlan;
+ u8 flags = 0;
+ bool encap;
+ u64 cmd;
+
+ has_vlan = !!skb_vlan_tag_present(skb);
+ encap = skb->encapsulation;
+
+ dma_addr = ionic_tx_map_single(q, skb->data, skb_headlen(skb));
+ if (dma_mapping_error(dev, dma_addr))
+ return -ENOMEM;
+
+ flags |= has_vlan ? IONIC_TXQ_DESC_FLAG_VLAN : 0;
+ flags |= encap ? IONIC_TXQ_DESC_FLAG_ENCAP : 0;
+
+ cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_NONE,
+ flags, skb_shinfo(skb)->nr_frags, dma_addr);
+ desc->cmd = cpu_to_le64(cmd);
+ desc->len = cpu_to_le16(skb_headlen(skb));
+ desc->vlan_tci = cpu_to_le16(skb_vlan_tag_get(skb));
+
+ stats->no_csum++;
+
+ return 0;
+}
+
+static int ionic_tx_skb_frags(struct ionic_queue *q, struct sk_buff *skb)
+{
+ struct ionic_txq_sg_desc *sg_desc = q->head->sg_desc;
+ unsigned int len_left = skb->len - skb_headlen(skb);
+ struct ionic_txq_sg_elem *elem = sg_desc->elems;
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ struct device *dev = q->lif->ionic->dev;
+ dma_addr_t dma_addr;
+ skb_frag_t *frag;
+ u16 len;
+
+ for (frag = skb_shinfo(skb)->frags; len_left; frag++, elem++) {
+ len = skb_frag_size(frag);
+ elem->len = cpu_to_le16(len);
+ dma_addr = ionic_tx_map_frag(q, frag, 0, len);
+ if (dma_mapping_error(dev, dma_addr))
+ return -ENOMEM;
+ elem->addr = cpu_to_le64(dma_addr);
+ len_left -= len;
+ stats->frags++;
+ }
+
+ return 0;
+}
+
+static int ionic_tx(struct ionic_queue *q, struct sk_buff *skb)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ int err;
+
+ /* set up the initial descriptor */
+ if (skb->ip_summed == CHECKSUM_PARTIAL)
+ err = ionic_tx_calc_csum(q, skb);
+ else
+ err = ionic_tx_calc_no_csum(q, skb);
+ if (err)
+ return err;
+
+ /* add frags */
+ err = ionic_tx_skb_frags(q, skb);
+ if (err)
+ return err;
+
+ skb_tx_timestamp(skb);
+ stats->pkts++;
+ stats->bytes += skb->len;
+
+ netdev_tx_sent_queue(q_to_ndq(q), skb->len);
+ ionic_txq_post(q, !netdev_xmit_more(), ionic_tx_clean, skb);
+
+ return 0;
+}
+
+static int ionic_tx_descs_needed(struct ionic_queue *q, struct sk_buff *skb)
+{
+ struct ionic_tx_stats *stats = q_to_tx_stats(q);
+ int err;
+
+ /* If TSO, need roundup(skb->len/mss) descs */
+ if (skb_is_gso(skb))
+ return (skb->len / skb_shinfo(skb)->gso_size) + 1;
+
+ /* If non-TSO, just need 1 desc and nr_frags sg elems */
+ if (skb_shinfo(skb)->nr_frags <= IONIC_TX_MAX_SG_ELEMS)
+ return 1;
+
+ /* Too many frags, so linearize */
+ err = skb_linearize(skb);
+ if (err)
+ return err;
+
+ stats->linearize++;
+
+ /* Need 1 desc and zero sg elems */
+ return 1;
+}
+
+static int ionic_maybe_stop_tx(struct ionic_queue *q, int ndescs)
+{
+ int stopped = 0;
+
+ if (unlikely(!ionic_q_has_space(q, ndescs))) {
+ netif_stop_subqueue(q->lif->netdev, q->index);
+ q->stop++;
+ stopped = 1;
+
+ /* Might race with ionic_tx_clean, check again */
+ smp_rmb();
+ if (ionic_q_has_space(q, ndescs)) {
+ netif_wake_subqueue(q->lif->netdev, q->index);
+ stopped = 0;
+ }
+ }
+
+ return stopped;
+}
+
+netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev)
+{
+ u16 queue_index = skb_get_queue_mapping(skb);
+ struct ionic_lif *lif = netdev_priv(netdev);
+ struct ionic_queue *q;
+ int ndescs;
+ int err;
+
+ if (unlikely(!test_bit(IONIC_LIF_UP, lif->state))) {
+ dev_kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
+ if (unlikely(!lif_to_txqcq(lif, queue_index)))
+ queue_index = 0;
+ q = lif_to_txq(lif, queue_index);
+
+ ndescs = ionic_tx_descs_needed(q, skb);
+ if (ndescs < 0)
+ goto err_out_drop;
+
+ if (unlikely(ionic_maybe_stop_tx(q, ndescs)))
+ return NETDEV_TX_BUSY;
+
+ if (skb_is_gso(skb))
+ err = ionic_tx_tso(q, skb);
+ else
+ err = ionic_tx(q, skb);
+
+ if (err)
+ goto err_out_drop;
+
+ /* Stop the queue if there aren't descriptors for the next packet.
+ * Since our SG lists per descriptor take care of most of the possible
+ * fragmentation, we don't need to have many descriptors available.
+ */
+ ionic_maybe_stop_tx(q, 4);
+
+ return NETDEV_TX_OK;
+
+err_out_drop:
+ q->stop++;
+ q->drop++;
+ dev_kfree_skb(skb);
+ return NETDEV_TX_OK;
+}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h
new file mode 100644
index 0000000..53775c6
--- /dev/null
+++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2017 - 2019 Pensando Systems, Inc */
+
+#ifndef _IONIC_TXRX_H_
+#define _IONIC_TXRX_H_
+
+void ionic_rx_flush(struct ionic_cq *cq);
+void ionic_tx_flush(struct ionic_cq *cq);
+
+void ionic_rx_fill(struct ionic_queue *q);
+void ionic_rx_empty(struct ionic_queue *q);
+int ionic_rx_napi(struct napi_struct *napi, int budget);
+netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev);
+
+#endif /* _IONIC_TXRX_H_ */
diff --git a/drivers/net/ethernet/qlogic/qed/qed_debug.c b/drivers/net/ethernet/qlogic/qed/qed_debug.c
index 5ea6c4f..859caa6 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_debug.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_debug.c
@@ -1756,6 +1756,15 @@ static u32 qed_read_unaligned_dword(u8 *buf)
return dword;
}
+/* Sets the value of the specified GRC param */
+static void qed_grc_set_param(struct qed_hwfn *p_hwfn,
+ enum dbg_grc_params grc_param, u32 val)
+{
+ struct dbg_tools_data *dev_data = &p_hwfn->dbg_info;
+
+ dev_data->grc.param_val[grc_param] = val;
+}
+
/* Returns the value of the specified GRC param */
static u32 qed_grc_get_param(struct qed_hwfn *p_hwfn,
enum dbg_grc_params grc_param)
@@ -5119,6 +5128,69 @@ bool qed_read_fw_info(struct qed_hwfn *p_hwfn,
return false;
}
+enum dbg_status qed_dbg_grc_config(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ enum dbg_grc_params grc_param, u32 val)
+{
+ enum dbg_status status;
+ int i;
+
+ DP_VERBOSE(p_hwfn, QED_MSG_DEBUG,
+ "dbg_grc_config: paramId = %d, val = %d\n", grc_param, val);
+
+ status = qed_dbg_dev_init(p_hwfn, p_ptt);
+ if (status != DBG_STATUS_OK)
+ return status;
+
+ /* Initializes the GRC parameters (if not initialized). Needed in order
+ * to set the default parameter values for the first time.
+ */
+ qed_dbg_grc_init_params(p_hwfn);
+
+ if (grc_param >= MAX_DBG_GRC_PARAMS)
+ return DBG_STATUS_INVALID_ARGS;
+ if (val < s_grc_param_defs[grc_param].min ||
+ val > s_grc_param_defs[grc_param].max)
+ return DBG_STATUS_INVALID_ARGS;
+
+ if (s_grc_param_defs[grc_param].is_preset) {
+ /* Preset param */
+
+ /* Disabling a preset is not allowed. Call
+ * dbg_grc_set_params_default instead.
+ */
+ if (!val)
+ return DBG_STATUS_INVALID_ARGS;
+
+ /* Update all params with the preset values */
+ for (i = 0; i < MAX_DBG_GRC_PARAMS; i++) {
+ u32 preset_val;
+
+ /* Skip persistent params */
+ if (s_grc_param_defs[i].is_persistent)
+ continue;
+
+ /* Find preset value */
+ if (grc_param == DBG_GRC_PARAM_EXCLUDE_ALL)
+ preset_val =
+ s_grc_param_defs[i].exclude_all_preset_val;
+ else if (grc_param == DBG_GRC_PARAM_CRASH)
+ preset_val =
+ s_grc_param_defs[i].crash_preset_val;
+ else
+ return DBG_STATUS_INVALID_ARGS;
+
+ qed_grc_set_param(p_hwfn,
+ (enum dbg_grc_params)i, preset_val);
+ }
+ } else {
+ /* Regular param - set its value */
+ qed_grc_set_param(p_hwfn, grc_param, val);
+ }
+
+ return DBG_STATUS_OK;
+}
+
/* Assign default GRC param values */
void qed_dbg_grc_set_params_default(struct qed_hwfn *p_hwfn)
{
@@ -7997,9 +8069,16 @@ static u32 qed_calc_regdump_header(enum debug_print_features feature,
int qed_dbg_all_data(struct qed_dev *cdev, void *buffer)
{
u8 cur_engine, omit_engine = 0, org_engine;
+ struct qed_hwfn *p_hwfn =
+ &cdev->hwfns[cdev->dbg_params.engine_for_debug];
+ struct dbg_tools_data *dev_data = &p_hwfn->dbg_info;
+ int grc_params[MAX_DBG_GRC_PARAMS], i;
u32 offset = 0, feature_size;
int rc;
+ for (i = 0; i < MAX_DBG_GRC_PARAMS; i++)
+ grc_params[i] = dev_data->grc.param_val[i];
+
if (cdev->num_hwfns == 1)
omit_engine = 1;
@@ -8087,6 +8166,9 @@ int qed_dbg_all_data(struct qed_dev *cdev, void *buffer)
rc);
}
+ for (i = 0; i < MAX_DBG_GRC_PARAMS; i++)
+ dev_data->grc.param_val[i] = grc_params[i];
+
/* GRC dump - must be last because when mcp stuck it will
* clutter idle_chk, reg_fifo, ...
*/
diff --git a/drivers/net/ethernet/qlogic/qed/qed_hsi.h b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
index 557a12e..cf3ceb6 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_hsi.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
@@ -3024,6 +3024,21 @@ void qed_read_regs(struct qed_hwfn *p_hwfn,
*/
bool qed_read_fw_info(struct qed_hwfn *p_hwfn,
struct qed_ptt *p_ptt, struct fw_info *fw_info);
+/**
+ * @brief qed_dbg_grc_config - Sets the value of a GRC parameter.
+ *
+ * @param p_hwfn - HW device data
+ * @param grc_param - GRC parameter
+ * @param val - Value to set.
+ *
+ * @return error if one of the following holds:
+ * - the version wasn't set
+ * - grc_param is invalid
+ * - val is outside the allowed boundaries
+ */
+enum dbg_status qed_dbg_grc_config(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ enum dbg_grc_params grc_param, u32 val);
/**
* @brief qed_dbg_grc_set_params_default - Reverts all GRC parameters to their
diff --git a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
index f380fae..65ec16a 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_iwarp.c
@@ -382,7 +382,7 @@ qed_iwarp2roce_state(enum qed_iwarp_qp_state state)
}
}
-const static char *iwarp_state_names[] = {
+static const char * const iwarp_state_names[] = {
"IDLE",
"RTS",
"TERMINATE",
diff --git a/drivers/net/ethernet/qlogic/qed/qed_main.c b/drivers/net/ethernet/qlogic/qed/qed_main.c
index 0a76459..ac1511a8 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_main.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_main.c
@@ -69,6 +69,8 @@
#define QED_RDMA_SRQS QED_ROCE_QPS
#define QED_NVM_CFG_SET_FLAGS 0xE
#define QED_NVM_CFG_SET_PF_FLAGS 0x1E
+#define QED_NVM_CFG_GET_FLAGS 0xA
+#define QED_NVM_CFG_GET_PF_FLAGS 0x1A
static char version[] =
"QLogic FastLinQ 4xxxx Core Module qed " DRV_MODULE_VERSION "\n";
@@ -1327,7 +1329,7 @@ static int qed_slowpath_start(struct qed_dev *cdev,
&drv_version);
if (rc) {
DP_NOTICE(cdev, "Failed sending drv version command\n");
- return rc;
+ goto err4;
}
}
@@ -1335,6 +1337,8 @@ static int qed_slowpath_start(struct qed_dev *cdev,
return 0;
+err4:
+ qed_ll2_dealloc_if(cdev);
err3:
qed_hw_stop(cdev);
err2:
@@ -2296,6 +2300,30 @@ static int qed_nvm_flash_cfg_write(struct qed_dev *cdev, const u8 **data)
return rc;
}
+static int qed_nvm_flash_cfg_read(struct qed_dev *cdev, u8 **data,
+ u32 cmd, u32 entity_id)
+{
+ struct qed_hwfn *hwfn = QED_LEADING_HWFN(cdev);
+ struct qed_ptt *ptt;
+ u32 flags, len;
+ int rc = 0;
+
+ ptt = qed_ptt_acquire(hwfn);
+ if (!ptt)
+ return -EAGAIN;
+
+ DP_VERBOSE(cdev, NETIF_MSG_DRV,
+ "Read config cmd = %d entity id %d\n", cmd, entity_id);
+ flags = entity_id ? QED_NVM_CFG_GET_PF_FLAGS : QED_NVM_CFG_GET_FLAGS;
+ rc = qed_mcp_nvm_get_cfg(hwfn, ptt, cmd, entity_id, flags, *data, &len);
+ if (rc)
+ DP_ERR(cdev, "Error %d reading %d\n", rc, cmd);
+
+ qed_ptt_release(hwfn, ptt);
+
+ return rc;
+}
+
static int qed_nvm_flash(struct qed_dev *cdev, const char *name)
{
const struct firmware *image;
@@ -2555,6 +2583,26 @@ static int qed_read_module_eeprom(struct qed_dev *cdev, char *buf,
return rc;
}
+static int qed_set_grc_config(struct qed_dev *cdev, u32 cfg_id, u32 val)
+{
+ struct qed_hwfn *hwfn = QED_LEADING_HWFN(cdev);
+ struct qed_ptt *ptt;
+ int rc = 0;
+
+ if (IS_VF(cdev))
+ return 0;
+
+ ptt = qed_ptt_acquire(hwfn);
+ if (!ptt)
+ return -EAGAIN;
+
+ rc = qed_dbg_grc_config(hwfn, ptt, cfg_id, val);
+
+ qed_ptt_release(hwfn, ptt);
+
+ return rc;
+}
+
static u8 qed_get_affin_hwfn_idx(struct qed_dev *cdev)
{
return QED_AFFIN_HWFN_IDX(cdev);
@@ -2608,6 +2656,8 @@ const struct qed_common_ops qed_common_ops_pass = {
.db_recovery_del = &qed_db_recovery_del,
.read_module_eeprom = &qed_read_module_eeprom,
.get_affin_hwfn_idx = &qed_get_affin_hwfn_idx,
+ .read_nvm_cfg = &qed_nvm_flash_cfg_read,
+ .set_grc_config = &qed_set_grc_config,
};
void qed_get_protocol_stats(struct qed_dev *cdev,
diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.c b/drivers/net/ethernet/qlogic/qed/qed_mcp.c
index 89462c4..36ddb89 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_mcp.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.c
@@ -3751,6 +3751,35 @@ int qed_mcp_get_ppfid_bitmap(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
return 0;
}
+int qed_mcp_nvm_get_cfg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
+ u16 option_id, u8 entity_id, u16 flags, u8 *p_buf,
+ u32 *p_len)
+{
+ u32 mb_param = 0, resp, param;
+ int rc;
+
+ QED_MFW_SET_FIELD(mb_param, DRV_MB_PARAM_NVM_CFG_OPTION_ID, option_id);
+ if (flags & QED_NVM_CFG_OPTION_INIT)
+ QED_MFW_SET_FIELD(mb_param,
+ DRV_MB_PARAM_NVM_CFG_OPTION_INIT, 1);
+ if (flags & QED_NVM_CFG_OPTION_FREE)
+ QED_MFW_SET_FIELD(mb_param,
+ DRV_MB_PARAM_NVM_CFG_OPTION_FREE, 1);
+ if (flags & QED_NVM_CFG_OPTION_ENTITY_SEL) {
+ QED_MFW_SET_FIELD(mb_param,
+ DRV_MB_PARAM_NVM_CFG_OPTION_ENTITY_SEL, 1);
+ QED_MFW_SET_FIELD(mb_param,
+ DRV_MB_PARAM_NVM_CFG_OPTION_ENTITY_ID,
+ entity_id);
+ }
+
+ rc = qed_mcp_nvm_rd_cmd(p_hwfn, p_ptt,
+ DRV_MSG_CODE_GET_NVM_CFG_OPTION,
+ mb_param, &resp, ¶m, p_len, (u32 *)p_buf);
+
+ return rc;
+}
+
int qed_mcp_nvm_set_cfg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
u16 option_id, u8 entity_id, u16 flags, u8 *p_buf,
u32 len)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.h b/drivers/net/ethernet/qlogic/qed/qed_mcp.h
index 83649a8..9c4c276 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_mcp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.h
@@ -1209,6 +1209,21 @@ int qed_mcp_get_engine_config(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
int qed_mcp_get_ppfid_bitmap(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
/**
+ * @brief Get NVM config attribute value.
+ *
+ * @param p_hwfn
+ * @param p_ptt
+ * @param option_id
+ * @param entity_id
+ * @param flags
+ * @param p_buf
+ * @param p_len
+ */
+int qed_mcp_nvm_get_cfg(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt,
+ u16 option_id, u8 entity_id, u16 flags, u8 *p_buf,
+ u32 *p_len);
+
+/**
* @brief Set NVM config attribute value.
*
* @param p_hwfn
diff --git a/drivers/net/ethernet/qlogic/qede/qede.h b/drivers/net/ethernet/qlogic/qede/qede.h
index 0e931c0..c303a92 100644
--- a/drivers/net/ethernet/qlogic/qede/qede.h
+++ b/drivers/net/ethernet/qlogic/qede/qede.h
@@ -177,6 +177,20 @@ enum qede_flags_bit {
QEDE_FLAGS_TX_TIMESTAMPING_EN
};
+#define QEDE_DUMP_MAX_ARGS 4
+enum qede_dump_cmd {
+ QEDE_DUMP_CMD_NONE = 0,
+ QEDE_DUMP_CMD_NVM_CFG,
+ QEDE_DUMP_CMD_GRCDUMP,
+ QEDE_DUMP_CMD_MAX
+};
+
+struct qede_dump_info {
+ enum qede_dump_cmd cmd;
+ u8 num_args;
+ u32 args[QEDE_DUMP_MAX_ARGS];
+};
+
struct qede_dev {
struct qed_dev *cdev;
struct net_device *ndev;
@@ -262,6 +276,7 @@ struct qede_dev {
struct qede_rdma_dev rdma_info;
struct bpf_prog *xdp_prog;
+ struct qede_dump_info dump_info;
};
enum QEDE_STATE {
diff --git a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
index abcee47..ec27a43 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
@@ -48,6 +48,9 @@
{QEDE_RQSTAT_OFFSET(stat_name), QEDE_RQSTAT_STRING(stat_name)}
#define QEDE_SELFTEST_POLL_COUNT 100
+#define QEDE_DUMP_VERSION 0x1
+#define QEDE_DUMP_NVM_BUF_LEN 32
+#define QEDE_DUMP_NVM_ARG_COUNT 2
static const struct {
u64 offset;
@@ -1973,6 +1976,114 @@ static int qede_get_module_eeprom(struct net_device *dev,
return rc;
}
+static int qede_set_dump(struct net_device *dev, struct ethtool_dump *val)
+{
+ struct qede_dev *edev = netdev_priv(dev);
+ int rc = 0;
+
+ if (edev->dump_info.cmd == QEDE_DUMP_CMD_NONE) {
+ if (val->flag > QEDE_DUMP_CMD_MAX) {
+ DP_ERR(edev, "Invalid command %d\n", val->flag);
+ return -EINVAL;
+ }
+ edev->dump_info.cmd = val->flag;
+ edev->dump_info.num_args = 0;
+ return 0;
+ }
+
+ if (edev->dump_info.num_args == QEDE_DUMP_MAX_ARGS) {
+ DP_ERR(edev, "Arg count = %d\n", edev->dump_info.num_args);
+ return -EINVAL;
+ }
+
+ switch (edev->dump_info.cmd) {
+ case QEDE_DUMP_CMD_NVM_CFG:
+ edev->dump_info.args[edev->dump_info.num_args] = val->flag;
+ edev->dump_info.num_args++;
+ break;
+ case QEDE_DUMP_CMD_GRCDUMP:
+ rc = edev->ops->common->set_grc_config(edev->cdev,
+ val->flag, 1);
+ break;
+ default:
+ break;
+ }
+
+ return rc;
+}
+
+static int qede_get_dump_flag(struct net_device *dev,
+ struct ethtool_dump *dump)
+{
+ struct qede_dev *edev = netdev_priv(dev);
+
+ if (!edev->ops || !edev->ops->common) {
+ DP_ERR(edev, "Edev ops not populated\n");
+ return -EINVAL;
+ }
+
+ dump->version = QEDE_DUMP_VERSION;
+ switch (edev->dump_info.cmd) {
+ case QEDE_DUMP_CMD_NVM_CFG:
+ dump->flag = QEDE_DUMP_CMD_NVM_CFG;
+ dump->len = QEDE_DUMP_NVM_BUF_LEN;
+ break;
+ case QEDE_DUMP_CMD_GRCDUMP:
+ dump->flag = QEDE_DUMP_CMD_GRCDUMP;
+ dump->len = edev->ops->common->dbg_all_data_size(edev->cdev);
+ break;
+ default:
+ DP_ERR(edev, "Invalid cmd = %d\n", edev->dump_info.cmd);
+ return -EINVAL;
+ }
+
+ DP_VERBOSE(edev, QED_MSG_DEBUG,
+ "dump->version = 0x%x dump->flag = %d dump->len = %d\n",
+ dump->version, dump->flag, dump->len);
+ return 0;
+}
+
+static int qede_get_dump_data(struct net_device *dev,
+ struct ethtool_dump *dump, void *buf)
+{
+ struct qede_dev *edev = netdev_priv(dev);
+ int rc = 0;
+
+ if (!edev->ops || !edev->ops->common) {
+ DP_ERR(edev, "Edev ops not populated\n");
+ edev->dump_info.cmd = QEDE_DUMP_CMD_NONE;
+ edev->dump_info.num_args = 0;
+ return -EINVAL;
+ }
+
+ switch (edev->dump_info.cmd) {
+ case QEDE_DUMP_CMD_NVM_CFG:
+ if (edev->dump_info.num_args != QEDE_DUMP_NVM_ARG_COUNT) {
+ DP_ERR(edev, "Arg count = %d required = %d\n",
+ edev->dump_info.num_args,
+ QEDE_DUMP_NVM_ARG_COUNT);
+ return -EINVAL;
+ }
+ rc = edev->ops->common->read_nvm_cfg(edev->cdev, (u8 **)&buf,
+ edev->dump_info.args[0],
+ edev->dump_info.args[1]);
+ break;
+ case QEDE_DUMP_CMD_GRCDUMP:
+ memset(buf, 0, dump->len);
+ rc = edev->ops->common->dbg_all_data(edev->cdev, buf);
+ break;
+ default:
+ DP_ERR(edev, "Invalid cmd = %d\n", edev->dump_info.cmd);
+ rc = -EINVAL;
+ break;
+ }
+
+ edev->dump_info.cmd = QEDE_DUMP_CMD_NONE;
+ edev->dump_info.num_args = 0;
+
+ return rc;
+}
+
static const struct ethtool_ops qede_ethtool_ops = {
.get_link_ksettings = qede_get_link_ksettings,
.set_link_ksettings = qede_set_link_ksettings,
@@ -2014,6 +2125,9 @@ static const struct ethtool_ops qede_ethtool_ops = {
.get_tunable = qede_get_tunable,
.set_tunable = qede_set_tunable,
.flash_device = qede_flash_device,
+ .get_dump_flag = qede_get_dump_flag,
+ .get_dump_data = qede_get_dump_data,
+ .set_dump = qede_set_dump,
};
static const struct ethtool_ops qede_vf_ethtool_ops = {
diff --git a/drivers/net/ethernet/qualcomm/qca_spi.c b/drivers/net/ethernet/qualcomm/qca_spi.c
index b28360b..5ecf61d 100644
--- a/drivers/net/ethernet/qualcomm/qca_spi.c
+++ b/drivers/net/ethernet/qualcomm/qca_spi.c
@@ -837,8 +837,7 @@ qcaspi_netdev_uninit(struct net_device *dev)
kfree(qca->rx_buffer);
qca->buffer_size = 0;
- if (qca->rx_skb)
- dev_kfree_skb(qca->rx_skb);
+ dev_kfree_skb(qca->rx_skb);
}
static const struct net_device_ops qcaspi_netdev_ops = {
diff --git a/drivers/net/ethernet/qualcomm/qca_uart.c b/drivers/net/ethernet/qualcomm/qca_uart.c
index 5906168..0981068 100644
--- a/drivers/net/ethernet/qualcomm/qca_uart.c
+++ b/drivers/net/ethernet/qualcomm/qca_uart.c
@@ -285,8 +285,7 @@ static void qcauart_netdev_uninit(struct net_device *dev)
{
struct qcauart *qca = netdev_priv(dev);
- if (qca->rx_skb)
- dev_kfree_skb(qca->rx_skb);
+ dev_kfree_skb(qca->rx_skb);
}
static const struct net_device_ops qcauart_netdev_ops = {
diff --git a/drivers/net/ethernet/realtek/Kconfig b/drivers/net/ethernet/realtek/Kconfig
index b18e7a9..5e0b9d2 100644
--- a/drivers/net/ethernet/realtek/Kconfig
+++ b/drivers/net/ethernet/realtek/Kconfig
@@ -96,14 +96,19 @@
old RX-reset behavior. If unsure, say N.
config R8169
- tristate "Realtek 8169 gigabit ethernet support"
+ tristate "Realtek 8169/8168/8101/8125 ethernet support"
depends on PCI
select FW_LOADER
select CRC32
select PHYLIB
select REALTEK_PHY
---help---
- Say Y here if you have a Realtek 8169 PCI Gigabit Ethernet adapter.
+ Say Y here if you have a Realtek Ethernet adapter belonging to
+ the following families:
+ RTL8169 Gigabit Ethernet
+ RTL8168 Gigabit Ethernet
+ RTL8101 Fast Ethernet
+ RTL8125 2.5GBit Ethernet
To compile this driver as a module, choose M here: the module
will be called r8169. This is recommended.
diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
index 9109441..0ef01db 100644
--- a/drivers/net/ethernet/realtek/r8169_main.c
+++ b/drivers/net/ethernet/realtek/r8169_main.c
@@ -55,6 +55,7 @@
#define FIRMWARE_8168H_2 "rtl_nic/rtl8168h-2.fw"
#define FIRMWARE_8107E_1 "rtl_nic/rtl8107e-1.fw"
#define FIRMWARE_8107E_2 "rtl_nic/rtl8107e-2.fw"
+#define FIRMWARE_8125A_3 "rtl_nic/rtl8125a-3.fw"
#define R8169_MSG_DEFAULT \
(NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_IFUP | NETIF_MSG_IFDOWN)
@@ -135,6 +136,8 @@ enum mac_version {
RTL_GIGA_MAC_VER_49,
RTL_GIGA_MAC_VER_50,
RTL_GIGA_MAC_VER_51,
+ RTL_GIGA_MAC_VER_60,
+ RTL_GIGA_MAC_VER_61,
RTL_GIGA_MAC_NONE
};
@@ -200,6 +203,8 @@ static const struct {
[RTL_GIGA_MAC_VER_49] = {"RTL8168ep/8111ep" },
[RTL_GIGA_MAC_VER_50] = {"RTL8168ep/8111ep" },
[RTL_GIGA_MAC_VER_51] = {"RTL8168ep/8111ep" },
+ [RTL_GIGA_MAC_VER_60] = {"RTL8125" },
+ [RTL_GIGA_MAC_VER_61] = {"RTL8125", FIRMWARE_8125A_3},
};
static const struct pci_device_id rtl8169_pci_tbl[] = {
@@ -220,6 +225,8 @@ static const struct pci_device_id rtl8169_pci_tbl[] = {
{ PCI_VDEVICE(USR, 0x0116) },
{ PCI_VENDOR_ID_LINKSYS, 0x1032, PCI_ANY_ID, 0x0024 },
{ 0x0001, 0x8168, PCI_ANY_ID, 0x2410 },
+ { PCI_VDEVICE(REALTEK, 0x8125) },
+ { PCI_VDEVICE(REALTEK, 0x3000) },
{}
};
@@ -384,6 +391,19 @@ enum rtl8168_registers {
#define EARLY_TALLY_EN (1 << 16)
};
+enum rtl8125_registers {
+ IntrMask_8125 = 0x38,
+ IntrStatus_8125 = 0x3c,
+ TxPoll_8125 = 0x90,
+ MAC0_BKP = 0x19e0,
+};
+
+#define RX_VLAN_INNER_8125 BIT(22)
+#define RX_VLAN_OUTER_8125 BIT(23)
+#define RX_VLAN_8125 (RX_VLAN_INNER_8125 | RX_VLAN_OUTER_8125)
+
+#define RX_FETCH_DFLT_8125 (8 << 27)
+
enum rtl_register_content {
/* InterruptStatusBits */
SYSErr = 0x8000,
@@ -645,7 +665,7 @@ struct rtl8169_private {
struct page *Rx_databuff[NUM_RX_DESC]; /* Rx data buffers */
struct ring_info tx_skb[NUM_TX_DESC]; /* Tx data buffers */
u16 cp_cmd;
- u16 irq_mask;
+ u32 irq_mask;
struct clk *clk;
struct {
@@ -695,6 +715,7 @@ MODULE_FIRMWARE(FIRMWARE_8168H_1);
MODULE_FIRMWARE(FIRMWARE_8168H_2);
MODULE_FIRMWARE(FIRMWARE_8107E_1);
MODULE_FIRMWARE(FIRMWARE_8107E_2);
+MODULE_FIRMWARE(FIRMWARE_8125A_3);
static inline struct device *tp_to_dev(struct rtl8169_private *tp)
{
@@ -727,10 +748,16 @@ static void rtl_tx_performance_tweak(struct rtl8169_private *tp, u16 force)
PCI_EXP_DEVCTL_READRQ, force);
}
+static bool rtl_is_8125(struct rtl8169_private *tp)
+{
+ return tp->mac_version >= RTL_GIGA_MAC_VER_60;
+}
+
static bool rtl_is_8168evl_up(struct rtl8169_private *tp)
{
return tp->mac_version >= RTL_GIGA_MAC_VER_34 &&
- tp->mac_version != RTL_GIGA_MAC_VER_39;
+ tp->mac_version != RTL_GIGA_MAC_VER_39 &&
+ tp->mac_version <= RTL_GIGA_MAC_VER_51;
}
static bool rtl_supports_eee(struct rtl8169_private *tp)
@@ -740,6 +767,14 @@ static bool rtl_supports_eee(struct rtl8169_private *tp)
tp->mac_version != RTL_GIGA_MAC_VER_39;
}
+static void rtl_read_mac_from_reg(struct rtl8169_private *tp, u8 *mac, int reg)
+{
+ int i;
+
+ for (i = 0; i < ETH_ALEN; i++)
+ mac[i] = RTL_R8(tp, reg + i);
+}
+
struct rtl_cond {
bool (*check)(struct rtl8169_private *);
const char *msg;
@@ -1014,7 +1049,7 @@ static void rtl_writephy(struct rtl8169_private *tp, int location, int val)
case RTL_GIGA_MAC_VER_31:
r8168dp_2_mdio_write(tp, location, val);
break;
- case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_51:
+ case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_61:
r8168g_mdio_write(tp, location, val);
break;
default:
@@ -1031,7 +1066,7 @@ static int rtl_readphy(struct rtl8169_private *tp, int location)
case RTL_GIGA_MAC_VER_28:
case RTL_GIGA_MAC_VER_31:
return r8168dp_2_mdio_read(tp, location);
- case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_51:
+ case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_61:
return r8168g_mdio_read(tp, location);
default:
return r8169_mdio_read(tp, location);
@@ -1313,14 +1348,28 @@ static u8 rtl8168d_efuse_read(struct rtl8169_private *tp, int reg_addr)
RTL_R32(tp, EFUSEAR) & EFUSEAR_DATA_MASK : ~0;
}
-static void rtl_ack_events(struct rtl8169_private *tp, u16 bits)
+static u32 rtl_get_events(struct rtl8169_private *tp)
{
- RTL_W16(tp, IntrStatus, bits);
+ if (rtl_is_8125(tp))
+ return RTL_R32(tp, IntrStatus_8125);
+ else
+ return RTL_R16(tp, IntrStatus);
+}
+
+static void rtl_ack_events(struct rtl8169_private *tp, u32 bits)
+{
+ if (rtl_is_8125(tp))
+ RTL_W32(tp, IntrStatus_8125, bits);
+ else
+ RTL_W16(tp, IntrStatus, bits);
}
static void rtl_irq_disable(struct rtl8169_private *tp)
{
- RTL_W16(tp, IntrMask, 0);
+ if (rtl_is_8125(tp))
+ RTL_W32(tp, IntrMask_8125, 0);
+ else
+ RTL_W16(tp, IntrMask, 0);
tp->irq_enabled = 0;
}
@@ -1331,13 +1380,16 @@ static void rtl_irq_disable(struct rtl8169_private *tp)
static void rtl_irq_enable(struct rtl8169_private *tp)
{
tp->irq_enabled = 1;
- RTL_W16(tp, IntrMask, tp->irq_mask);
+ if (rtl_is_8125(tp))
+ RTL_W32(tp, IntrMask_8125, tp->irq_mask);
+ else
+ RTL_W16(tp, IntrMask, tp->irq_mask);
}
static void rtl8169_irq_mask_and_ack(struct rtl8169_private *tp)
{
rtl_irq_disable(tp);
- rtl_ack_events(tp, 0xffff);
+ rtl_ack_events(tp, 0xffffffff);
/* PCI commit */
RTL_R8(tp, ChipCmd);
}
@@ -1396,7 +1448,6 @@ static void rtl8169_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
static void __rtl8169_set_wol(struct rtl8169_private *tp, u32 wolopts)
{
- unsigned int i, tmp;
static const struct {
u32 opt;
u16 reg;
@@ -1409,20 +1460,25 @@ static void __rtl8169_set_wol(struct rtl8169_private *tp, u32 wolopts)
{ WAKE_ANY, Config5, LanWake },
{ WAKE_MAGIC, Config3, MagicPacket }
};
+ unsigned int i, tmp = ARRAY_SIZE(cfg);
u8 options;
rtl_unlock_config_regs(tp);
if (rtl_is_8168evl_up(tp)) {
- tmp = ARRAY_SIZE(cfg) - 1;
+ tmp--;
if (wolopts & WAKE_MAGIC)
rtl_eri_set_bits(tp, 0x0dc, ERIAR_MASK_0100,
MagicPacket_v2);
else
rtl_eri_clear_bits(tp, 0x0dc, ERIAR_MASK_0100,
MagicPacket_v2);
- } else {
- tmp = ARRAY_SIZE(cfg);
+ } else if (rtl_is_8125(tp)) {
+ tmp--;
+ if (wolopts & WAKE_MAGIC)
+ r8168_mac_ocp_modify(tp, 0xc0b6, 0, BIT(0));
+ else
+ r8168_mac_ocp_modify(tp, 0xc0b6, BIT(0), 0);
}
for (i = 0; i < tmp; i++) {
@@ -1528,6 +1584,13 @@ static int rtl8169_set_features(struct net_device *dev,
else
rx_config &= ~(AcceptErr | AcceptRunt);
+ if (rtl_is_8125(tp)) {
+ if (features & NETIF_F_HW_VLAN_CTAG_RX)
+ rx_config |= RX_VLAN_8125;
+ else
+ rx_config &= ~RX_VLAN_8125;
+ }
+
RTL_W32(tp, RxConfig, rx_config);
if (features & NETIF_F_RXCSUM)
@@ -1535,10 +1598,12 @@ static int rtl8169_set_features(struct net_device *dev,
else
tp->cp_cmd &= ~RxChkSum;
- if (features & NETIF_F_HW_VLAN_CTAG_RX)
- tp->cp_cmd |= RxVlan;
- else
- tp->cp_cmd &= ~RxVlan;
+ if (!rtl_is_8125(tp)) {
+ if (features & NETIF_F_HW_VLAN_CTAG_RX)
+ tp->cp_cmd |= RxVlan;
+ else
+ tp->cp_cmd &= ~RxVlan;
+ }
RTL_W16(tp, CPlusCmd, tp->cp_cmd);
RTL_R16(tp, CPlusCmd);
@@ -1837,6 +1902,9 @@ static int rtl_get_coalesce(struct net_device *dev, struct ethtool_coalesce *ec)
int i;
u16 w;
+ if (rtl_is_8125(tp))
+ return -EOPNOTSUPP;
+
memset(ec, 0, sizeof(*ec));
/* get rx/tx scale corresponding to current speed and CPlusCmd[0:1] */
@@ -1905,6 +1973,9 @@ static int rtl_set_coalesce(struct net_device *dev, struct ethtool_coalesce *ec)
u16 w = 0, cp01;
int i;
+ if (rtl_is_8125(tp))
+ return -EOPNOTSUPP;
+
scale = rtl_coalesce_choose_scale(dev,
max(p[0].usecs, p[1].usecs) * 1000, &cp01);
if (IS_ERR(scale))
@@ -2051,6 +2122,10 @@ static void rtl8169_get_mac_version(struct rtl8169_private *tp)
u16 val;
u16 mac_version;
} mac_info[] = {
+ /* 8125 family. */
+ { 0x7cf, 0x608, RTL_GIGA_MAC_VER_60 },
+ { 0x7c8, 0x608, RTL_GIGA_MAC_VER_61 },
+
/* 8168EP family. */
{ 0x7cf, 0x502, RTL_GIGA_MAC_VER_51 },
{ 0x7cf, 0x501, RTL_GIGA_MAC_VER_50 },
@@ -2196,6 +2271,12 @@ static void rtl8168_config_eee_mac(struct rtl8169_private *tp)
rtl_eri_set_bits(tp, 0x1b0, ERIAR_MASK_1111, 0x0003);
}
+static void rtl8125_config_eee_mac(struct rtl8169_private *tp)
+{
+ r8168_mac_ocp_modify(tp, 0xe040, 0, BIT(1) | BIT(0));
+ r8168_mac_ocp_modify(tp, 0xeb62, 0, BIT(2) | BIT(1));
+}
+
static void rtl8168f_config_eee_phy(struct rtl8169_private *tp)
{
struct phy_device *phydev = tp->phydev;
@@ -2226,6 +2307,16 @@ static void rtl8168h_config_eee_phy(struct rtl8169_private *tp)
phy_modify_paged(phydev, 0xa42, 0x14, 0x0000, 0x0080);
}
+static void rtl8125_config_eee_phy(struct rtl8169_private *tp)
+{
+ struct phy_device *phydev = tp->phydev;
+
+ rtl8168h_config_eee_phy(tp);
+
+ phy_modify_paged(phydev, 0xa6d, 0x12, 0x0001, 0x0000);
+ phy_modify_paged(phydev, 0xa6d, 0x14, 0x0010, 0x0000);
+}
+
static void rtl8169s_hw_phy_config(struct rtl8169_private *tp)
{
static const struct phy_reg phy_reg_init[] = {
@@ -3546,6 +3637,134 @@ static void rtl8106e_hw_phy_config(struct rtl8169_private *tp)
rtl_eri_write(tp, 0x1d0, ERIAR_MASK_0011, 0x0000);
}
+static void rtl8125_1_hw_phy_config(struct rtl8169_private *tp)
+{
+ struct phy_device *phydev = tp->phydev;
+
+ phy_modify_paged(phydev, 0xad4, 0x10, 0x03ff, 0x0084);
+ phy_modify_paged(phydev, 0xad4, 0x17, 0x0000, 0x0010);
+ phy_modify_paged(phydev, 0xad1, 0x13, 0x03ff, 0x0006);
+ phy_modify_paged(phydev, 0xad3, 0x11, 0x003f, 0x0006);
+ phy_modify_paged(phydev, 0xac0, 0x14, 0x0000, 0x1100);
+ phy_modify_paged(phydev, 0xac8, 0x15, 0xf000, 0x7000);
+ phy_modify_paged(phydev, 0xad1, 0x14, 0x0000, 0x0400);
+ phy_modify_paged(phydev, 0xad1, 0x15, 0x0000, 0x03ff);
+ phy_modify_paged(phydev, 0xad1, 0x16, 0x0000, 0x03ff);
+
+ phy_write(phydev, 0x1f, 0x0a43);
+ phy_write(phydev, 0x13, 0x80ea);
+ phy_modify(phydev, 0x14, 0xff00, 0xc400);
+ phy_write(phydev, 0x13, 0x80eb);
+ phy_modify(phydev, 0x14, 0x0700, 0x0300);
+ phy_write(phydev, 0x13, 0x80f8);
+ phy_modify(phydev, 0x14, 0xff00, 0x1c00);
+ phy_write(phydev, 0x13, 0x80f1);
+ phy_modify(phydev, 0x14, 0xff00, 0x3000);
+ phy_write(phydev, 0x13, 0x80fe);
+ phy_modify(phydev, 0x14, 0xff00, 0xa500);
+ phy_write(phydev, 0x13, 0x8102);
+ phy_modify(phydev, 0x14, 0xff00, 0x5000);
+ phy_write(phydev, 0x13, 0x8105);
+ phy_modify(phydev, 0x14, 0xff00, 0x3300);
+ phy_write(phydev, 0x13, 0x8100);
+ phy_modify(phydev, 0x14, 0xff00, 0x7000);
+ phy_write(phydev, 0x13, 0x8104);
+ phy_modify(phydev, 0x14, 0xff00, 0xf000);
+ phy_write(phydev, 0x13, 0x8106);
+ phy_modify(phydev, 0x14, 0xff00, 0x6500);
+ phy_write(phydev, 0x13, 0x80dc);
+ phy_modify(phydev, 0x14, 0xff00, 0xed00);
+ phy_write(phydev, 0x13, 0x80df);
+ phy_set_bits(phydev, 0x14, BIT(8));
+ phy_write(phydev, 0x13, 0x80e1);
+ phy_clear_bits(phydev, 0x14, BIT(8));
+ phy_write(phydev, 0x1f, 0x0000);
+
+ phy_modify_paged(phydev, 0xbf0, 0x13, 0x003f, 0x0038);
+ phy_write_paged(phydev, 0xa43, 0x13, 0x819f);
+ phy_write_paged(phydev, 0xa43, 0x14, 0xd0b6);
+
+ phy_write_paged(phydev, 0xbc3, 0x12, 0x5555);
+ phy_modify_paged(phydev, 0xbf0, 0x15, 0x0e00, 0x0a00);
+ phy_modify_paged(phydev, 0xa5c, 0x10, 0x0400, 0x0000);
+ phy_modify_paged(phydev, 0xa44, 0x11, 0x0000, 0x0800);
+
+ rtl8125_config_eee_phy(tp);
+ rtl_enable_eee(tp);
+}
+
+static void rtl8125_2_hw_phy_config(struct rtl8169_private *tp)
+{
+ struct phy_device *phydev = tp->phydev;
+ int i;
+
+ phy_modify_paged(phydev, 0xad4, 0x17, 0x0000, 0x0010);
+ phy_modify_paged(phydev, 0xad1, 0x13, 0x03ff, 0x03ff);
+ phy_modify_paged(phydev, 0xad3, 0x11, 0x003f, 0x0006);
+ phy_modify_paged(phydev, 0xac0, 0x14, 0x1100, 0x0000);
+ phy_modify_paged(phydev, 0xacc, 0x10, 0x0003, 0x0002);
+ phy_modify_paged(phydev, 0xad4, 0x10, 0x00e7, 0x0044);
+ phy_modify_paged(phydev, 0xac1, 0x12, 0x0080, 0x0000);
+ phy_modify_paged(phydev, 0xac8, 0x10, 0x0300, 0x0000);
+ phy_modify_paged(phydev, 0xac5, 0x17, 0x0007, 0x0002);
+ phy_write_paged(phydev, 0xad4, 0x16, 0x00a8);
+ phy_write_paged(phydev, 0xac5, 0x16, 0x01ff);
+ phy_modify_paged(phydev, 0xac8, 0x15, 0x00f0, 0x0030);
+
+ phy_write(phydev, 0x1f, 0x0b87);
+ phy_write(phydev, 0x16, 0x80a2);
+ phy_write(phydev, 0x17, 0x0153);
+ phy_write(phydev, 0x16, 0x809c);
+ phy_write(phydev, 0x17, 0x0153);
+ phy_write(phydev, 0x1f, 0x0000);
+
+ phy_write(phydev, 0x1f, 0x0a43);
+ phy_write(phydev, 0x13, 0x81B3);
+ phy_write(phydev, 0x14, 0x0043);
+ phy_write(phydev, 0x14, 0x00A7);
+ phy_write(phydev, 0x14, 0x00D6);
+ phy_write(phydev, 0x14, 0x00EC);
+ phy_write(phydev, 0x14, 0x00F6);
+ phy_write(phydev, 0x14, 0x00FB);
+ phy_write(phydev, 0x14, 0x00FD);
+ phy_write(phydev, 0x14, 0x00FF);
+ phy_write(phydev, 0x14, 0x00BB);
+ phy_write(phydev, 0x14, 0x0058);
+ phy_write(phydev, 0x14, 0x0029);
+ phy_write(phydev, 0x14, 0x0013);
+ phy_write(phydev, 0x14, 0x0009);
+ phy_write(phydev, 0x14, 0x0004);
+ phy_write(phydev, 0x14, 0x0002);
+ for (i = 0; i < 25; i++)
+ phy_write(phydev, 0x14, 0x0000);
+
+ phy_write(phydev, 0x13, 0x8257);
+ phy_write(phydev, 0x14, 0x020F);
+
+ phy_write(phydev, 0x13, 0x80EA);
+ phy_write(phydev, 0x14, 0x7843);
+ phy_write(phydev, 0x1f, 0x0000);
+
+ rtl_apply_firmware(tp);
+
+ phy_modify_paged(phydev, 0xd06, 0x14, 0x0000, 0x2000);
+
+ phy_write(phydev, 0x1f, 0x0a43);
+ phy_write(phydev, 0x13, 0x81a2);
+ phy_set_bits(phydev, 0x14, BIT(8));
+ phy_write(phydev, 0x1f, 0x0000);
+
+ phy_modify_paged(phydev, 0xb54, 0x16, 0xff00, 0xdb00);
+ phy_modify_paged(phydev, 0xa45, 0x12, 0x0001, 0x0000);
+ phy_modify_paged(phydev, 0xa5d, 0x12, 0x0000, 0x0020);
+ phy_modify_paged(phydev, 0xad4, 0x17, 0x0010, 0x0000);
+ phy_modify_paged(phydev, 0xa86, 0x15, 0x0001, 0x0000);
+ phy_modify_paged(phydev, 0xa44, 0x11, 0x0000, 0x0800);
+
+ rtl8125_config_eee_phy(tp);
+ rtl_enable_eee(tp);
+}
+
static void rtl_hw_phy_config(struct net_device *dev)
{
static const rtl_generic_fct phy_configs[] = {
@@ -3601,6 +3820,8 @@ static void rtl_hw_phy_config(struct net_device *dev)
[RTL_GIGA_MAC_VER_49] = rtl8168ep_1_hw_phy_config,
[RTL_GIGA_MAC_VER_50] = rtl8168ep_2_hw_phy_config,
[RTL_GIGA_MAC_VER_51] = rtl8168ep_2_hw_phy_config,
+ [RTL_GIGA_MAC_VER_60] = rtl8125_1_hw_phy_config,
+ [RTL_GIGA_MAC_VER_61] = rtl8125_2_hw_phy_config,
};
struct rtl8169_private *tp = netdev_priv(dev);
@@ -3728,6 +3949,8 @@ static void rtl_pll_power_down(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_48:
case RTL_GIGA_MAC_VER_50:
case RTL_GIGA_MAC_VER_51:
+ case RTL_GIGA_MAC_VER_60:
+ case RTL_GIGA_MAC_VER_61:
RTL_W8(tp, PMCH, RTL_R8(tp, PMCH) & ~0x80);
break;
case RTL_GIGA_MAC_VER_40:
@@ -3757,6 +3980,8 @@ static void rtl_pll_power_up(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_48:
case RTL_GIGA_MAC_VER_50:
case RTL_GIGA_MAC_VER_51:
+ case RTL_GIGA_MAC_VER_60:
+ case RTL_GIGA_MAC_VER_61:
RTL_W8(tp, PMCH, RTL_R8(tp, PMCH) | 0xc0);
break;
case RTL_GIGA_MAC_VER_40:
@@ -3789,6 +4014,10 @@ static void rtl_init_rxcfg(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_51:
RTL_W32(tp, RxConfig, RX128_INT_EN | RX_MULTI_EN | RX_DMA_BURST | RX_EARLY_OFF);
break;
+ case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_61:
+ RTL_W32(tp, RxConfig, RX_FETCH_DFLT_8125 | RX_VLAN_8125 |
+ RX_DMA_BURST);
+ break;
default:
RTL_W32(tp, RxConfig, RX128_INT_EN | RX_DMA_BURST);
break;
@@ -5006,6 +5235,128 @@ static void rtl_hw_start_8106(struct rtl8169_private *tp)
rtl_hw_aspm_clkreq_enable(tp, true);
}
+DECLARE_RTL_COND(rtl_mac_ocp_e00e_cond)
+{
+ return r8168_mac_ocp_read(tp, 0xe00e) & BIT(13);
+}
+
+static void rtl_hw_start_8125_common(struct rtl8169_private *tp)
+{
+ rtl_pcie_state_l2l3_disable(tp);
+
+ RTL_W16(tp, 0x382, 0x221b);
+ RTL_W8(tp, 0x4500, 0);
+ RTL_W16(tp, 0x4800, 0);
+
+ /* disable UPS */
+ r8168_mac_ocp_modify(tp, 0xd40a, 0x0010, 0x0000);
+
+ RTL_W8(tp, Config1, RTL_R8(tp, Config1) & ~0x10);
+
+ r8168_mac_ocp_write(tp, 0xc140, 0xffff);
+ r8168_mac_ocp_write(tp, 0xc142, 0xffff);
+
+ r8168_mac_ocp_modify(tp, 0xd3e2, 0x0fff, 0x03a9);
+ r8168_mac_ocp_modify(tp, 0xd3e4, 0x00ff, 0x0000);
+ r8168_mac_ocp_modify(tp, 0xe860, 0x0000, 0x0080);
+
+ /* disable new tx descriptor format */
+ r8168_mac_ocp_modify(tp, 0xeb58, 0x0001, 0x0000);
+
+ r8168_mac_ocp_modify(tp, 0xe614, 0x0700, 0x0400);
+ r8168_mac_ocp_modify(tp, 0xe63e, 0x0c30, 0x0020);
+ r8168_mac_ocp_modify(tp, 0xc0b4, 0x0000, 0x000c);
+ r8168_mac_ocp_modify(tp, 0xeb6a, 0x00ff, 0x0033);
+ r8168_mac_ocp_modify(tp, 0xeb50, 0x03e0, 0x0040);
+ r8168_mac_ocp_modify(tp, 0xe056, 0x00f0, 0x0030);
+ r8168_mac_ocp_modify(tp, 0xe040, 0x1000, 0x0000);
+ r8168_mac_ocp_modify(tp, 0xe0c0, 0x4f0f, 0x4403);
+ r8168_mac_ocp_modify(tp, 0xe052, 0x0080, 0x0067);
+ r8168_mac_ocp_modify(tp, 0xc0ac, 0x0080, 0x1f00);
+ r8168_mac_ocp_modify(tp, 0xd430, 0x0fff, 0x047f);
+ r8168_mac_ocp_modify(tp, 0xe84c, 0x0000, 0x00c0);
+ r8168_mac_ocp_modify(tp, 0xea1c, 0x0004, 0x0000);
+ r8168_mac_ocp_modify(tp, 0xeb54, 0x0000, 0x0001);
+ udelay(1);
+ r8168_mac_ocp_modify(tp, 0xeb54, 0x0001, 0x0000);
+ RTL_W16(tp, 0x1880, RTL_R16(tp, 0x1880) & ~0x0030);
+
+ r8168_mac_ocp_write(tp, 0xe098, 0xc302);
+
+ rtl_udelay_loop_wait_low(tp, &rtl_mac_ocp_e00e_cond, 1000, 10);
+
+ rtl8125_config_eee_mac(tp);
+
+ RTL_W32(tp, MISC, RTL_R32(tp, MISC) & ~RXDV_GATED_EN);
+ udelay(10);
+}
+
+static void rtl_hw_start_8125_1(struct rtl8169_private *tp)
+{
+ static const struct ephy_info e_info_8125_1[] = {
+ { 0x01, 0xffff, 0xa812 },
+ { 0x09, 0xffff, 0x520c },
+ { 0x04, 0xffff, 0xd000 },
+ { 0x0d, 0xffff, 0xf702 },
+ { 0x0a, 0xffff, 0x8653 },
+ { 0x06, 0xffff, 0x001e },
+ { 0x08, 0xffff, 0x3595 },
+ { 0x20, 0xffff, 0x9455 },
+ { 0x21, 0xffff, 0x99ff },
+ { 0x02, 0xffff, 0x6046 },
+ { 0x29, 0xffff, 0xfe00 },
+ { 0x23, 0xffff, 0xab62 },
+
+ { 0x41, 0xffff, 0xa80c },
+ { 0x49, 0xffff, 0x520c },
+ { 0x44, 0xffff, 0xd000 },
+ { 0x4d, 0xffff, 0xf702 },
+ { 0x4a, 0xffff, 0x8653 },
+ { 0x46, 0xffff, 0x001e },
+ { 0x48, 0xffff, 0x3595 },
+ { 0x60, 0xffff, 0x9455 },
+ { 0x61, 0xffff, 0x99ff },
+ { 0x42, 0xffff, 0x6046 },
+ { 0x69, 0xffff, 0xfe00 },
+ { 0x63, 0xffff, 0xab62 },
+ };
+
+ rtl_set_def_aspm_entry_latency(tp);
+
+ /* disable aspm and clock request before access ephy */
+ rtl_hw_aspm_clkreq_enable(tp, false);
+ rtl_ephy_init(tp, e_info_8125_1);
+
+ rtl_hw_start_8125_common(tp);
+}
+
+static void rtl_hw_start_8125_2(struct rtl8169_private *tp)
+{
+ static const struct ephy_info e_info_8125_2[] = {
+ { 0x04, 0xffff, 0xd000 },
+ { 0x0a, 0xffff, 0x8653 },
+ { 0x23, 0xffff, 0xab66 },
+ { 0x20, 0xffff, 0x9455 },
+ { 0x21, 0xffff, 0x99ff },
+ { 0x29, 0xffff, 0xfe04 },
+
+ { 0x44, 0xffff, 0xd000 },
+ { 0x4a, 0xffff, 0x8653 },
+ { 0x63, 0xffff, 0xab66 },
+ { 0x60, 0xffff, 0x9455 },
+ { 0x61, 0xffff, 0x99ff },
+ { 0x69, 0xffff, 0xfe04 },
+ };
+
+ rtl_set_def_aspm_entry_latency(tp);
+
+ /* disable aspm and clock request before access ephy */
+ rtl_hw_aspm_clkreq_enable(tp, false);
+ rtl_ephy_init(tp, e_info_8125_2);
+
+ rtl_hw_start_8125_common(tp);
+}
+
static void rtl_hw_config(struct rtl8169_private *tp)
{
static const rtl_generic_fct hw_configs[] = {
@@ -5054,12 +5405,25 @@ static void rtl_hw_config(struct rtl8169_private *tp)
[RTL_GIGA_MAC_VER_49] = rtl_hw_start_8168ep_1,
[RTL_GIGA_MAC_VER_50] = rtl_hw_start_8168ep_2,
[RTL_GIGA_MAC_VER_51] = rtl_hw_start_8168ep_3,
+ [RTL_GIGA_MAC_VER_60] = rtl_hw_start_8125_1,
+ [RTL_GIGA_MAC_VER_61] = rtl_hw_start_8125_2,
};
if (hw_configs[tp->mac_version])
hw_configs[tp->mac_version](tp);
}
+static void rtl_hw_start_8125(struct rtl8169_private *tp)
+{
+ int i;
+
+ /* disable interrupt coalescing */
+ for (i = 0xa00; i < 0xb00; i += 4)
+ RTL_W32(tp, i, 0);
+
+ rtl_hw_config(tp);
+}
+
static void rtl_hw_start_8168(struct rtl8169_private *tp)
{
if (tp->mac_version == RTL_GIGA_MAC_VER_13 ||
@@ -5073,6 +5437,9 @@ static void rtl_hw_start_8168(struct rtl8169_private *tp)
RTL_W8(tp, MaxTxPacketSize, TxPacketMax);
rtl_hw_config(tp);
+
+ /* disable interrupt coalescing */
+ RTL_W16(tp, IntrMitigate, 0x0000);
}
static void rtl_hw_start_8169(struct rtl8169_private *tp)
@@ -5096,6 +5463,9 @@ static void rtl_hw_start_8169(struct rtl8169_private *tp)
rtl8169_set_magic_reg(tp, tp->mac_version);
RTL_W32(tp, RxMissed, 0);
+
+ /* disable interrupt coalescing */
+ RTL_W16(tp, IntrMitigate, 0x0000);
}
static void rtl_hw_start(struct rtl8169_private *tp)
@@ -5107,6 +5477,8 @@ static void rtl_hw_start(struct rtl8169_private *tp)
if (tp->mac_version <= RTL_GIGA_MAC_VER_06)
rtl_hw_start_8169(tp);
+ else if (rtl_is_8125(tp))
+ rtl_hw_start_8125(tp);
else
rtl_hw_start_8168(tp);
@@ -5114,10 +5486,8 @@ static void rtl_hw_start(struct rtl8169_private *tp)
rtl_set_rx_tx_desc_registers(tp);
rtl_lock_config_regs(tp);
- /* disable interrupt coalescing */
- RTL_W16(tp, IntrMitigate, 0x0000);
/* Initially a 10 us delay. Turned it into a PCI commit. - FR */
- RTL_R8(tp, IntrMask);
+ RTL_R16(tp, CPlusCmd);
RTL_W8(tp, ChipCmd, CmdTxEnb | CmdRxEnb);
rtl_init_rxcfg(tp);
rtl_set_tx_config_registers(tp);
@@ -5492,6 +5862,14 @@ static bool rtl_chip_supports_csum_v2(struct rtl8169_private *tp)
}
}
+static void rtl8169_doorbell(struct rtl8169_private *tp)
+{
+ if (rtl_is_8125(tp))
+ RTL_W16(tp, TxPoll_8125, BIT(0));
+ else
+ RTL_W8(tp, TxPoll, NPQ);
+}
+
static netdev_tx_t rtl8169_start_xmit(struct sk_buff *skb,
struct net_device *dev)
{
@@ -5571,7 +5949,7 @@ static netdev_tx_t rtl8169_start_xmit(struct sk_buff *skb,
}
if (door_bell)
- RTL_W8(tp, TxPoll, NPQ);
+ rtl8169_doorbell(tp);
if (unlikely(stop_queue)) {
/* Sync with rtl_tx:
@@ -5695,7 +6073,7 @@ static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp,
rtl8169_unmap_tx_skb(tp_to_dev(tp), tx_skb,
tp->TxDescArray + entry);
- if (status & LastFrag) {
+ if (tx_skb->skb) {
pkts_compl++;
bytes_compl += tx_skb->skb->len;
napi_consume_skb(tx_skb->skb, budget);
@@ -5733,7 +6111,7 @@ static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp,
* it is slow enough). -- FR
*/
if (tp->cur_tx != dirty_tx)
- RTL_W8(tp, TxPoll, NPQ);
+ rtl8169_doorbell(tp);
}
}
@@ -5807,21 +6185,24 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, u32 budget
goto release_descriptor;
}
- dma_sync_single_for_cpu(tp_to_dev(tp),
- le64_to_cpu(desc->addr),
- pkt_size, DMA_FROM_DEVICE);
-
skb = napi_alloc_skb(&tp->napi, pkt_size);
if (unlikely(!skb)) {
dev->stats.rx_dropped++;
goto release_descriptor;
}
+ dma_sync_single_for_cpu(tp_to_dev(tp),
+ le64_to_cpu(desc->addr),
+ pkt_size, DMA_FROM_DEVICE);
prefetch(rx_buf);
skb_copy_to_linear_data(skb, rx_buf, pkt_size);
skb->tail += pkt_size;
skb->len = pkt_size;
+ dma_sync_single_for_device(tp_to_dev(tp),
+ le64_to_cpu(desc->addr),
+ pkt_size, DMA_FROM_DEVICE);
+
rtl8169_rx_csum(skb, status);
skb->protocol = eth_type_trans(skb, dev);
@@ -5851,9 +6232,10 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, u32 budget
static irqreturn_t rtl8169_interrupt(int irq, void *dev_instance)
{
struct rtl8169_private *tp = dev_instance;
- u16 status = RTL_R16(tp, IntrStatus);
+ u32 status = rtl_get_events(tp);
- if (!tp->irq_enabled || status == 0xffff || !(status & tp->irq_mask))
+ if (!tp->irq_enabled || (status & 0xffff) == 0xffff ||
+ !(status & tp->irq_mask))
return IRQ_NONE;
if (unlikely(status & SYSErr)) {
@@ -6451,6 +6833,8 @@ static void rtl_read_mac_address(struct rtl8169_private *tp,
value = rtl_eri_read(tp, 0xe4);
mac_addr[4] = (value >> 0) & 0xff;
mac_addr[5] = (value >> 8) & 0xff;
+ } else if (rtl_is_8125(tp)) {
+ rtl_read_mac_from_reg(tp, mac_addr, MAC0_BKP);
}
}
@@ -6548,6 +6932,31 @@ static void rtl_hw_init_8168g(struct rtl8169_private *tp)
rtl_udelay_loop_wait_high(tp, &rtl_link_list_ready_cond, 100, 42);
}
+static void rtl_hw_init_8125(struct rtl8169_private *tp)
+{
+ tp->ocp_base = OCP_STD_PHY_BASE;
+
+ RTL_W32(tp, MISC, RTL_R32(tp, MISC) | RXDV_GATED_EN);
+
+ if (!rtl_udelay_loop_wait_high(tp, &rtl_rxtx_empty_cond, 100, 42))
+ return;
+
+ RTL_W8(tp, ChipCmd, RTL_R8(tp, ChipCmd) & ~(CmdTxEnb | CmdRxEnb));
+ msleep(1);
+ RTL_W8(tp, MCU, RTL_R8(tp, MCU) & ~NOW_IS_OOB);
+
+ r8168_mac_ocp_modify(tp, 0xe8de, BIT(14), 0);
+
+ if (!rtl_udelay_loop_wait_high(tp, &rtl_link_list_ready_cond, 100, 42))
+ return;
+
+ r8168_mac_ocp_write(tp, 0xc0aa, 0x07d0);
+ r8168_mac_ocp_write(tp, 0xc0a6, 0x0150);
+ r8168_mac_ocp_write(tp, 0xc01e, 0x5555);
+
+ rtl_udelay_loop_wait_high(tp, &rtl_link_list_ready_cond, 100, 42);
+}
+
static void rtl_hw_initialize(struct rtl8169_private *tp)
{
switch (tp->mac_version) {
@@ -6557,6 +6966,9 @@ static void rtl_hw_initialize(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_40 ... RTL_GIGA_MAC_VER_48:
rtl_hw_init_8168g(tp);
break;
+ case RTL_GIGA_MAC_VER_60 ... RTL_GIGA_MAC_VER_61:
+ rtl_hw_init_8125(tp);
+ break;
default:
break;
}
@@ -6620,7 +7032,7 @@ static void rtl_init_mac_address(struct rtl8169_private *tp)
{
struct net_device *dev = tp->dev;
u8 *mac_addr = dev->dev_addr;
- int rc, i;
+ int rc;
rc = eth_platform_get_mac_address(tp_to_dev(tp), mac_addr);
if (!rc)
@@ -6630,8 +7042,7 @@ static void rtl_init_mac_address(struct rtl8169_private *tp)
if (is_valid_ether_addr(mac_addr))
goto done;
- for (i = 0; i < ETH_ALEN; i++)
- mac_addr[i] = RTL_R8(tp, MAC0 + i);
+ rtl_read_mac_from_reg(tp, mac_addr, MAC0);
if (is_valid_ether_addr(mac_addr))
goto done;
@@ -6753,8 +7164,10 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
NETIF_F_HIGHDMA;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
- tp->cp_cmd |= RxChkSum | RxVlan;
-
+ tp->cp_cmd |= RxChkSum;
+ /* RTL8125 uses register RxConfig for VLAN offloading config */
+ if (!rtl_is_8125(tp))
+ tp->cp_cmd |= RxVlan;
/*
* Pretend we are using VLANs; This bypasses a nasty bug where
* Interrupts stop flowing on high load on 8110SCd controllers.
diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h
index ac9195a..a9c89d5 100644
--- a/drivers/net/ethernet/renesas/ravb.h
+++ b/drivers/net/ethernet/renesas/ravb.h
@@ -193,16 +193,12 @@ enum ravb_reg {
GECMR = 0x05b0,
MAHR = 0x05c0,
MALR = 0x05c8,
- TROCR = 0x0700, /* Undocumented? */
- CDCR = 0x0708, /* Undocumented? */
- LCCR = 0x0710, /* Undocumented? */
+ TROCR = 0x0700, /* R-Car Gen3 only */
CEFCR = 0x0740,
FRECR = 0x0748,
TSFRCR = 0x0750,
TLFRCR = 0x0758,
RFCR = 0x0760,
- CERCR = 0x0768, /* Undocumented? */
- CEECR = 0x0770, /* Undocumented? */
MAFCR = 0x0778,
};
@@ -220,7 +216,6 @@ enum CCC_BIT {
CCC_CSEL_HPB = 0x00010000,
CCC_CSEL_ETH_TX = 0x00020000,
CCC_CSEL_GMII_REF = 0x00030000,
- CCC_BOC = 0x00100000, /* Undocumented? */
CCC_LBME = 0x01000000,
};
@@ -317,7 +312,7 @@ enum UFCD_BIT {
/* SFO */
enum SFO_BIT {
- SFO_FPB = 0x0000003F,
+ SFO_FBP = 0x0000003F,
};
/* RTC */
diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c
index 6cacd5e..de9aa8c 100644
--- a/drivers/net/ethernet/renesas/ravb_main.c
+++ b/drivers/net/ethernet/renesas/ravb_main.c
@@ -447,12 +447,6 @@ static int ravb_dmac_init(struct net_device *ndev)
ravb_ring_format(ndev, RAVB_BE);
ravb_ring_format(ndev, RAVB_NC);
-#if defined(__LITTLE_ENDIAN)
- ravb_modify(ndev, CCC, CCC_BOC, 0);
-#else
- ravb_modify(ndev, CCC, CCC_BOC, CCC_BOC);
-#endif
-
/* Set AVB RX */
ravb_write(ndev,
RCR_EFFS | RCR_ENCF | RCR_ETS0 | RCR_ESF | 0x18000000, RCR);
@@ -1627,17 +1621,10 @@ static struct net_device_stats *ravb_get_stats(struct net_device *ndev)
stats0 = &priv->stats[RAVB_BE];
stats1 = &priv->stats[RAVB_NC];
- nstats->tx_dropped += ravb_read(ndev, TROCR);
- ravb_write(ndev, 0, TROCR); /* (write clear) */
- nstats->collisions += ravb_read(ndev, CDCR);
- ravb_write(ndev, 0, CDCR); /* (write clear) */
- nstats->tx_carrier_errors += ravb_read(ndev, LCCR);
- ravb_write(ndev, 0, LCCR); /* (write clear) */
-
- nstats->tx_carrier_errors += ravb_read(ndev, CERCR);
- ravb_write(ndev, 0, CERCR); /* (write clear) */
- nstats->tx_carrier_errors += ravb_read(ndev, CEECR);
- ravb_write(ndev, 0, CEECR); /* (write clear) */
+ if (priv->chip_id == RCAR_GEN3) {
+ nstats->tx_dropped += ravb_read(ndev, TROCR);
+ ravb_write(ndev, 0, TROCR); /* (write clear) */
+ }
nstats->rx_packets = stats0->rx_packets + stats1->rx_packets;
nstats->tx_packets = stats0->tx_packets + stats1->tx_packets;
diff --git a/drivers/net/ethernet/rocker/rocker_main.c b/drivers/net/ethernet/rocker/rocker_main.c
index 2c5d3f5..786b158 100644
--- a/drivers/net/ethernet/rocker/rocker_main.c
+++ b/drivers/net/ethernet/rocker/rocker_main.c
@@ -2189,6 +2189,9 @@ static int rocker_router_fib_event(struct notifier_block *nb,
struct rocker_fib_event_work *fib_work;
struct fib_notifier_info *info = ptr;
+ if (!net_eq(info->net, &init_net))
+ return NOTIFY_DONE;
+
if (info->family != AF_INET)
return NOTIFY_DONE;
diff --git a/drivers/net/ethernet/seeq/sgiseeq.c b/drivers/net/ethernet/seeq/sgiseeq.c
index 7a5e6c5..276c7ca 100644
--- a/drivers/net/ethernet/seeq/sgiseeq.c
+++ b/drivers/net/ethernet/seeq/sgiseeq.c
@@ -794,15 +794,16 @@ static int sgiseeq_probe(struct platform_device *pdev)
printk(KERN_ERR "Sgiseeq: Cannot register net device, "
"aborting.\n");
err = -ENODEV;
- goto err_out_free_page;
+ goto err_out_free_attrs;
}
printk(KERN_INFO "%s: %s %pM\n", dev->name, sgiseeqstr, dev->dev_addr);
return 0;
-err_out_free_page:
- free_page((unsigned long) sp->srings);
+err_out_free_attrs:
+ dma_free_attrs(&pdev->dev, sizeof(*sp->srings), sp->srings,
+ sp->srings_dma, DMA_ATTR_NON_CONSISTENT);
err_out_free_dev:
free_netdev(dev);
diff --git a/drivers/net/ethernet/sgi/ioc3-eth.c b/drivers/net/ethernet/sgi/ioc3-eth.c
index 358e66b..deb636d 100644
--- a/drivers/net/ethernet/sgi/ioc3-eth.c
+++ b/drivers/net/ethernet/sgi/ioc3-eth.c
@@ -1,9 +1,5 @@
-/*
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- *
- * Driver for SGI's IOC3 based Ethernet cards as found in the PCI card.
+// SPDX-License-Identifier: GPL-2.0
+/* Driver for SGI's IOC3 based Ethernet cards as found in the PCI card.
*
* Copyright (C) 1999, 2000, 01, 03, 06 Ralf Baechle
* Copyright (C) 1995, 1999, 2000, 2001 by Silicon Graphics, Inc.
@@ -15,11 +11,8 @@
*
* To do:
*
- * o Handle allocation failures in ioc3_alloc_skb() more gracefully.
- * o Handle allocation failures in ioc3_init_rings().
* o Use prefetching for large packets. What is a good lower limit for
* prefetching?
- * o We're probably allocating a bit too much memory.
* o Use hardware checksums.
* o Convert to using a IOC3 meta driver.
* o Which PHYs might possibly be attached to the IOC3 in real live,
@@ -39,10 +32,10 @@
#include <linux/crc32.h>
#include <linux/mii.h>
#include <linux/in.h>
+#include <linux/io.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/udp.h>
-#include <linux/dma-mapping.h>
#include <linux/gfp.h>
#ifdef CONFIG_SERIAL_8250
@@ -55,32 +48,52 @@
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
#include <linux/skbuff.h>
+#include <linux/dma-direct.h>
+
#include <net/ip.h>
#include <asm/byteorder.h>
-#include <asm/io.h>
#include <asm/pgtable.h>
#include <linux/uaccess.h>
#include <asm/sn/types.h>
#include <asm/sn/ioc3.h>
#include <asm/pci/bridge.h>
-/*
- * 64 RX buffers. This is tunable in the range of 16 <= x < 512. The
- * value must be a power of two.
+/* Number of RX buffers. This is tunable in the range of 16 <= x < 512.
+ * The value must be a power of two.
*/
-#define RX_BUFFS 64
+#define RX_BUFFS 64
+#define RX_RING_ENTRIES 512 /* fixed in hardware */
+#define RX_RING_MASK (RX_RING_ENTRIES - 1)
+#define RX_RING_SIZE (RX_RING_ENTRIES * sizeof(u64))
-#define ETCSR_FD ((17<<ETCSR_IPGR2_SHIFT) | (11<<ETCSR_IPGR1_SHIFT) | 21)
-#define ETCSR_HD ((21<<ETCSR_IPGR2_SHIFT) | (21<<ETCSR_IPGR1_SHIFT) | 21)
+/* 128 TX buffers (not tunable) */
+#define TX_RING_ENTRIES 128
+#define TX_RING_MASK (TX_RING_ENTRIES - 1)
+#define TX_RING_SIZE (TX_RING_ENTRIES * sizeof(struct ioc3_etxd))
+
+/* IOC3 does dma transfers in 128 byte blocks */
+#define IOC3_DMA_XFER_LEN 128UL
+
+/* Every RX buffer starts with 8 byte descriptor data */
+#define RX_OFFSET (sizeof(struct ioc3_erxbuf) + NET_IP_ALIGN)
+#define RX_BUF_SIZE (13 * IOC3_DMA_XFER_LEN)
+
+#define ETCSR_FD ((21 << ETCSR_IPGR2_SHIFT) | (21 << ETCSR_IPGR1_SHIFT) | 21)
+#define ETCSR_HD ((17 << ETCSR_IPGR2_SHIFT) | (11 << ETCSR_IPGR1_SHIFT) | 21)
/* Private per NIC data of the driver. */
struct ioc3_private {
- struct ioc3 *regs;
+ struct ioc3_ethregs *regs;
+ struct ioc3 *all_regs;
+ struct device *dma_dev;
+ u32 *ssram;
unsigned long *rxr; /* pointer to receiver ring */
struct ioc3_etxd *txr;
- struct sk_buff *rx_skbs[512];
- struct sk_buff *tx_skbs[128];
+ dma_addr_t rxr_dma;
+ dma_addr_t txr_dma;
+ struct sk_buff *rx_skbs[RX_RING_ENTRIES];
+ struct sk_buff *tx_skbs[TX_RING_ENTRIES];
int rx_ci; /* RX consumer index */
int rx_pi; /* RX producer index */
int tx_ci; /* TX consumer index */
@@ -102,190 +115,138 @@ static void ioc3_set_multicast_list(struct net_device *dev);
static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev);
static void ioc3_timeout(struct net_device *dev);
static inline unsigned int ioc3_hash(const unsigned char *addr);
+static void ioc3_start(struct ioc3_private *ip);
static inline void ioc3_stop(struct ioc3_private *ip);
static void ioc3_init(struct net_device *dev);
+static int ioc3_alloc_rx_bufs(struct net_device *dev);
+static void ioc3_free_rx_bufs(struct ioc3_private *ip);
+static inline void ioc3_clean_tx_ring(struct ioc3_private *ip);
static const char ioc3_str[] = "IOC3 Ethernet";
static const struct ethtool_ops ioc3_ethtool_ops;
-/* We use this to acquire receive skb's that we can DMA directly into. */
-
-#define IOC3_CACHELINE 128UL
static inline unsigned long aligned_rx_skb_addr(unsigned long addr)
{
- return (~addr + 1) & (IOC3_CACHELINE - 1UL);
+ return (~addr + 1) & (IOC3_DMA_XFER_LEN - 1UL);
}
-static inline struct sk_buff * ioc3_alloc_skb(unsigned long length,
- unsigned int gfp_mask)
+static inline int ioc3_alloc_skb(struct ioc3_private *ip, struct sk_buff **skb,
+ struct ioc3_erxbuf **rxb, dma_addr_t *rxb_dma)
{
- struct sk_buff *skb;
+ struct sk_buff *new_skb;
+ dma_addr_t d;
+ int offset;
- skb = alloc_skb(length + IOC3_CACHELINE - 1, gfp_mask);
- if (likely(skb)) {
- int offset = aligned_rx_skb_addr((unsigned long) skb->data);
- if (offset)
- skb_reserve(skb, offset);
+ new_skb = alloc_skb(RX_BUF_SIZE + IOC3_DMA_XFER_LEN - 1, GFP_ATOMIC);
+ if (!new_skb)
+ return -ENOMEM;
+
+ /* ensure buffer is aligned to IOC3_DMA_XFER_LEN */
+ offset = aligned_rx_skb_addr((unsigned long)new_skb->data);
+ if (offset)
+ skb_reserve(new_skb, offset);
+
+ d = dma_map_single(ip->dma_dev, new_skb->data,
+ RX_BUF_SIZE, DMA_FROM_DEVICE);
+
+ if (dma_mapping_error(ip->dma_dev, d)) {
+ dev_kfree_skb_any(new_skb);
+ return -ENOMEM;
}
+ *rxb_dma = d;
+ *rxb = (struct ioc3_erxbuf *)new_skb->data;
+ skb_reserve(new_skb, RX_OFFSET);
+ *skb = new_skb;
- return skb;
+ return 0;
}
-static inline unsigned long ioc3_map(void *ptr, unsigned long vdev)
+#ifdef CONFIG_PCI_XTALK_BRIDGE
+static inline unsigned long ioc3_map(dma_addr_t addr, unsigned long attr)
{
-#ifdef CONFIG_SGI_IP27
- vdev <<= 57; /* Shift to PCI64_ATTR_VIRTUAL */
-
- return vdev | (0xaUL << PCI64_ATTR_TARG_SHFT) | PCI64_ATTR_PREF |
- ((unsigned long)ptr & TO_PHYS_MASK);
-#else
- return virt_to_bus(ptr);
-#endif
+ return (addr & ~PCI64_ATTR_BAR) | attr;
}
-/* BEWARE: The IOC3 documentation documents the size of rx buffers as
- 1644 while it's actually 1664. This one was nasty to track down ... */
-#define RX_OFFSET 10
-#define RX_BUF_ALLOC_SIZE (1664 + RX_OFFSET + IOC3_CACHELINE)
+#define ERBAR_VAL (ERBAR_BARRIER_BIT << ERBAR_RXBARR_SHIFT)
+#else
+static inline unsigned long ioc3_map(dma_addr_t addr, unsigned long attr)
+{
+ return addr;
+}
-/* DMA barrier to separate cached and uncached accesses. */
-#define BARRIER() \
- __asm__("sync" ::: "memory")
-
+#define ERBAR_VAL 0
+#endif
#define IOC3_SIZE 0x100000
-/*
- * IOC3 is a big endian device
- *
- * Unorthodox but makes the users of these macros more readable - the pointer
- * to the IOC3's memory mapped registers is expected as struct ioc3 * ioc3
- * in the environment.
- */
-#define ioc3_r_mcr() be32_to_cpu(ioc3->mcr)
-#define ioc3_w_mcr(v) do { ioc3->mcr = cpu_to_be32(v); } while (0)
-#define ioc3_w_gpcr_s(v) do { ioc3->gpcr_s = cpu_to_be32(v); } while (0)
-#define ioc3_r_emcr() be32_to_cpu(ioc3->emcr)
-#define ioc3_w_emcr(v) do { ioc3->emcr = cpu_to_be32(v); } while (0)
-#define ioc3_r_eisr() be32_to_cpu(ioc3->eisr)
-#define ioc3_w_eisr(v) do { ioc3->eisr = cpu_to_be32(v); } while (0)
-#define ioc3_r_eier() be32_to_cpu(ioc3->eier)
-#define ioc3_w_eier(v) do { ioc3->eier = cpu_to_be32(v); } while (0)
-#define ioc3_r_ercsr() be32_to_cpu(ioc3->ercsr)
-#define ioc3_w_ercsr(v) do { ioc3->ercsr = cpu_to_be32(v); } while (0)
-#define ioc3_r_erbr_h() be32_to_cpu(ioc3->erbr_h)
-#define ioc3_w_erbr_h(v) do { ioc3->erbr_h = cpu_to_be32(v); } while (0)
-#define ioc3_r_erbr_l() be32_to_cpu(ioc3->erbr_l)
-#define ioc3_w_erbr_l(v) do { ioc3->erbr_l = cpu_to_be32(v); } while (0)
-#define ioc3_r_erbar() be32_to_cpu(ioc3->erbar)
-#define ioc3_w_erbar(v) do { ioc3->erbar = cpu_to_be32(v); } while (0)
-#define ioc3_r_ercir() be32_to_cpu(ioc3->ercir)
-#define ioc3_w_ercir(v) do { ioc3->ercir = cpu_to_be32(v); } while (0)
-#define ioc3_r_erpir() be32_to_cpu(ioc3->erpir)
-#define ioc3_w_erpir(v) do { ioc3->erpir = cpu_to_be32(v); } while (0)
-#define ioc3_r_ertr() be32_to_cpu(ioc3->ertr)
-#define ioc3_w_ertr(v) do { ioc3->ertr = cpu_to_be32(v); } while (0)
-#define ioc3_r_etcsr() be32_to_cpu(ioc3->etcsr)
-#define ioc3_w_etcsr(v) do { ioc3->etcsr = cpu_to_be32(v); } while (0)
-#define ioc3_r_ersr() be32_to_cpu(ioc3->ersr)
-#define ioc3_w_ersr(v) do { ioc3->ersr = cpu_to_be32(v); } while (0)
-#define ioc3_r_etcdc() be32_to_cpu(ioc3->etcdc)
-#define ioc3_w_etcdc(v) do { ioc3->etcdc = cpu_to_be32(v); } while (0)
-#define ioc3_r_ebir() be32_to_cpu(ioc3->ebir)
-#define ioc3_w_ebir(v) do { ioc3->ebir = cpu_to_be32(v); } while (0)
-#define ioc3_r_etbr_h() be32_to_cpu(ioc3->etbr_h)
-#define ioc3_w_etbr_h(v) do { ioc3->etbr_h = cpu_to_be32(v); } while (0)
-#define ioc3_r_etbr_l() be32_to_cpu(ioc3->etbr_l)
-#define ioc3_w_etbr_l(v) do { ioc3->etbr_l = cpu_to_be32(v); } while (0)
-#define ioc3_r_etcir() be32_to_cpu(ioc3->etcir)
-#define ioc3_w_etcir(v) do { ioc3->etcir = cpu_to_be32(v); } while (0)
-#define ioc3_r_etpir() be32_to_cpu(ioc3->etpir)
-#define ioc3_w_etpir(v) do { ioc3->etpir = cpu_to_be32(v); } while (0)
-#define ioc3_r_emar_h() be32_to_cpu(ioc3->emar_h)
-#define ioc3_w_emar_h(v) do { ioc3->emar_h = cpu_to_be32(v); } while (0)
-#define ioc3_r_emar_l() be32_to_cpu(ioc3->emar_l)
-#define ioc3_w_emar_l(v) do { ioc3->emar_l = cpu_to_be32(v); } while (0)
-#define ioc3_r_ehar_h() be32_to_cpu(ioc3->ehar_h)
-#define ioc3_w_ehar_h(v) do { ioc3->ehar_h = cpu_to_be32(v); } while (0)
-#define ioc3_r_ehar_l() be32_to_cpu(ioc3->ehar_l)
-#define ioc3_w_ehar_l(v) do { ioc3->ehar_l = cpu_to_be32(v); } while (0)
-#define ioc3_r_micr() be32_to_cpu(ioc3->micr)
-#define ioc3_w_micr(v) do { ioc3->micr = cpu_to_be32(v); } while (0)
-#define ioc3_r_midr_r() be32_to_cpu(ioc3->midr_r)
-#define ioc3_w_midr_r(v) do { ioc3->midr_r = cpu_to_be32(v); } while (0)
-#define ioc3_r_midr_w() be32_to_cpu(ioc3->midr_w)
-#define ioc3_w_midr_w(v) do { ioc3->midr_w = cpu_to_be32(v); } while (0)
-
static inline u32 mcr_pack(u32 pulse, u32 sample)
{
return (pulse << 10) | (sample << 2);
}
-static int nic_wait(struct ioc3 *ioc3)
+static int nic_wait(u32 __iomem *mcr)
{
- u32 mcr;
+ u32 m;
- do {
- mcr = ioc3_r_mcr();
- } while (!(mcr & 2));
+ do {
+ m = readl(mcr);
+ } while (!(m & 2));
- return mcr & 1;
+ return m & 1;
}
-static int nic_reset(struct ioc3 *ioc3)
+static int nic_reset(u32 __iomem *mcr)
{
- int presence;
+ int presence;
- ioc3_w_mcr(mcr_pack(500, 65));
- presence = nic_wait(ioc3);
+ writel(mcr_pack(500, 65), mcr);
+ presence = nic_wait(mcr);
- ioc3_w_mcr(mcr_pack(0, 500));
- nic_wait(ioc3);
+ writel(mcr_pack(0, 500), mcr);
+ nic_wait(mcr);
- return presence;
+ return presence;
}
-static inline int nic_read_bit(struct ioc3 *ioc3)
+static inline int nic_read_bit(u32 __iomem *mcr)
{
int result;
- ioc3_w_mcr(mcr_pack(6, 13));
- result = nic_wait(ioc3);
- ioc3_w_mcr(mcr_pack(0, 100));
- nic_wait(ioc3);
+ writel(mcr_pack(6, 13), mcr);
+ result = nic_wait(mcr);
+ writel(mcr_pack(0, 100), mcr);
+ nic_wait(mcr);
return result;
}
-static inline void nic_write_bit(struct ioc3 *ioc3, int bit)
+static inline void nic_write_bit(u32 __iomem *mcr, int bit)
{
if (bit)
- ioc3_w_mcr(mcr_pack(6, 110));
+ writel(mcr_pack(6, 110), mcr);
else
- ioc3_w_mcr(mcr_pack(80, 30));
+ writel(mcr_pack(80, 30), mcr);
- nic_wait(ioc3);
+ nic_wait(mcr);
}
-/*
- * Read a byte from an iButton device
+/* Read a byte from an iButton device
*/
-static u32 nic_read_byte(struct ioc3 *ioc3)
+static u32 nic_read_byte(u32 __iomem *mcr)
{
u32 result = 0;
int i;
for (i = 0; i < 8; i++)
- result = (result >> 1) | (nic_read_bit(ioc3) << 7);
+ result = (result >> 1) | (nic_read_bit(mcr) << 7);
return result;
}
-/*
- * Write a byte to an iButton device
+/* Write a byte to an iButton device
*/
-static void nic_write_byte(struct ioc3 *ioc3, int byte)
+static void nic_write_byte(u32 __iomem *mcr, int byte)
{
int i, bit;
@@ -293,26 +254,26 @@ static void nic_write_byte(struct ioc3 *ioc3, int byte)
bit = byte & 1;
byte >>= 1;
- nic_write_bit(ioc3, bit);
+ nic_write_bit(mcr, bit);
}
}
-static u64 nic_find(struct ioc3 *ioc3, int *last)
+static u64 nic_find(u32 __iomem *mcr, int *last)
{
int a, b, index, disc;
u64 address = 0;
- nic_reset(ioc3);
+ nic_reset(mcr);
/* Search ROM. */
- nic_write_byte(ioc3, 0xf0);
+ nic_write_byte(mcr, 0xf0);
/* Algorithm from ``Book of iButton Standards''. */
for (index = 0, disc = 0; index < 64; index++) {
- a = nic_read_bit(ioc3);
- b = nic_read_bit(ioc3);
+ a = nic_read_bit(mcr);
+ b = nic_read_bit(mcr);
if (a && b) {
- printk("NIC search failed (not fatal).\n");
+ pr_warn("NIC search failed (not fatal).\n");
*last = 0;
return 0;
}
@@ -323,16 +284,17 @@ static u64 nic_find(struct ioc3 *ioc3, int *last)
} else if (index > *last) {
address &= ~(1UL << index);
disc = index;
- } else if ((address & (1UL << index)) == 0)
+ } else if ((address & (1UL << index)) == 0) {
disc = index;
- nic_write_bit(ioc3, address & (1UL << index));
+ }
+ nic_write_bit(mcr, address & (1UL << index));
continue;
} else {
if (a)
address |= 1UL << index;
else
address &= ~(1UL << index);
- nic_write_bit(ioc3, a);
+ nic_write_bit(mcr, a);
continue;
}
}
@@ -342,7 +304,7 @@ static u64 nic_find(struct ioc3 *ioc3, int *last)
return address;
}
-static int nic_init(struct ioc3 *ioc3)
+static int nic_init(u32 __iomem *mcr)
{
const char *unknown = "unknown";
const char *type = unknown;
@@ -352,7 +314,8 @@ static int nic_init(struct ioc3 *ioc3)
while (1) {
u64 reg;
- reg = nic_find(ioc3, &save);
+
+ reg = nic_find(mcr, &save);
switch (reg & 0xff) {
case 0x91:
@@ -366,12 +329,12 @@ static int nic_init(struct ioc3 *ioc3)
continue;
}
- nic_reset(ioc3);
+ nic_reset(mcr);
/* Match ROM. */
- nic_write_byte(ioc3, 0x55);
+ nic_write_byte(mcr, 0x55);
for (i = 0; i < 8; i++)
- nic_write_byte(ioc3, (reg >> (i << 3)) & 0xff);
+ nic_write_byte(mcr, (reg >> (i << 3)) & 0xff);
reg >>= 8; /* Shift out type. */
for (i = 0; i < 6; i++) {
@@ -382,52 +345,50 @@ static int nic_init(struct ioc3 *ioc3)
break;
}
- printk("Found %s NIC", type);
+ pr_info("Found %s NIC", type);
if (type != unknown)
- printk (" registration number %pM, CRC %02x", serial, crc);
- printk(".\n");
+ pr_cont(" registration number %pM, CRC %02x", serial, crc);
+ pr_cont(".\n");
return 0;
}
-/*
- * Read the NIC (Number-In-a-Can) device used to store the MAC address on
+/* Read the NIC (Number-In-a-Can) device used to store the MAC address on
* SN0 / SN00 nodeboards and PCI cards.
*/
static void ioc3_get_eaddr_nic(struct ioc3_private *ip)
{
- struct ioc3 *ioc3 = ip->regs;
- u8 nic[14];
+ u32 __iomem *mcr = &ip->all_regs->mcr;
int tries = 2; /* There may be some problem with the battery? */
+ u8 nic[14];
int i;
- ioc3_w_gpcr_s(1 << 21);
+ writel(1 << 21, &ip->all_regs->gpcr_s);
while (tries--) {
- if (!nic_init(ioc3))
+ if (!nic_init(mcr))
break;
udelay(500);
}
if (tries < 0) {
- printk("Failed to read MAC address\n");
+ pr_err("Failed to read MAC address\n");
return;
}
/* Read Memory. */
- nic_write_byte(ioc3, 0xf0);
- nic_write_byte(ioc3, 0x00);
- nic_write_byte(ioc3, 0x00);
+ nic_write_byte(mcr, 0xf0);
+ nic_write_byte(mcr, 0x00);
+ nic_write_byte(mcr, 0x00);
for (i = 13; i >= 0; i--)
- nic[i] = nic_read_byte(ioc3);
+ nic[i] = nic_read_byte(mcr);
for (i = 2; i < 8; i++)
ip->dev->dev_addr[i - 2] = nic[i];
}
-/*
- * Ok, this is hosed by design. It's necessary to know what machine the
+/* Ok, this is hosed by design. It's necessary to know what machine the
* NIC is in in order to know how to read the NIC address. We also have
* to know if it's a PCI card or a NIC in on the node board ...
*/
@@ -435,17 +396,21 @@ static void ioc3_get_eaddr(struct ioc3_private *ip)
{
ioc3_get_eaddr_nic(ip);
- printk("Ethernet address is %pM.\n", ip->dev->dev_addr);
+ pr_info("Ethernet address is %pM.\n", ip->dev->dev_addr);
}
static void __ioc3_set_mac_address(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
- ioc3_w_emar_h((dev->dev_addr[5] << 8) | dev->dev_addr[4]);
- ioc3_w_emar_l((dev->dev_addr[3] << 24) | (dev->dev_addr[2] << 16) |
- (dev->dev_addr[1] << 8) | dev->dev_addr[0]);
+ writel((dev->dev_addr[5] << 8) |
+ dev->dev_addr[4],
+ &ip->regs->emar_h);
+ writel((dev->dev_addr[3] << 24) |
+ (dev->dev_addr[2] << 16) |
+ (dev->dev_addr[1] << 8) |
+ dev->dev_addr[0],
+ &ip->regs->emar_l);
}
static int ioc3_set_mac_address(struct net_device *dev, void *addr)
@@ -462,31 +427,35 @@ static int ioc3_set_mac_address(struct net_device *dev, void *addr)
return 0;
}
-/*
- * Caller must hold the ioc3_lock ever for MII readers. This is also
+/* Caller must hold the ioc3_lock ever for MII readers. This is also
* used to protect the transmitter side but it's low contention.
*/
static int ioc3_mdio_read(struct net_device *dev, int phy, int reg)
{
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
- while (ioc3_r_micr() & MICR_BUSY);
- ioc3_w_micr((phy << MICR_PHYADDR_SHIFT) | reg | MICR_READTRIG);
- while (ioc3_r_micr() & MICR_BUSY);
+ while (readl(®s->micr) & MICR_BUSY)
+ ;
+ writel((phy << MICR_PHYADDR_SHIFT) | reg | MICR_READTRIG,
+ ®s->micr);
+ while (readl(®s->micr) & MICR_BUSY)
+ ;
- return ioc3_r_midr_r() & MIDR_DATA_MASK;
+ return readl(®s->midr_r) & MIDR_DATA_MASK;
}
static void ioc3_mdio_write(struct net_device *dev, int phy, int reg, int data)
{
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
- while (ioc3_r_micr() & MICR_BUSY);
- ioc3_w_midr_w(data);
- ioc3_w_micr((phy << MICR_PHYADDR_SHIFT) | reg);
- while (ioc3_r_micr() & MICR_BUSY);
+ while (readl(®s->micr) & MICR_BUSY)
+ ;
+ writel(data, ®s->midr_w);
+ writel((phy << MICR_PHYADDR_SHIFT) | reg, ®s->micr);
+ while (readl(®s->micr) & MICR_BUSY)
+ ;
}
static int ioc3_mii_init(struct ioc3_private *ip);
@@ -494,23 +463,22 @@ static int ioc3_mii_init(struct ioc3_private *ip);
static struct net_device_stats *ioc3_get_stats(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
- dev->stats.collisions += (ioc3_r_etcdc() & ETCDC_COLLCNT_MASK);
+ dev->stats.collisions += readl(®s->etcdc) & ETCDC_COLLCNT_MASK;
return &dev->stats;
}
-static void ioc3_tcpudp_checksum(struct sk_buff *skb, uint32_t hwsum, int len)
+static void ioc3_tcpudp_checksum(struct sk_buff *skb, u32 hwsum, int len)
{
struct ethhdr *eh = eth_hdr(skb);
- uint32_t csum, ehsum;
unsigned int proto;
- struct iphdr *ih;
- uint16_t *ew;
unsigned char *cp;
+ struct iphdr *ih;
+ u32 csum, ehsum;
+ u16 *ew;
- /*
- * Did hardware handle the checksum at all? The cases we can handle
+ /* Did hardware handle the checksum at all? The cases we can handle
* are:
*
* - TCP and UDP checksums of IPv4 only.
@@ -526,7 +494,7 @@ static void ioc3_tcpudp_checksum(struct sk_buff *skb, uint32_t hwsum, int len)
if (eh->h_proto != htons(ETH_P_IP))
return;
- ih = (struct iphdr *) ((char *)eh + ETH_HLEN);
+ ih = (struct iphdr *)((char *)eh + ETH_HLEN);
if (ip_is_fragment(ih))
return;
@@ -537,12 +505,12 @@ static void ioc3_tcpudp_checksum(struct sk_buff *skb, uint32_t hwsum, int len)
/* Same as tx - compute csum of pseudo header */
csum = hwsum +
(ih->tot_len - (ih->ihl << 2)) +
- htons((uint16_t)ih->protocol) +
+ htons((u16)ih->protocol) +
(ih->saddr >> 16) + (ih->saddr & 0xffff) +
(ih->daddr >> 16) + (ih->daddr & 0xffff);
/* Sum up ethernet dest addr, src addr and protocol */
- ew = (uint16_t *) eh;
+ ew = (u16 *)eh;
ehsum = ew[0] + ew[1] + ew[2] + ew[3] + ew[4] + ew[5] + ew[6];
ehsum = (ehsum & 0xffff) + (ehsum >> 16);
@@ -551,14 +519,15 @@ static void ioc3_tcpudp_checksum(struct sk_buff *skb, uint32_t hwsum, int len)
csum += 0xffff ^ ehsum;
/* In the next step we also subtract the 1's complement
- checksum of the trailing ethernet CRC. */
+ * checksum of the trailing ethernet CRC.
+ */
cp = (char *)eh + len; /* points at trailing CRC */
if (len & 1) {
- csum += 0xffff ^ (uint16_t) ((cp[1] << 8) | cp[0]);
- csum += 0xffff ^ (uint16_t) ((cp[3] << 8) | cp[2]);
+ csum += 0xffff ^ (u16)((cp[1] << 8) | cp[0]);
+ csum += 0xffff ^ (u16)((cp[3] << 8) | cp[2]);
} else {
- csum += 0xffff ^ (uint16_t) ((cp[0] << 8) | cp[1]);
- csum += 0xffff ^ (uint16_t) ((cp[2] << 8) | cp[3]);
+ csum += 0xffff ^ (u16)((cp[0] << 8) | cp[1]);
+ csum += 0xffff ^ (u16)((cp[2] << 8) | cp[3]);
}
csum = (csum & 0xffff) + (csum >> 16);
@@ -572,10 +541,10 @@ static inline void ioc3_rx(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
struct sk_buff *skb, *new_skb;
- struct ioc3 *ioc3 = ip->regs;
int rx_entry, n_entry, len;
struct ioc3_erxbuf *rxb;
unsigned long *rxr;
+ dma_addr_t d;
u32 w0, err;
rxr = ip->rxr; /* Ring base */
@@ -583,64 +552,67 @@ static inline void ioc3_rx(struct net_device *dev)
n_entry = ip->rx_pi;
skb = ip->rx_skbs[rx_entry];
- rxb = (struct ioc3_erxbuf *) (skb->data - RX_OFFSET);
+ rxb = (struct ioc3_erxbuf *)(skb->data - RX_OFFSET);
w0 = be32_to_cpu(rxb->w0);
while (w0 & ERXBUF_V) {
err = be32_to_cpu(rxb->err); /* It's valid ... */
if (err & ERXBUF_GOODPKT) {
len = ((w0 >> ERXBUF_BYTECNT_SHIFT) & 0x7ff) - 4;
- skb_trim(skb, len);
+ skb_put(skb, len);
skb->protocol = eth_type_trans(skb, dev);
- new_skb = ioc3_alloc_skb(RX_BUF_ALLOC_SIZE, GFP_ATOMIC);
- if (!new_skb) {
+ if (ioc3_alloc_skb(ip, &new_skb, &rxb, &d)) {
/* Ouch, drop packet and just recycle packet
- to keep the ring filled. */
+ * to keep the ring filled.
+ */
dev->stats.rx_dropped++;
new_skb = skb;
+ d = rxr[rx_entry];
goto next;
}
if (likely(dev->features & NETIF_F_RXCSUM))
ioc3_tcpudp_checksum(skb,
- w0 & ERXBUF_IPCKSUM_MASK, len);
+ w0 & ERXBUF_IPCKSUM_MASK,
+ len);
+
+ dma_unmap_single(ip->dma_dev, rxr[rx_entry],
+ RX_BUF_SIZE, DMA_FROM_DEVICE);
netif_rx(skb);
ip->rx_skbs[rx_entry] = NULL; /* Poison */
- /* Because we reserve afterwards. */
- skb_put(new_skb, (1664 + RX_OFFSET));
- rxb = (struct ioc3_erxbuf *) new_skb->data;
- skb_reserve(new_skb, RX_OFFSET);
-
dev->stats.rx_packets++; /* Statistics */
dev->stats.rx_bytes += len;
} else {
/* The frame is invalid and the skb never
- reached the network layer so we can just
- recycle it. */
+ * reached the network layer so we can just
+ * recycle it.
+ */
new_skb = skb;
+ d = rxr[rx_entry];
dev->stats.rx_errors++;
}
if (err & ERXBUF_CRCERR) /* Statistics */
dev->stats.rx_crc_errors++;
if (err & ERXBUF_FRAMERR)
dev->stats.rx_frame_errors++;
+
next:
ip->rx_skbs[n_entry] = new_skb;
- rxr[n_entry] = cpu_to_be64(ioc3_map(rxb, 1));
+ rxr[n_entry] = cpu_to_be64(ioc3_map(d, PCI64_ATTR_BAR));
rxb->w0 = 0; /* Clear valid flag */
- n_entry = (n_entry + 1) & 511; /* Update erpir */
+ n_entry = (n_entry + 1) & RX_RING_MASK; /* Update erpir */
/* Now go on to the next ring entry. */
- rx_entry = (rx_entry + 1) & 511;
+ rx_entry = (rx_entry + 1) & RX_RING_MASK;
skb = ip->rx_skbs[rx_entry];
- rxb = (struct ioc3_erxbuf *) (skb->data - RX_OFFSET);
+ rxb = (struct ioc3_erxbuf *)(skb->data - RX_OFFSET);
w0 = be32_to_cpu(rxb->w0);
}
- ioc3_w_erpir((n_entry << 3) | ERPIR_ARM);
+ writel((n_entry << 3) | ERPIR_ARM, &ip->regs->erpir);
ip->rx_pi = n_entry;
ip->rx_ci = rx_entry;
}
@@ -648,16 +620,16 @@ static inline void ioc3_rx(struct net_device *dev)
static inline void ioc3_tx(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
+ struct ioc3_ethregs *regs = ip->regs;
unsigned long packets, bytes;
- struct ioc3 *ioc3 = ip->regs;
int tx_entry, o_entry;
struct sk_buff *skb;
u32 etcir;
spin_lock(&ip->ioc3_lock);
- etcir = ioc3_r_etcir();
+ etcir = readl(®s->etcir);
- tx_entry = (etcir >> 7) & 127;
+ tx_entry = (etcir >> 7) & TX_RING_MASK;
o_entry = ip->tx_ci;
packets = 0;
bytes = 0;
@@ -669,25 +641,24 @@ static inline void ioc3_tx(struct net_device *dev)
dev_consume_skb_irq(skb);
ip->tx_skbs[o_entry] = NULL;
- o_entry = (o_entry + 1) & 127; /* Next */
+ o_entry = (o_entry + 1) & TX_RING_MASK; /* Next */
- etcir = ioc3_r_etcir(); /* More pkts sent? */
- tx_entry = (etcir >> 7) & 127;
+ etcir = readl(®s->etcir); /* More pkts sent? */
+ tx_entry = (etcir >> 7) & TX_RING_MASK;
}
dev->stats.tx_packets += packets;
dev->stats.tx_bytes += bytes;
ip->txqlen -= packets;
- if (ip->txqlen < 128)
+ if (netif_queue_stopped(dev) && ip->txqlen < TX_RING_ENTRIES)
netif_wake_queue(dev);
ip->tx_ci = o_entry;
spin_unlock(&ip->ioc3_lock);
}
-/*
- * Deal with fatal IOC3 errors. This condition might be caused by a hard or
+/* Deal with fatal IOC3 errors. This condition might be caused by a hard or
* software problems, so we should try to recover
* more gracefully if this ever happens. In theory we might be flooded
* with such error interrupts if something really goes wrong, so we might
@@ -696,25 +667,33 @@ static inline void ioc3_tx(struct net_device *dev)
static void ioc3_error(struct net_device *dev, u32 eisr)
{
struct ioc3_private *ip = netdev_priv(dev);
- unsigned char *iface = dev->name;
spin_lock(&ip->ioc3_lock);
if (eisr & EISR_RXOFLO)
- printk(KERN_ERR "%s: RX overflow.\n", iface);
+ net_err_ratelimited("%s: RX overflow.\n", dev->name);
if (eisr & EISR_RXBUFOFLO)
- printk(KERN_ERR "%s: RX buffer overflow.\n", iface);
+ net_err_ratelimited("%s: RX buffer overflow.\n", dev->name);
if (eisr & EISR_RXMEMERR)
- printk(KERN_ERR "%s: RX PCI error.\n", iface);
+ net_err_ratelimited("%s: RX PCI error.\n", dev->name);
if (eisr & EISR_RXPARERR)
- printk(KERN_ERR "%s: RX SSRAM parity error.\n", iface);
+ net_err_ratelimited("%s: RX SSRAM parity error.\n", dev->name);
if (eisr & EISR_TXBUFUFLO)
- printk(KERN_ERR "%s: TX buffer underflow.\n", iface);
+ net_err_ratelimited("%s: TX buffer underflow.\n", dev->name);
if (eisr & EISR_TXMEMERR)
- printk(KERN_ERR "%s: TX PCI error.\n", iface);
+ net_err_ratelimited("%s: TX PCI error.\n", dev->name);
ioc3_stop(ip);
+ ioc3_free_rx_bufs(ip);
+ ioc3_clean_tx_ring(ip);
+
ioc3_init(dev);
+ if (ioc3_alloc_rx_bufs(dev)) {
+ netdev_err(dev, "%s: rx buffer allocation failed\n", __func__);
+ spin_unlock(&ip->ioc3_lock);
+ return;
+ }
+ ioc3_start(ip);
ioc3_mii_init(ip);
netif_wake_queue(dev);
@@ -723,45 +702,45 @@ static void ioc3_error(struct net_device *dev, u32 eisr)
}
/* The interrupt handler does all of the Rx thread work and cleans up
- after the Tx thread. */
-static irqreturn_t ioc3_interrupt(int irq, void *_dev)
+ * after the Tx thread.
+ */
+static irqreturn_t ioc3_interrupt(int irq, void *dev_id)
{
- struct net_device *dev = (struct net_device *)_dev;
- struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
- const u32 enabled = EISR_RXTIMERINT | EISR_RXOFLO | EISR_RXBUFOFLO |
- EISR_RXMEMERR | EISR_RXPARERR | EISR_TXBUFUFLO |
- EISR_TXEXPLICIT | EISR_TXMEMERR;
+ struct ioc3_private *ip = netdev_priv(dev_id);
+ struct ioc3_ethregs *regs = ip->regs;
u32 eisr;
- eisr = ioc3_r_eisr() & enabled;
-
- ioc3_w_eisr(eisr);
- (void) ioc3_r_eisr(); /* Flush */
+ eisr = readl(®s->eisr);
+ writel(eisr, ®s->eisr);
+ readl(®s->eisr); /* Flush */
if (eisr & (EISR_RXOFLO | EISR_RXBUFOFLO | EISR_RXMEMERR |
- EISR_RXPARERR | EISR_TXBUFUFLO | EISR_TXMEMERR))
- ioc3_error(dev, eisr);
+ EISR_RXPARERR | EISR_TXBUFUFLO | EISR_TXMEMERR))
+ ioc3_error(dev_id, eisr);
if (eisr & EISR_RXTIMERINT)
- ioc3_rx(dev);
+ ioc3_rx(dev_id);
if (eisr & EISR_TXEXPLICIT)
- ioc3_tx(dev);
+ ioc3_tx(dev_id);
return IRQ_HANDLED;
}
static inline void ioc3_setup_duplex(struct ioc3_private *ip)
{
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
+
+ spin_lock_irq(&ip->ioc3_lock);
if (ip->mii.full_duplex) {
- ioc3_w_etcsr(ETCSR_FD);
+ writel(ETCSR_FD, ®s->etcsr);
ip->emcr |= EMCR_DUPLEX;
} else {
- ioc3_w_etcsr(ETCSR_HD);
+ writel(ETCSR_HD, ®s->etcsr);
ip->emcr &= ~EMCR_DUPLEX;
}
- ioc3_w_emcr(ip->emcr);
+ writel(ip->emcr, ®s->emcr);
+
+ spin_unlock_irq(&ip->ioc3_lock);
}
static void ioc3_timer(struct timer_list *t)
@@ -772,12 +751,11 @@ static void ioc3_timer(struct timer_list *t)
mii_check_media(&ip->mii, 1, 0);
ioc3_setup_duplex(ip);
- ip->ioc3_timer.expires = jiffies + ((12 * HZ)/10); /* 1.2s */
+ ip->ioc3_timer.expires = jiffies + ((12 * HZ) / 10); /* 1.2s */
add_timer(&ip->ioc3_timer);
}
-/*
- * Try to find a PHY. There is no apparent relation between the MII addresses
+/* Try to find a PHY. There is no apparent relation between the MII addresses
* in the SGI documentation and what we find in reality, so we simply probe
* for the PHY. It seems IOC3 PHYs usually live on address 31. One of my
* onboard IOC3s has the special oddity that probing doesn't seem to find it
@@ -786,8 +764,8 @@ static void ioc3_timer(struct timer_list *t)
*/
static int ioc3_mii_init(struct ioc3_private *ip)
{
- int i, found = 0, res = 0;
int ioc3_phy_workaround = 1;
+ int i, found = 0, res = 0;
u16 word;
for (i = 0; i < 32; i++) {
@@ -800,9 +778,9 @@ static int ioc3_mii_init(struct ioc3_private *ip)
}
if (!found) {
- if (ioc3_phy_workaround)
+ if (ioc3_phy_workaround) {
i = 31;
- else {
+ } else {
ip->mii.phy_id = -1;
res = -ENODEV;
goto out;
@@ -817,27 +795,27 @@ static int ioc3_mii_init(struct ioc3_private *ip)
static void ioc3_mii_start(struct ioc3_private *ip)
{
- ip->ioc3_timer.expires = jiffies + (12 * HZ)/10; /* 1.2 sec. */
+ ip->ioc3_timer.expires = jiffies + (12 * HZ) / 10; /* 1.2 sec. */
add_timer(&ip->ioc3_timer);
}
-static inline void ioc3_clean_rx_ring(struct ioc3_private *ip)
+static inline void ioc3_tx_unmap(struct ioc3_private *ip, int entry)
{
- struct sk_buff *skb;
- int i;
+ struct ioc3_etxd *desc;
+ u32 cmd, bufcnt, len;
- for (i = ip->rx_ci; i & 15; i++) {
- ip->rx_skbs[ip->rx_pi] = ip->rx_skbs[ip->rx_ci];
- ip->rxr[ip->rx_pi++] = ip->rxr[ip->rx_ci++];
+ desc = &ip->txr[entry];
+ cmd = be32_to_cpu(desc->cmd);
+ bufcnt = be32_to_cpu(desc->bufcnt);
+ if (cmd & ETXD_B1V) {
+ len = (bufcnt & ETXD_B1CNT_MASK) >> ETXD_B1CNT_SHIFT;
+ dma_unmap_single(ip->dma_dev, be64_to_cpu(desc->p1),
+ len, DMA_TO_DEVICE);
}
- ip->rx_pi &= 511;
- ip->rx_ci &= 511;
-
- for (i = ip->rx_ci; i != ip->rx_pi; i = (i+1) & 511) {
- struct ioc3_erxbuf *rxb;
- skb = ip->rx_skbs[i];
- rxb = (struct ioc3_erxbuf *) (skb->data - RX_OFFSET);
- rxb->w0 = 0;
+ if (cmd & ETXD_B2V) {
+ len = (bufcnt & ETXD_B2CNT_MASK) >> ETXD_B2CNT_SHIFT;
+ dma_unmap_single(ip->dma_dev, be64_to_cpu(desc->p2),
+ len, DMA_TO_DEVICE);
}
}
@@ -846,9 +824,10 @@ static inline void ioc3_clean_tx_ring(struct ioc3_private *ip)
struct sk_buff *skb;
int i;
- for (i=0; i < 128; i++) {
+ for (i = 0; i < TX_RING_ENTRIES; i++) {
skb = ip->tx_skbs[i];
if (skb) {
+ ioc3_tx_unmap(ip, i);
ip->tx_skbs[i] = NULL;
dev_kfree_skb_any(skb);
}
@@ -858,179 +837,137 @@ static inline void ioc3_clean_tx_ring(struct ioc3_private *ip)
ip->tx_ci = 0;
}
-static void ioc3_free_rings(struct ioc3_private *ip)
+static void ioc3_free_rx_bufs(struct ioc3_private *ip)
{
- struct sk_buff *skb;
int rx_entry, n_entry;
+ struct sk_buff *skb;
- if (ip->txr) {
- ioc3_clean_tx_ring(ip);
- free_pages((unsigned long)ip->txr, 2);
- ip->txr = NULL;
- }
+ n_entry = ip->rx_ci;
+ rx_entry = ip->rx_pi;
- if (ip->rxr) {
- n_entry = ip->rx_ci;
- rx_entry = ip->rx_pi;
-
- while (n_entry != rx_entry) {
- skb = ip->rx_skbs[n_entry];
- if (skb)
- dev_kfree_skb_any(skb);
-
- n_entry = (n_entry + 1) & 511;
+ while (n_entry != rx_entry) {
+ skb = ip->rx_skbs[n_entry];
+ if (skb) {
+ dma_unmap_single(ip->dma_dev,
+ be64_to_cpu(ip->rxr[n_entry]),
+ RX_BUF_SIZE, DMA_FROM_DEVICE);
+ dev_kfree_skb_any(skb);
}
- free_page((unsigned long)ip->rxr);
- ip->rxr = NULL;
+ n_entry = (n_entry + 1) & RX_RING_MASK;
}
}
-static void ioc3_alloc_rings(struct net_device *dev)
+static int ioc3_alloc_rx_bufs(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
struct ioc3_erxbuf *rxb;
- unsigned long *rxr;
+ dma_addr_t d;
int i;
- if (ip->rxr == NULL) {
- /* Allocate and initialize rx ring. 4kb = 512 entries */
- ip->rxr = (unsigned long *) get_zeroed_page(GFP_ATOMIC);
- rxr = ip->rxr;
- if (!rxr)
- printk("ioc3_alloc_rings(): get_zeroed_page() failed!\n");
+ /* Now the rx buffers. The RX ring may be larger but
+ * we only allocate 16 buffers for now. Need to tune
+ * this for performance and memory later.
+ */
+ for (i = 0; i < RX_BUFFS; i++) {
+ if (ioc3_alloc_skb(ip, &ip->rx_skbs[i], &rxb, &d))
+ return -ENOMEM;
- /* Now the rx buffers. The RX ring may be larger but
- we only allocate 16 buffers for now. Need to tune
- this for performance and memory later. */
- for (i = 0; i < RX_BUFFS; i++) {
- struct sk_buff *skb;
-
- skb = ioc3_alloc_skb(RX_BUF_ALLOC_SIZE, GFP_ATOMIC);
- if (!skb) {
- show_free_areas(0, NULL);
- continue;
- }
-
- ip->rx_skbs[i] = skb;
-
- /* Because we reserve afterwards. */
- skb_put(skb, (1664 + RX_OFFSET));
- rxb = (struct ioc3_erxbuf *) skb->data;
- rxr[i] = cpu_to_be64(ioc3_map(rxb, 1));
- skb_reserve(skb, RX_OFFSET);
- }
- ip->rx_ci = 0;
- ip->rx_pi = RX_BUFFS;
+ rxb->w0 = 0; /* Clear valid flag */
+ ip->rxr[i] = cpu_to_be64(ioc3_map(d, PCI64_ATTR_BAR));
}
+ ip->rx_ci = 0;
+ ip->rx_pi = RX_BUFFS;
- if (ip->txr == NULL) {
- /* Allocate and initialize tx rings. 16kb = 128 bufs. */
- ip->txr = (struct ioc3_etxd *)__get_free_pages(GFP_KERNEL, 2);
- if (!ip->txr)
- printk("ioc3_alloc_rings(): __get_free_pages() failed!\n");
- ip->tx_pi = 0;
- ip->tx_ci = 0;
- }
-}
-
-static void ioc3_init_rings(struct net_device *dev)
-{
- struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
- unsigned long ring;
-
- ioc3_free_rings(ip);
- ioc3_alloc_rings(dev);
-
- ioc3_clean_rx_ring(ip);
- ioc3_clean_tx_ring(ip);
-
- /* Now the rx ring base, consume & produce registers. */
- ring = ioc3_map(ip->rxr, 0);
- ioc3_w_erbr_h(ring >> 32);
- ioc3_w_erbr_l(ring & 0xffffffff);
- ioc3_w_ercir(ip->rx_ci << 3);
- ioc3_w_erpir((ip->rx_pi << 3) | ERPIR_ARM);
-
- ring = ioc3_map(ip->txr, 0);
-
- ip->txqlen = 0; /* nothing queued */
-
- /* Now the tx ring base, consume & produce registers. */
- ioc3_w_etbr_h(ring >> 32);
- ioc3_w_etbr_l(ring & 0xffffffff);
- ioc3_w_etpir(ip->tx_pi << 7);
- ioc3_w_etcir(ip->tx_ci << 7);
- (void) ioc3_r_etcir(); /* Flush */
+ return 0;
}
static inline void ioc3_ssram_disc(struct ioc3_private *ip)
{
- struct ioc3 *ioc3 = ip->regs;
- volatile u32 *ssram0 = &ioc3->ssram[0x0000];
- volatile u32 *ssram1 = &ioc3->ssram[0x4000];
- unsigned int pattern = 0x5555;
+ struct ioc3_ethregs *regs = ip->regs;
+ u32 *ssram0 = &ip->ssram[0x0000];
+ u32 *ssram1 = &ip->ssram[0x4000];
+ u32 pattern = 0x5555;
/* Assume the larger size SSRAM and enable parity checking */
- ioc3_w_emcr(ioc3_r_emcr() | (EMCR_BUFSIZ | EMCR_RAMPAR));
+ writel(readl(®s->emcr) | (EMCR_BUFSIZ | EMCR_RAMPAR), ®s->emcr);
+ readl(®s->emcr); /* Flush */
- *ssram0 = pattern;
- *ssram1 = ~pattern & IOC3_SSRAM_DM;
+ writel(pattern, ssram0);
+ writel(~pattern & IOC3_SSRAM_DM, ssram1);
- if ((*ssram0 & IOC3_SSRAM_DM) != pattern ||
- (*ssram1 & IOC3_SSRAM_DM) != (~pattern & IOC3_SSRAM_DM)) {
+ if ((readl(ssram0) & IOC3_SSRAM_DM) != pattern ||
+ (readl(ssram1) & IOC3_SSRAM_DM) != (~pattern & IOC3_SSRAM_DM)) {
/* set ssram size to 64 KB */
- ip->emcr = EMCR_RAMPAR;
- ioc3_w_emcr(ioc3_r_emcr() & ~EMCR_BUFSIZ);
- } else
- ip->emcr = EMCR_BUFSIZ | EMCR_RAMPAR;
+ ip->emcr |= EMCR_RAMPAR;
+ writel(readl(®s->emcr) & ~EMCR_BUFSIZ, ®s->emcr);
+ } else {
+ ip->emcr |= EMCR_BUFSIZ | EMCR_RAMPAR;
+ }
}
static void ioc3_init(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
del_timer_sync(&ip->ioc3_timer); /* Kill if running */
- ioc3_w_emcr(EMCR_RST); /* Reset */
- (void) ioc3_r_emcr(); /* Flush WB */
+ writel(EMCR_RST, ®s->emcr); /* Reset */
+ readl(®s->emcr); /* Flush WB */
udelay(4); /* Give it time ... */
- ioc3_w_emcr(0);
- (void) ioc3_r_emcr();
+ writel(0, ®s->emcr);
+ readl(®s->emcr);
/* Misc registers */
-#ifdef CONFIG_SGI_IP27
- ioc3_w_erbar(PCI64_ATTR_BAR >> 32); /* Barrier on last store */
-#else
- ioc3_w_erbar(0); /* Let PCI API get it right */
-#endif
- (void) ioc3_r_etcdc(); /* Clear on read */
- ioc3_w_ercsr(15); /* RX low watermark */
- ioc3_w_ertr(0); /* Interrupt immediately */
+ writel(ERBAR_VAL, ®s->erbar);
+ readl(®s->etcdc); /* Clear on read */
+ writel(15, ®s->ercsr); /* RX low watermark */
+ writel(0, ®s->ertr); /* Interrupt immediately */
__ioc3_set_mac_address(dev);
- ioc3_w_ehar_h(ip->ehar_h);
- ioc3_w_ehar_l(ip->ehar_l);
- ioc3_w_ersr(42); /* XXX should be random */
+ writel(ip->ehar_h, ®s->ehar_h);
+ writel(ip->ehar_l, ®s->ehar_l);
+ writel(42, ®s->ersr); /* XXX should be random */
+}
- ioc3_init_rings(dev);
+static void ioc3_start(struct ioc3_private *ip)
+{
+ struct ioc3_ethregs *regs = ip->regs;
+ unsigned long ring;
+
+ /* Now the rx ring base, consume & produce registers. */
+ ring = ioc3_map(ip->rxr_dma, PCI64_ATTR_PREC);
+ writel(ring >> 32, ®s->erbr_h);
+ writel(ring & 0xffffffff, ®s->erbr_l);
+ writel(ip->rx_ci << 3, ®s->ercir);
+ writel((ip->rx_pi << 3) | ERPIR_ARM, ®s->erpir);
+
+ ring = ioc3_map(ip->txr_dma, PCI64_ATTR_PREC);
+
+ ip->txqlen = 0; /* nothing queued */
+
+ /* Now the tx ring base, consume & produce registers. */
+ writel(ring >> 32, ®s->etbr_h);
+ writel(ring & 0xffffffff, ®s->etbr_l);
+ writel(ip->tx_pi << 7, ®s->etpir);
+ writel(ip->tx_ci << 7, ®s->etcir);
+ readl(®s->etcir); /* Flush */
ip->emcr |= ((RX_OFFSET / 2) << EMCR_RXOFF_SHIFT) | EMCR_TXDMAEN |
- EMCR_TXEN | EMCR_RXDMAEN | EMCR_RXEN | EMCR_PADEN;
- ioc3_w_emcr(ip->emcr);
- ioc3_w_eier(EISR_RXTIMERINT | EISR_RXOFLO | EISR_RXBUFOFLO |
- EISR_RXMEMERR | EISR_RXPARERR | EISR_TXBUFUFLO |
- EISR_TXEXPLICIT | EISR_TXMEMERR);
- (void) ioc3_r_eier();
+ EMCR_TXEN | EMCR_RXDMAEN | EMCR_RXEN | EMCR_PADEN;
+ writel(ip->emcr, ®s->emcr);
+ writel(EISR_RXTIMERINT | EISR_RXOFLO | EISR_RXBUFOFLO |
+ EISR_RXMEMERR | EISR_RXPARERR | EISR_TXBUFUFLO |
+ EISR_TXEXPLICIT | EISR_TXMEMERR, ®s->eier);
+ readl(®s->eier);
}
static inline void ioc3_stop(struct ioc3_private *ip)
{
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
- ioc3_w_emcr(0); /* Shutup */
- ioc3_w_eier(0); /* Disable interrupts */
- (void) ioc3_r_eier(); /* Flush */
+ writel(0, ®s->emcr); /* Shutup */
+ writel(0, ®s->eier); /* Disable interrupts */
+ readl(®s->eier); /* Flush */
}
static int ioc3_open(struct net_device *dev)
@@ -1038,14 +975,20 @@ static int ioc3_open(struct net_device *dev)
struct ioc3_private *ip = netdev_priv(dev);
if (request_irq(dev->irq, ioc3_interrupt, IRQF_SHARED, ioc3_str, dev)) {
- printk(KERN_ERR "%s: Can't get irq %d\n", dev->name, dev->irq);
+ netdev_err(dev, "Can't get irq %d\n", dev->irq);
return -EAGAIN;
}
ip->ehar_h = 0;
ip->ehar_l = 0;
+
ioc3_init(dev);
+ if (ioc3_alloc_rx_bufs(dev)) {
+ netdev_err(dev, "%s: rx buffer allocation failed\n", __func__);
+ return -ENOMEM;
+ }
+ ioc3_start(ip);
ioc3_mii_start(ip);
netif_start_queue(dev);
@@ -1063,12 +1006,13 @@ static int ioc3_close(struct net_device *dev)
ioc3_stop(ip);
free_irq(dev->irq, dev);
- ioc3_free_rings(ip);
+ ioc3_free_rx_bufs(ip);
+ ioc3_clean_tx_ring(ip);
+
return 0;
}
-/*
- * MENET cards have four IOC3 chips, which are attached to two sets of
+/* MENET cards have four IOC3 chips, which are attached to two sets of
* PCI slot resources each: the primary connections are on slots
* 0..3 and the secondaries are on 4..7
*
@@ -1085,7 +1029,7 @@ static int ioc3_adjacent_is_ioc3(struct pci_dev *pdev, int slot)
if (dev) {
if (dev->vendor == PCI_VENDOR_ID_SGI &&
- dev->device == PCI_DEVICE_ID_SGI_IOC3)
+ dev->device == PCI_DEVICE_ID_SGI_IOC3)
ret = 1;
pci_dev_put(dev);
}
@@ -1095,15 +1039,14 @@ static int ioc3_adjacent_is_ioc3(struct pci_dev *pdev, int slot)
static int ioc3_is_menet(struct pci_dev *pdev)
{
- return pdev->bus->parent == NULL &&
+ return !pdev->bus->parent &&
ioc3_adjacent_is_ioc3(pdev, 0) &&
ioc3_adjacent_is_ioc3(pdev, 1) &&
ioc3_adjacent_is_ioc3(pdev, 2);
}
#ifdef CONFIG_SERIAL_8250
-/*
- * Note about serial ports and consoles:
+/* Note about serial ports and consoles:
* For console output, everyone uses the IOC3 UARTA (offset 0x178)
* connected to the master node (look in ip27_setup_console() and
* ip27prom_console_write()).
@@ -1140,31 +1083,32 @@ static void ioc3_8250_register(struct ioc3_uartregs __iomem *uart)
#define COSMISC_CONSTANT 6
struct uart_8250_port port = {
- .port = {
+ .port = {
.irq = 0,
.flags = UPF_SKIP_TEST | UPF_BOOT_AUTOCONF,
.iotype = UPIO_MEM,
.regshift = 0,
.uartclk = (22000000 << 1) / COSMISC_CONSTANT,
- .membase = (unsigned char __iomem *) uart,
- .mapbase = (unsigned long) uart,
- }
+ .membase = (unsigned char __iomem *)uart,
+ .mapbase = (unsigned long)uart,
+ }
};
unsigned char lcr;
- lcr = uart->iu_lcr;
- uart->iu_lcr = lcr | UART_LCR_DLAB;
- uart->iu_scr = COSMISC_CONSTANT,
- uart->iu_lcr = lcr;
- uart->iu_lcr;
+ lcr = readb(&uart->iu_lcr);
+ writeb(lcr | UART_LCR_DLAB, &uart->iu_lcr);
+ writeb(COSMISC_CONSTANT, &uart->iu_scr);
+ writeb(lcr, &uart->iu_lcr);
+ readb(&uart->iu_lcr);
serial8250_register_8250_port(&port);
}
static void ioc3_serial_probe(struct pci_dev *pdev, struct ioc3 *ioc3)
{
- /*
- * We need to recognice and treat the fourth MENET serial as it
+ u32 sio_iec;
+
+ /* We need to recognice and treat the fourth MENET serial as it
* does not have an SuperIO chip attached to it, therefore attempting
* to access it will result in bus errors. We call something an
* MENET if PCI slot 0, 1, 2 and 3 of a master PCI bus all have an IOC3
@@ -1175,33 +1119,34 @@ static void ioc3_serial_probe(struct pci_dev *pdev, struct ioc3 *ioc3)
if (ioc3_is_menet(pdev) && PCI_SLOT(pdev->devfn) == 3)
return;
- /*
- * Switch IOC3 to PIO mode. It probably already was but let's be
+ /* Switch IOC3 to PIO mode. It probably already was but let's be
* paranoid
*/
- ioc3->gpcr_s = GPCR_UARTA_MODESEL | GPCR_UARTB_MODESEL;
- ioc3->gpcr_s;
- ioc3->gppr_6 = 0;
- ioc3->gppr_6;
- ioc3->gppr_7 = 0;
- ioc3->gppr_7;
- ioc3->sscr_a = ioc3->sscr_a & ~SSCR_DMA_EN;
- ioc3->sscr_a;
- ioc3->sscr_b = ioc3->sscr_b & ~SSCR_DMA_EN;
- ioc3->sscr_b;
+ writel(GPCR_UARTA_MODESEL | GPCR_UARTB_MODESEL, &ioc3->gpcr_s);
+ readl(&ioc3->gpcr_s);
+ writel(0, &ioc3->gppr[6]);
+ readl(&ioc3->gppr[6]);
+ writel(0, &ioc3->gppr[7]);
+ readl(&ioc3->gppr[7]);
+ writel(readl(&ioc3->port_a.sscr) & ~SSCR_DMA_EN, &ioc3->port_a.sscr);
+ readl(&ioc3->port_a.sscr);
+ writel(readl(&ioc3->port_b.sscr) & ~SSCR_DMA_EN, &ioc3->port_b.sscr);
+ readl(&ioc3->port_b.sscr);
/* Disable all SA/B interrupts except for SA/B_INT in SIO_IEC. */
- ioc3->sio_iec &= ~ (SIO_IR_SA_TX_MT | SIO_IR_SA_RX_FULL |
- SIO_IR_SA_RX_HIGH | SIO_IR_SA_RX_TIMER |
- SIO_IR_SA_DELTA_DCD | SIO_IR_SA_DELTA_CTS |
- SIO_IR_SA_TX_EXPLICIT | SIO_IR_SA_MEMERR);
- ioc3->sio_iec |= SIO_IR_SA_INT;
- ioc3->sscr_a = 0;
- ioc3->sio_iec &= ~ (SIO_IR_SB_TX_MT | SIO_IR_SB_RX_FULL |
- SIO_IR_SB_RX_HIGH | SIO_IR_SB_RX_TIMER |
- SIO_IR_SB_DELTA_DCD | SIO_IR_SB_DELTA_CTS |
- SIO_IR_SB_TX_EXPLICIT | SIO_IR_SB_MEMERR);
- ioc3->sio_iec |= SIO_IR_SB_INT;
- ioc3->sscr_b = 0;
+ sio_iec = readl(&ioc3->sio_iec);
+ sio_iec &= ~(SIO_IR_SA_TX_MT | SIO_IR_SA_RX_FULL |
+ SIO_IR_SA_RX_HIGH | SIO_IR_SA_RX_TIMER |
+ SIO_IR_SA_DELTA_DCD | SIO_IR_SA_DELTA_CTS |
+ SIO_IR_SA_TX_EXPLICIT | SIO_IR_SA_MEMERR);
+ sio_iec |= SIO_IR_SA_INT;
+ sio_iec &= ~(SIO_IR_SB_TX_MT | SIO_IR_SB_RX_FULL |
+ SIO_IR_SB_RX_HIGH | SIO_IR_SB_RX_TIMER |
+ SIO_IR_SB_DELTA_DCD | SIO_IR_SB_DELTA_CTS |
+ SIO_IR_SB_TX_EXPLICIT | SIO_IR_SB_MEMERR);
+ sio_iec |= SIO_IR_SB_INT;
+ writel(sio_iec, &ioc3->sio_iec);
+ writel(0, &ioc3->port_a.sscr);
+ writel(0, &ioc3->port_b.sscr);
ioc3_8250_register(&ioc3->sregs.uarta);
ioc3_8250_register(&ioc3->sregs.uartb);
@@ -1236,15 +1181,15 @@ static int ioc3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
pci_using_dac = 1;
err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
if (err < 0) {
- printk(KERN_ERR "%s: Unable to obtain 64 bit DMA "
- "for consistent allocations\n", pci_name(pdev));
+ pr_err("%s: Unable to obtain 64 bit DMA for consistent allocations\n",
+ pci_name(pdev));
goto out;
}
} else {
err = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
if (err) {
- printk(KERN_ERR "%s: No usable DMA configuration, "
- "aborting.\n", pci_name(pdev));
+ pr_err("%s: No usable DMA configuration, aborting.\n",
+ pci_name(pdev));
goto out;
}
pci_using_dac = 0;
@@ -1270,19 +1215,22 @@ static int ioc3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
ip = netdev_priv(dev);
ip->dev = dev;
+ ip->dma_dev = &pdev->dev;
dev->irq = pdev->irq;
ioc3_base = pci_resource_start(pdev, 0);
ioc3_size = pci_resource_len(pdev, 0);
- ioc3 = (struct ioc3 *) ioremap(ioc3_base, ioc3_size);
+ ioc3 = (struct ioc3 *)ioremap(ioc3_base, ioc3_size);
if (!ioc3) {
- printk(KERN_CRIT "ioc3eth(%s): ioremap failed, goodbye.\n",
+ pr_err("ioc3eth(%s): ioremap failed, goodbye.\n",
pci_name(pdev));
err = -ENOMEM;
goto out_res;
}
- ip->regs = ioc3;
+ ip->regs = &ioc3->eth;
+ ip->ssram = ioc3->ssram;
+ ip->all_regs = ioc3;
#ifdef CONFIG_SERIAL_8250
ioc3_serial_probe(pdev, ioc3);
@@ -1292,6 +1240,26 @@ static int ioc3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
timer_setup(&ip->ioc3_timer, ioc3_timer, 0);
ioc3_stop(ip);
+
+ /* Allocate rx ring. 4kb = 512 entries, must be 4kb aligned */
+ ip->rxr = dma_direct_alloc_pages(ip->dma_dev, RX_RING_SIZE,
+ &ip->rxr_dma, GFP_ATOMIC, 0);
+ if (!ip->rxr) {
+ pr_err("ioc3-eth: rx ring allocation failed\n");
+ err = -ENOMEM;
+ goto out_stop;
+ }
+
+ /* Allocate tx rings. 16kb = 128 bufs, must be 16kb aligned */
+ ip->txr = dma_direct_alloc_pages(ip->dma_dev, TX_RING_SIZE,
+ &ip->txr_dma,
+ GFP_KERNEL | __GFP_ZERO, 0);
+ if (!ip->txr) {
+ pr_err("ioc3-eth: tx ring allocation failed\n");
+ err = -ENOMEM;
+ goto out_stop;
+ }
+
ioc3_init(dev);
ip->pdev = pdev;
@@ -1305,7 +1273,7 @@ static int ioc3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
ioc3_mii_init(ip);
if (ip->mii.phy_id == -1) {
- printk(KERN_CRIT "ioc3-eth(%s): Didn't find a PHY, goodbye.\n",
+ pr_err("ioc3-eth(%s): Didn't find a PHY, goodbye.\n",
pci_name(pdev));
err = -ENODEV;
goto out_stop;
@@ -1335,24 +1303,27 @@ static int ioc3_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
vendor = (sw_physid1 << 12) | (sw_physid2 >> 4);
model = (sw_physid2 >> 4) & 0x3f;
rev = sw_physid2 & 0xf;
- printk(KERN_INFO "%s: Using PHY %d, vendor 0x%x, model %d, "
- "rev %d.\n", dev->name, ip->mii.phy_id, vendor, model, rev);
- printk(KERN_INFO "%s: IOC3 SSRAM has %d kbyte.\n", dev->name,
- ip->emcr & EMCR_BUFSIZ ? 128 : 64);
+ netdev_info(dev, "Using PHY %d, vendor 0x%x, model %d, rev %d.\n",
+ ip->mii.phy_id, vendor, model, rev);
+ netdev_info(dev, "IOC3 SSRAM has %d kbyte.\n",
+ ip->emcr & EMCR_BUFSIZ ? 128 : 64);
return 0;
out_stop:
- ioc3_stop(ip);
del_timer_sync(&ip->ioc3_timer);
- ioc3_free_rings(ip);
+ if (ip->rxr)
+ dma_direct_free_pages(ip->dma_dev, RX_RING_SIZE, ip->rxr,
+ ip->rxr_dma, 0);
+ if (ip->txr)
+ dma_direct_free_pages(ip->dma_dev, TX_RING_SIZE, ip->txr,
+ ip->txr_dma, 0);
out_res:
pci_release_regions(pdev);
out_free:
free_netdev(dev);
out_disable:
- /*
- * We should call pci_disable_device(pdev); here if the IOC3 wasn't
+ /* We should call pci_disable_device(pdev); here if the IOC3 wasn't
* such a weird device ...
*/
out:
@@ -1363,16 +1334,19 @@ static void ioc3_remove_one(struct pci_dev *pdev)
{
struct net_device *dev = pci_get_drvdata(pdev);
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
+
+ dma_direct_free_pages(ip->dma_dev, RX_RING_SIZE, ip->rxr,
+ ip->rxr_dma, 0);
+ dma_direct_free_pages(ip->dma_dev, TX_RING_SIZE, ip->txr,
+ ip->txr_dma, 0);
unregister_netdev(dev);
del_timer_sync(&ip->ioc3_timer);
- iounmap(ioc3);
+ iounmap(ip->all_regs);
pci_release_regions(pdev);
free_netdev(dev);
- /*
- * We should call pci_disable_device(pdev); here if the IOC3 wasn't
+ /* We should call pci_disable_device(pdev); here if the IOC3 wasn't
* such a weird device ...
*/
}
@@ -1392,16 +1366,14 @@ static struct pci_driver ioc3_driver = {
static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev)
{
- unsigned long data;
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
- unsigned int len;
struct ioc3_etxd *desc;
- uint32_t w0 = 0;
+ unsigned long data;
+ unsigned int len;
int produce;
+ u32 w0 = 0;
- /*
- * IOC3 has a fairly simple minded checksumming hardware which simply
+ /* IOC3 has a fairly simple minded checksumming hardware which simply
* adds up the 1's complement checksum for the entire packet and
* inserts it at an offset which can be specified in the descriptor
* into the transmit packet. This means we have to compensate for the
@@ -1412,25 +1384,23 @@ static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev)
const struct iphdr *ih = ip_hdr(skb);
const int proto = ntohs(ih->protocol);
unsigned int csoff;
- uint32_t csum, ehsum;
- uint16_t *eh;
+ u32 csum, ehsum;
+ u16 *eh;
/* The MAC header. skb->mac seem the logic approach
- to find the MAC header - except it's a NULL pointer ... */
- eh = (uint16_t *) skb->data;
+ * to find the MAC header - except it's a NULL pointer ...
+ */
+ eh = (u16 *)skb->data;
/* Sum up dest addr, src addr and protocol */
ehsum = eh[0] + eh[1] + eh[2] + eh[3] + eh[4] + eh[5] + eh[6];
- /* Fold ehsum. can't use csum_fold which negates also ... */
- ehsum = (ehsum & 0xffff) + (ehsum >> 16);
- ehsum = (ehsum & 0xffff) + (ehsum >> 16);
-
/* Skip IP header; it's sum is always zero and was
- already filled in by ip_output.c */
+ * already filled in by ip_output.c
+ */
csum = csum_tcpudp_nofold(ih->saddr, ih->daddr,
- ih->tot_len - (ih->ihl << 2),
- proto, 0xffff ^ ehsum);
+ ih->tot_len - (ih->ihl << 2),
+ proto, csum_fold(ehsum));
csum = (csum & 0xffff) + (csum >> 16); /* Fold again */
csum = (csum & 0xffff) + (csum >> 16);
@@ -1450,7 +1420,7 @@ static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev)
spin_lock_irq(&ip->ioc3_lock);
- data = (unsigned long) skb->data;
+ data = (unsigned long)skb->data;
len = skb->len;
produce = ip->tx_pi;
@@ -1470,47 +1440,78 @@ static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev)
unsigned long b2 = (data | 0x3fffUL) + 1UL;
unsigned long s1 = b2 - data;
unsigned long s2 = data + len - b2;
+ dma_addr_t d1, d2;
desc->cmd = cpu_to_be32(len | ETXD_INTWHENDONE |
- ETXD_B1V | ETXD_B2V | w0);
+ ETXD_B1V | ETXD_B2V | w0);
desc->bufcnt = cpu_to_be32((s1 << ETXD_B1CNT_SHIFT) |
- (s2 << ETXD_B2CNT_SHIFT));
- desc->p1 = cpu_to_be64(ioc3_map(skb->data, 1));
- desc->p2 = cpu_to_be64(ioc3_map((void *) b2, 1));
+ (s2 << ETXD_B2CNT_SHIFT));
+ d1 = dma_map_single(ip->dma_dev, skb->data, s1, DMA_TO_DEVICE);
+ if (dma_mapping_error(ip->dma_dev, d1))
+ goto drop_packet;
+ d2 = dma_map_single(ip->dma_dev, (void *)b2, s1, DMA_TO_DEVICE);
+ if (dma_mapping_error(ip->dma_dev, d2)) {
+ dma_unmap_single(ip->dma_dev, d1, len, DMA_TO_DEVICE);
+ goto drop_packet;
+ }
+ desc->p1 = cpu_to_be64(ioc3_map(d1, PCI64_ATTR_PREF));
+ desc->p2 = cpu_to_be64(ioc3_map(d2, PCI64_ATTR_PREF));
} else {
+ dma_addr_t d;
+
/* Normal sized packet that doesn't cross a page boundary. */
desc->cmd = cpu_to_be32(len | ETXD_INTWHENDONE | ETXD_B1V | w0);
desc->bufcnt = cpu_to_be32(len << ETXD_B1CNT_SHIFT);
- desc->p1 = cpu_to_be64(ioc3_map(skb->data, 1));
+ d = dma_map_single(ip->dma_dev, skb->data, len, DMA_TO_DEVICE);
+ if (dma_mapping_error(ip->dma_dev, d))
+ goto drop_packet;
+ desc->p1 = cpu_to_be64(ioc3_map(d, PCI64_ATTR_PREF));
}
- BARRIER();
+ mb(); /* make sure all descriptor changes are visible */
ip->tx_skbs[produce] = skb; /* Remember skb */
- produce = (produce + 1) & 127;
+ produce = (produce + 1) & TX_RING_MASK;
ip->tx_pi = produce;
- ioc3_w_etpir(produce << 7); /* Fire ... */
+ writel(produce << 7, &ip->regs->etpir); /* Fire ... */
ip->txqlen++;
- if (ip->txqlen >= 127)
+ if (ip->txqlen >= (TX_RING_ENTRIES - 1))
netif_stop_queue(dev);
spin_unlock_irq(&ip->ioc3_lock);
return NETDEV_TX_OK;
+
+drop_packet:
+ dev_kfree_skb_any(skb);
+ dev->stats.tx_dropped++;
+
+ spin_unlock_irq(&ip->ioc3_lock);
+
+ return NETDEV_TX_OK;
}
static void ioc3_timeout(struct net_device *dev)
{
struct ioc3_private *ip = netdev_priv(dev);
- printk(KERN_ERR "%s: transmit timed out, resetting\n", dev->name);
+ netdev_err(dev, "transmit timed out, resetting\n");
spin_lock_irq(&ip->ioc3_lock);
ioc3_stop(ip);
+ ioc3_free_rx_bufs(ip);
+ ioc3_clean_tx_ring(ip);
+
ioc3_init(dev);
+ if (ioc3_alloc_rx_bufs(dev)) {
+ netdev_err(dev, "%s: rx buffer allocation failed\n", __func__);
+ spin_unlock_irq(&ip->ioc3_lock);
+ return;
+ }
+ ioc3_start(ip);
ioc3_mii_init(ip);
ioc3_mii_start(ip);
@@ -1519,16 +1520,14 @@ static void ioc3_timeout(struct net_device *dev)
netif_wake_queue(dev);
}
-/*
- * Given a multicast ethernet address, this routine calculates the
+/* Given a multicast ethernet address, this routine calculates the
* address's bit index in the logical address filter mask
*/
-
static inline unsigned int ioc3_hash(const unsigned char *addr)
{
unsigned int temp = 0;
- u32 crc;
int bits;
+ u32 crc;
crc = ether_crc_le(ETH_ALEN, addr);
@@ -1542,8 +1541,8 @@ static inline unsigned int ioc3_hash(const unsigned char *addr)
return temp;
}
-static void ioc3_get_drvinfo (struct net_device *dev,
- struct ethtool_drvinfo *info)
+static void ioc3_get_drvinfo(struct net_device *dev,
+ struct ethtool_drvinfo *info)
{
struct ioc3_private *ip = netdev_priv(dev);
@@ -1623,27 +1622,28 @@ static int ioc3_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
static void ioc3_set_multicast_list(struct net_device *dev)
{
- struct netdev_hw_addr *ha;
struct ioc3_private *ip = netdev_priv(dev);
- struct ioc3 *ioc3 = ip->regs;
+ struct ioc3_ethregs *regs = ip->regs;
+ struct netdev_hw_addr *ha;
u64 ehar = 0;
- netif_stop_queue(dev); /* Lock out others. */
+ spin_lock_irq(&ip->ioc3_lock);
if (dev->flags & IFF_PROMISC) { /* Set promiscuous. */
ip->emcr |= EMCR_PROMISC;
- ioc3_w_emcr(ip->emcr);
- (void) ioc3_r_emcr();
+ writel(ip->emcr, ®s->emcr);
+ readl(®s->emcr);
} else {
ip->emcr &= ~EMCR_PROMISC;
- ioc3_w_emcr(ip->emcr); /* Clear promiscuous. */
- (void) ioc3_r_emcr();
+ writel(ip->emcr, ®s->emcr); /* Clear promiscuous. */
+ readl(®s->emcr);
if ((dev->flags & IFF_ALLMULTI) ||
(netdev_mc_count(dev) > 64)) {
/* Too many for hashing to make sense or we want all
- multicast packets anyway, so skip computing all the
- hashes and just accept all packets. */
+ * multicast packets anyway, so skip computing all the
+ * hashes and just accept all packets.
+ */
ip->ehar_h = 0xffffffff;
ip->ehar_l = 0xffffffff;
} else {
@@ -1653,11 +1653,11 @@ static void ioc3_set_multicast_list(struct net_device *dev)
ip->ehar_h = ehar >> 32;
ip->ehar_l = ehar & 0xffffffff;
}
- ioc3_w_ehar_h(ip->ehar_h);
- ioc3_w_ehar_l(ip->ehar_l);
+ writel(ip->ehar_h, ®s->ehar_h);
+ writel(ip->ehar_l, ®s->ehar_l);
}
- netif_wake_queue(dev); /* Let us get going again. */
+ spin_unlock_irq(&ip->ioc3_lock);
}
module_pci_driver(ioc3_driver);
diff --git a/drivers/net/ethernet/sgi/meth.c b/drivers/net/ethernet/sgi/meth.c
index 00660dd..539bc5d 100644
--- a/drivers/net/ethernet/sgi/meth.c
+++ b/drivers/net/ethernet/sgi/meth.c
@@ -247,8 +247,7 @@ static void meth_free_tx_ring(struct meth_private *priv)
/* Remove any pending skb */
for (i = 0; i < TX_RING_ENTRIES; i++) {
- if (priv->tx_skbs[i])
- dev_kfree_skb(priv->tx_skbs[i]);
+ dev_kfree_skb(priv->tx_skbs[i]);
priv->tx_skbs[i] = NULL;
}
dma_free_coherent(&priv->pdev->dev, TX_RING_BUFFER_SIZE, priv->tx_ring,
diff --git a/drivers/net/ethernet/smsc/smc91x.c b/drivers/net/ethernet/smsc/smc91x.c
index 601e76a..3a67611 100644
--- a/drivers/net/ethernet/smsc/smc91x.c
+++ b/drivers/net/ethernet/smsc/smc91x.c
@@ -378,8 +378,7 @@ static void smc_shutdown(struct net_device *dev)
pending_skb = lp->pending_tx_skb;
lp->pending_tx_skb = NULL;
spin_unlock_irq(&lp->lock);
- if (pending_skb)
- dev_kfree_skb(pending_skb);
+ dev_kfree_skb(pending_skb);
/* and tell the card to stay away from that nasty outside world */
SMC_SELECT_BANK(lp, 0);
diff --git a/drivers/net/ethernet/stmicro/stmmac/Kconfig b/drivers/net/ethernet/stmicro/stmmac/Kconfig
index 2325b40..338e25a 100644
--- a/drivers/net/ethernet/stmicro/stmmac/Kconfig
+++ b/drivers/net/ethernet/stmicro/stmmac/Kconfig
@@ -200,6 +200,7 @@
config STMMAC_PCI
tristate "STMMAC PCI bus support"
depends on STMMAC_ETH && PCI
+ depends on COMMON_CLK
---help---
This selects the platform specific bus support for the stmmac driver.
This driver was tested on XLINX XC2V3000 FF1152AMT0221
diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h
index 49aa56c..912bbb6 100644
--- a/drivers/net/ethernet/stmicro/stmmac/common.h
+++ b/drivers/net/ethernet/stmicro/stmmac/common.h
@@ -360,6 +360,8 @@ struct dma_features {
unsigned int sphen;
unsigned int vlins;
unsigned int dvlan;
+ unsigned int l3l4fnum;
+ unsigned int arpoffsel;
};
/* GMAC TX FIFO is 8K, Rx FIFO is 16K */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c
index 4644b2a..e2e469c 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c
@@ -1194,10 +1194,8 @@ static int phy_power_on(struct rk_priv_data *bsp_priv, bool enable)
int ret;
struct device *dev = &bsp_priv->pdev->dev;
- if (!ldo) {
- dev_err(dev, "no regulator found\n");
- return -1;
- }
+ if (!ldo)
+ return 0;
if (enable) {
ret = regulator_enable(ldo);
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4.h b/drivers/net/ethernet/stmicro/stmmac/dwmac4.h
index 2ed11a5..89a3420 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4.h
@@ -16,7 +16,10 @@
#define GMAC_CONFIG 0x00000000
#define GMAC_PACKET_FILTER 0x00000008
#define GMAC_HASH_TAB(x) (0x10 + (x) * 4)
+#define GMAC_VLAN_TAG 0x00000050
+#define GMAC_VLAN_HASH_TABLE 0x00000058
#define GMAC_RX_FLOW_CTRL 0x00000090
+#define GMAC_VLAN_INCL 0x00000060
#define GMAC_QX_TX_FLOW_CTRL(x) (0x70 + x * 4)
#define GMAC_TXQ_PRTY_MAP0 0x98
#define GMAC_TXQ_PRTY_MAP1 0x9C
@@ -37,6 +40,7 @@
#define GMAC_HW_FEATURE3 0x00000128
#define GMAC_MDIO_ADDR 0x00000200
#define GMAC_MDIO_DATA 0x00000204
+#define GMAC_ARP_ADDR 0x00000210
#define GMAC_ADDR_HIGH(reg) (0x300 + reg * 8)
#define GMAC_ADDR_LOW(reg) (0x304 + reg * 8)
@@ -62,9 +66,22 @@
#define GMAC_PACKET_FILTER_PM BIT(4)
#define GMAC_PACKET_FILTER_PCF BIT(7)
#define GMAC_PACKET_FILTER_HPF BIT(10)
+#define GMAC_PACKET_FILTER_VTFE BIT(16)
#define GMAC_MAX_PERFECT_ADDRESSES 128
+/* MAC VLAN */
+#define GMAC_VLAN_EDVLP BIT(26)
+#define GMAC_VLAN_VTHM BIT(25)
+#define GMAC_VLAN_DOVLTC BIT(20)
+#define GMAC_VLAN_ESVL BIT(18)
+#define GMAC_VLAN_ETV BIT(16)
+#define GMAC_VLAN_VID GENMASK(15, 0)
+#define GMAC_VLAN_VLTI BIT(20)
+#define GMAC_VLAN_CSVL BIT(19)
+#define GMAC_VLAN_VLC GENMASK(17, 16)
+#define GMAC_VLAN_VLC_SHIFT 16
+
/* MAC RX Queue Enable */
#define GMAC_RX_QUEUE_CLEAR(queue) ~(GENMASK(1, 0) << ((queue) * 2))
#define GMAC_RX_AV_QUEUE_ENABLE(queue) BIT((queue) * 2)
@@ -149,6 +166,9 @@ enum power_event {
#define GMAC_DEBUG_RPESTS BIT(0)
/* MAC config */
+#define GMAC_CONFIG_ARPEN BIT(31)
+#define GMAC_CONFIG_SARC GENMASK(30, 28)
+#define GMAC_CONFIG_SARC_SHIFT 28
#define GMAC_CONFIG_IPC BIT(27)
#define GMAC_CONFIG_2K BIT(22)
#define GMAC_CONFIG_ACS BIT(20)
@@ -164,11 +184,13 @@ enum power_event {
#define GMAC_CONFIG_RE BIT(0)
/* MAC HW features0 bitmap */
+#define GMAC_HW_FEAT_SAVLANINS BIT(27)
#define GMAC_HW_FEAT_ADDMAC BIT(18)
#define GMAC_HW_FEAT_RXCOESEL BIT(16)
#define GMAC_HW_FEAT_TXCOSEL BIT(14)
#define GMAC_HW_FEAT_EEESEL BIT(13)
#define GMAC_HW_FEAT_TSSEL BIT(12)
+#define GMAC_HW_FEAT_ARPOFFSEL BIT(9)
#define GMAC_HW_FEAT_MMCSEL BIT(8)
#define GMAC_HW_FEAT_MGKSEL BIT(7)
#define GMAC_HW_FEAT_RWKSEL BIT(6)
@@ -198,6 +220,7 @@ enum power_event {
#define GMAC_HW_FEAT_FRPES GENMASK(14, 13)
#define GMAC_HW_FEAT_FRPBS GENMASK(12, 11)
#define GMAC_HW_FEAT_FRPSEL BIT(10)
+#define GMAC_HW_FEAT_DVLAN BIT(5)
/* MAC HW ADDR regs */
#define GMAC_HI_DCS GENMASK(18, 16)
@@ -352,7 +375,8 @@ enum power_event {
/* Default operating mode of the MAC */
#define GMAC_CORE_INIT (GMAC_CONFIG_JD | GMAC_CONFIG_PS | \
- GMAC_CONFIG_BE | GMAC_CONFIG_DCRS)
+ GMAC_CONFIG_BE | GMAC_CONFIG_DCRS | \
+ GMAC_CONFIG_JE)
/* To dump the core regs excluding the Address Registers */
#define GMAC_REG_NUM 132
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
index fc9954e..9b4b5f6 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
@@ -25,15 +25,9 @@ static void dwmac4_core_init(struct mac_device_info *hw,
{
void __iomem *ioaddr = hw->pcsr;
u32 value = readl(ioaddr + GMAC_CONFIG);
- int mtu = dev->mtu;
value |= GMAC_CORE_INIT;
- if (mtu > 1500)
- value |= GMAC_CONFIG_2K;
- if (mtu > 2000)
- value |= GMAC_CONFIG_JE;
-
if (hw->ps) {
value |= GMAC_CONFIG_TE;
@@ -737,6 +731,73 @@ static void dwmac4_set_mac_loopback(void __iomem *ioaddr, bool enable)
writel(value, ioaddr + GMAC_CONFIG);
}
+static void dwmac4_update_vlan_hash(struct mac_device_info *hw, u32 hash,
+ bool is_double)
+{
+ void __iomem *ioaddr = hw->pcsr;
+
+ writel(hash, ioaddr + GMAC_VLAN_HASH_TABLE);
+
+ if (hash) {
+ u32 value = GMAC_VLAN_VTHM | GMAC_VLAN_ETV;
+ if (is_double) {
+ value |= GMAC_VLAN_EDVLP;
+ value |= GMAC_VLAN_ESVL;
+ value |= GMAC_VLAN_DOVLTC;
+ }
+
+ writel(value, ioaddr + GMAC_VLAN_TAG);
+ } else {
+ u32 value = readl(ioaddr + GMAC_VLAN_TAG);
+
+ value &= ~(GMAC_VLAN_VTHM | GMAC_VLAN_ETV);
+ value &= ~(GMAC_VLAN_EDVLP | GMAC_VLAN_ESVL);
+ value &= ~GMAC_VLAN_DOVLTC;
+ value &= ~GMAC_VLAN_VID;
+
+ writel(value, ioaddr + GMAC_VLAN_TAG);
+ }
+}
+
+static void dwmac4_sarc_configure(void __iomem *ioaddr, int val)
+{
+ u32 value = readl(ioaddr + GMAC_CONFIG);
+
+ value &= ~GMAC_CONFIG_SARC;
+ value |= val << GMAC_CONFIG_SARC_SHIFT;
+
+ writel(value, ioaddr + GMAC_CONFIG);
+}
+
+static void dwmac4_enable_vlan(struct mac_device_info *hw, u32 type)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+
+ value = readl(ioaddr + GMAC_VLAN_INCL);
+ value |= GMAC_VLAN_VLTI;
+ value |= GMAC_VLAN_CSVL; /* Only use SVLAN */
+ value &= ~GMAC_VLAN_VLC;
+ value |= (type << GMAC_VLAN_VLC_SHIFT) & GMAC_VLAN_VLC;
+ writel(value, ioaddr + GMAC_VLAN_INCL);
+}
+
+static void dwmac4_set_arp_offload(struct mac_device_info *hw, bool en,
+ u32 addr)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+
+ writel(addr, ioaddr + GMAC_ARP_ADDR);
+
+ value = readl(ioaddr + GMAC_CONFIG);
+ if (en)
+ value |= GMAC_CONFIG_ARPEN;
+ else
+ value &= ~GMAC_CONFIG_ARPEN;
+ writel(value, ioaddr + GMAC_CONFIG);
+}
+
const struct stmmac_ops dwmac4_ops = {
.core_init = dwmac4_core_init,
.set_mac = stmmac_set_mac,
@@ -767,6 +828,10 @@ const struct stmmac_ops dwmac4_ops = {
.debug = dwmac4_debug,
.set_filter = dwmac4_set_filter,
.set_mac_loopback = dwmac4_set_mac_loopback,
+ .update_vlan_hash = dwmac4_update_vlan_hash,
+ .sarc_configure = dwmac4_sarc_configure,
+ .enable_vlan = dwmac4_enable_vlan,
+ .set_arp_offload = dwmac4_set_arp_offload,
};
const struct stmmac_ops dwmac410_ops = {
@@ -799,6 +864,10 @@ const struct stmmac_ops dwmac410_ops = {
.debug = dwmac4_debug,
.set_filter = dwmac4_set_filter,
.set_mac_loopback = dwmac4_set_mac_loopback,
+ .update_vlan_hash = dwmac4_update_vlan_hash,
+ .sarc_configure = dwmac4_sarc_configure,
+ .enable_vlan = dwmac4_enable_vlan,
+ .set_arp_offload = dwmac4_set_arp_offload,
};
const struct stmmac_ops dwmac510_ops = {
@@ -836,6 +905,10 @@ const struct stmmac_ops dwmac510_ops = {
.rxp_config = dwmac5_rxp_config,
.flex_pps_config = dwmac5_flex_pps_config,
.set_mac_loopback = dwmac4_set_mac_loopback,
+ .update_vlan_hash = dwmac4_update_vlan_hash,
+ .sarc_configure = dwmac4_sarc_configure,
+ .enable_vlan = dwmac4_enable_vlan,
+ .set_arp_offload = dwmac4_set_arp_offload,
};
int dwmac4_setup(struct stmmac_priv *priv)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
index dbde23e..15eb1ab 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
@@ -443,6 +443,13 @@ static void dwmac4_clear(struct dma_desc *p)
p->des3 = 0;
}
+static void dwmac4_set_sarc(struct dma_desc *p, u32 sarc_type)
+{
+ sarc_type <<= TDES3_SA_INSERT_CTRL_SHIFT;
+
+ p->des3 |= cpu_to_le32(sarc_type & TDES3_SA_INSERT_CTRL_MASK);
+}
+
static int set_16kib_bfsize(int mtu)
{
int ret = 0;
@@ -452,6 +459,39 @@ static int set_16kib_bfsize(int mtu)
return ret;
}
+static void dwmac4_set_vlan_tag(struct dma_desc *p, u16 tag, u16 inner_tag,
+ u32 inner_type)
+{
+ p->des0 = 0;
+ p->des1 = 0;
+ p->des2 = 0;
+ p->des3 = 0;
+
+ /* Inner VLAN */
+ if (inner_type) {
+ u32 des = inner_tag << TDES2_IVT_SHIFT;
+
+ des &= TDES2_IVT_MASK;
+ p->des2 = cpu_to_le32(des);
+
+ des = inner_type << TDES3_IVTIR_SHIFT;
+ des &= TDES3_IVTIR_MASK;
+ p->des3 = cpu_to_le32(des | TDES3_IVLTV);
+ }
+
+ /* Outer VLAN */
+ p->des3 |= cpu_to_le32(tag & TDES3_VLAN_TAG);
+ p->des3 |= cpu_to_le32(TDES3_VLTV);
+
+ p->des3 |= cpu_to_le32(TDES3_CONTEXT_TYPE);
+}
+
+static void dwmac4_set_vlan(struct dma_desc *p, u32 type)
+{
+ type <<= TDES2_VLAN_TAG_SHIFT;
+ p->des2 |= cpu_to_le32(type & TDES2_VLAN_TAG_MASK);
+}
+
const struct stmmac_desc_ops dwmac4_desc_ops = {
.tx_status = dwmac4_wrback_get_tx_status,
.rx_status = dwmac4_wrback_get_rx_status,
@@ -476,6 +516,9 @@ const struct stmmac_desc_ops dwmac4_desc_ops = {
.get_addr = dwmac4_get_addr,
.set_addr = dwmac4_set_addr,
.clear = dwmac4_clear,
+ .set_sarc = dwmac4_set_sarc,
+ .set_vlan_tag = dwmac4_set_vlan_tag,
+ .set_vlan = dwmac4_set_vlan,
};
const struct stmmac_mode_ops dwmac4_ring_mode_ops = {
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.h b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.h
index f581911..0d7b3bb 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.h
@@ -18,13 +18,21 @@
/* TDES2 (read format) */
#define TDES2_BUFFER1_SIZE_MASK GENMASK(13, 0)
#define TDES2_VLAN_TAG_MASK GENMASK(15, 14)
+#define TDES2_VLAN_TAG_SHIFT 14
#define TDES2_BUFFER2_SIZE_MASK GENMASK(29, 16)
#define TDES2_BUFFER2_SIZE_MASK_SHIFT 16
+#define TDES3_IVTIR_MASK GENMASK(19, 18)
+#define TDES3_IVTIR_SHIFT 18
+#define TDES3_IVLTV BIT(17)
#define TDES2_TIMESTAMP_ENABLE BIT(30)
+#define TDES2_IVT_MASK GENMASK(31, 16)
+#define TDES2_IVT_SHIFT 16
#define TDES2_INTERRUPT_ON_COMPLETION BIT(31)
/* TDES3 (read format) */
#define TDES3_PACKET_SIZE_MASK GENMASK(14, 0)
+#define TDES3_VLAN_TAG GENMASK(15, 0)
+#define TDES3_VLTV BIT(16)
#define TDES3_CHECKSUM_INSERTION_MASK GENMASK(17, 16)
#define TDES3_CHECKSUM_INSERTION_SHIFT 16
#define TDES3_TCP_PKT_PAYLOAD_MASK GENMASK(17, 0)
@@ -32,6 +40,7 @@
#define TDES3_HDR_LEN_SHIFT 19
#define TDES3_SLOT_NUMBER_MASK GENMASK(22, 19)
#define TDES3_SA_INSERT_CTRL_MASK GENMASK(25, 23)
+#define TDES3_SA_INSERT_CTRL_SHIFT 23
#define TDES3_CRC_PAD_CTRL_MASK GENMASK(27, 26)
/* TDES3 (write back format) */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c
index 3ed5508..68c1579 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c
@@ -333,7 +333,7 @@ static void dwmac4_get_hw_feature(void __iomem *ioaddr,
dma_cap->mbps_10_100 = (hw_cap & GMAC_HW_FEAT_MIISEL);
dma_cap->mbps_1000 = (hw_cap & GMAC_HW_FEAT_GMIISEL) >> 1;
dma_cap->half_duplex = (hw_cap & GMAC_HW_FEAT_HDSEL) >> 2;
- dma_cap->hash_filter = (hw_cap & GMAC_HW_FEAT_VLHASH) >> 4;
+ dma_cap->vlhash = (hw_cap & GMAC_HW_FEAT_VLHASH) >> 4;
dma_cap->multi_addr = (hw_cap & GMAC_HW_FEAT_ADDMAC) >> 18;
dma_cap->pcs = (hw_cap & GMAC_HW_FEAT_PCSSEL) >> 3;
dma_cap->sma_mdio = (hw_cap & GMAC_HW_FEAT_SMASEL) >> 5;
@@ -348,6 +348,8 @@ static void dwmac4_get_hw_feature(void __iomem *ioaddr,
/* TX and RX csum */
dma_cap->tx_coe = (hw_cap & GMAC_HW_FEAT_TXCOSEL) >> 14;
dma_cap->rx_coe = (hw_cap & GMAC_HW_FEAT_RXCOESEL) >> 16;
+ dma_cap->vlins = (hw_cap & GMAC_HW_FEAT_SAVLANINS) >> 27;
+ dma_cap->arpoffsel = (hw_cap & GMAC_HW_FEAT_ARPOFFSEL) >> 9;
/* MAC HW feature1 */
hw_cap = readl(ioaddr + GMAC_HW_FEATURE1);
@@ -385,6 +387,7 @@ static void dwmac4_get_hw_feature(void __iomem *ioaddr,
dma_cap->frpes = (hw_cap & GMAC_HW_FEAT_FRPES) >> 13;
dma_cap->frpbs = (hw_cap & GMAC_HW_FEAT_FRPBS) >> 11;
dma_cap->frpsel = (hw_cap & GMAC_HW_FEAT_FRPSEL) >> 10;
+ dma_cap->dvlan = (hw_cap & GMAC_HW_FEAT_DVLAN) >> 5;
}
/* Enable/disable TSO feature and set MSS */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
index 7357b8b..5923ca6 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
@@ -44,9 +44,11 @@
#define XGMAC_CONFIG_CST BIT(2)
#define XGMAC_CONFIG_ACS BIT(1)
#define XGMAC_CONFIG_RE BIT(0)
-#define XGMAC_CORE_INIT_RX 0
+#define XGMAC_CORE_INIT_RX (XGMAC_CONFIG_GPSLCE | XGMAC_CONFIG_WD | \
+ (XGMAC_JUMBO_LEN << XGMAC_CONFIG_GPSL_SHIFT))
#define XGMAC_PACKET_FILTER 0x00000008
#define XGMAC_FILTER_RA BIT(31)
+#define XGMAC_FILTER_IPFE BIT(20)
#define XGMAC_FILTER_VTFE BIT(16)
#define XGMAC_FILTER_HPF BIT(10)
#define XGMAC_FILTER_PCF BIT(7)
@@ -119,6 +121,7 @@
#define XGMAC_HWFEAT_VLHASH BIT(4)
#define XGMAC_HWFEAT_GMIISEL BIT(1)
#define XGMAC_HW_FEATURE1 0x00000120
+#define XGMAC_HWFEAT_L3L4FNUM GENMASK(30, 27)
#define XGMAC_HWFEAT_RSSEN BIT(20)
#define XGMAC_HWFEAT_TSOEN BIT(18)
#define XGMAC_HWFEAT_SPHEN BIT(17)
@@ -150,6 +153,34 @@
#define XGMAC_DCS GENMASK(19, 16)
#define XGMAC_DCS_SHIFT 16
#define XGMAC_ADDRx_LOW(x) (0x00000304 + (x) * 0x8)
+#define XGMAC_L3L4_ADDR_CTRL 0x00000c00
+#define XGMAC_IDDR GENMASK(15, 8)
+#define XGMAC_IDDR_SHIFT 8
+#define XGMAC_IDDR_FNUM 4
+#define XGMAC_TT BIT(1)
+#define XGMAC_XB BIT(0)
+#define XGMAC_L3L4_DATA 0x00000c04
+#define XGMAC_L3L4_CTRL 0x0
+#define XGMAC_L4DPIM0 BIT(21)
+#define XGMAC_L4DPM0 BIT(20)
+#define XGMAC_L4SPIM0 BIT(19)
+#define XGMAC_L4SPM0 BIT(18)
+#define XGMAC_L4PEN0 BIT(16)
+#define XGMAC_L3HDBM0 GENMASK(15, 11)
+#define XGMAC_L3HSBM0 GENMASK(10, 6)
+#define XGMAC_L3DAIM0 BIT(5)
+#define XGMAC_L3DAM0 BIT(4)
+#define XGMAC_L3SAIM0 BIT(3)
+#define XGMAC_L3SAM0 BIT(2)
+#define XGMAC_L3PEN0 BIT(0)
+#define XGMAC_L4_ADDR 0x1
+#define XGMAC_L4DP0 GENMASK(31, 16)
+#define XGMAC_L4DP0_SHIFT 16
+#define XGMAC_L4SP0 GENMASK(15, 0)
+#define XGMAC_L3_ADDR0 0x4
+#define XGMAC_L3_ADDR1 0x5
+#define XGMAC_L3_ADDR2 0x6
+#define XMGAC_L3_ADDR3 0x7
#define XGMAC_ARP_ADDR 0x00000c10
#define XGMAC_RSS_CTRL 0x00000c80
#define XGMAC_UDP4TE BIT(3)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
index e534a3a..d5173dd 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
@@ -15,7 +15,6 @@ static void dwxgmac2_core_init(struct mac_device_info *hw,
struct net_device *dev)
{
void __iomem *ioaddr = hw->pcsr;
- int mtu = dev->mtu;
u32 tx, rx;
tx = readl(ioaddr + XGMAC_TX_CONFIG);
@@ -24,16 +23,6 @@ static void dwxgmac2_core_init(struct mac_device_info *hw,
tx |= XGMAC_CORE_INIT_TX;
rx |= XGMAC_CORE_INIT_RX;
- if (mtu >= 9000) {
- rx |= XGMAC_CONFIG_GPSLCE;
- rx |= XGMAC_JUMBO_LEN << XGMAC_CONFIG_GPSL_SHIFT;
- rx |= XGMAC_CONFIG_WD;
- } else if (mtu > 2000) {
- rx |= XGMAC_CONFIG_JE;
- } else if (mtu > 1500) {
- rx |= XGMAC_CONFIG_S2KP;
- }
-
if (hw->ps) {
tx |= XGMAC_CONFIG_TE;
tx &= ~hw->link.speed_mask;
@@ -579,7 +568,7 @@ static void dwxgmac2_update_vlan_hash(struct mac_device_info *hw, u32 hash,
writel(value, ioaddr + XGMAC_PACKET_FILTER);
- value |= XGMAC_VLAN_VTHM | XGMAC_VLAN_ETV;
+ value = XGMAC_VLAN_VTHM | XGMAC_VLAN_ETV;
if (is_double) {
value |= XGMAC_VLAN_EDVLP;
value |= XGMAC_VLAN_ESVL;
@@ -1163,6 +1152,197 @@ static void dwxgmac2_enable_vlan(struct mac_device_info *hw, u32 type)
writel(value, ioaddr + XGMAC_VLAN_INCL);
}
+static int dwxgmac2_filter_wait(struct mac_device_info *hw)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+
+ if (readl_poll_timeout(ioaddr + XGMAC_L3L4_ADDR_CTRL, value,
+ !(value & XGMAC_XB), 100, 10000))
+ return -EBUSY;
+ return 0;
+}
+
+static int dwxgmac2_filter_read(struct mac_device_info *hw, u32 filter_no,
+ u8 reg, u32 *data)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+ int ret;
+
+ ret = dwxgmac2_filter_wait(hw);
+ if (ret)
+ return ret;
+
+ value = ((filter_no << XGMAC_IDDR_FNUM) | reg) << XGMAC_IDDR_SHIFT;
+ value |= XGMAC_TT | XGMAC_XB;
+ writel(value, ioaddr + XGMAC_L3L4_ADDR_CTRL);
+
+ ret = dwxgmac2_filter_wait(hw);
+ if (ret)
+ return ret;
+
+ *data = readl(ioaddr + XGMAC_L3L4_DATA);
+ return 0;
+}
+
+static int dwxgmac2_filter_write(struct mac_device_info *hw, u32 filter_no,
+ u8 reg, u32 data)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+ int ret;
+
+ ret = dwxgmac2_filter_wait(hw);
+ if (ret)
+ return ret;
+
+ writel(data, ioaddr + XGMAC_L3L4_DATA);
+
+ value = ((filter_no << XGMAC_IDDR_FNUM) | reg) << XGMAC_IDDR_SHIFT;
+ value |= XGMAC_XB;
+ writel(value, ioaddr + XGMAC_L3L4_ADDR_CTRL);
+
+ return dwxgmac2_filter_wait(hw);
+}
+
+static int dwxgmac2_config_l3_filter(struct mac_device_info *hw, u32 filter_no,
+ bool en, bool ipv6, bool sa, bool inv,
+ u32 match)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+ int ret;
+
+ value = readl(ioaddr + XGMAC_PACKET_FILTER);
+ value |= XGMAC_FILTER_IPFE;
+ writel(value, ioaddr + XGMAC_PACKET_FILTER);
+
+ ret = dwxgmac2_filter_read(hw, filter_no, XGMAC_L3L4_CTRL, &value);
+ if (ret)
+ return ret;
+
+ /* For IPv6 not both SA/DA filters can be active */
+ if (ipv6) {
+ value |= XGMAC_L3PEN0;
+ value &= ~(XGMAC_L3SAM0 | XGMAC_L3SAIM0);
+ value &= ~(XGMAC_L3DAM0 | XGMAC_L3DAIM0);
+ if (sa) {
+ value |= XGMAC_L3SAM0;
+ if (inv)
+ value |= XGMAC_L3SAIM0;
+ } else {
+ value |= XGMAC_L3DAM0;
+ if (inv)
+ value |= XGMAC_L3DAIM0;
+ }
+ } else {
+ value &= ~XGMAC_L3PEN0;
+ if (sa) {
+ value |= XGMAC_L3SAM0;
+ if (inv)
+ value |= XGMAC_L3SAIM0;
+ } else {
+ value |= XGMAC_L3DAM0;
+ if (inv)
+ value |= XGMAC_L3DAIM0;
+ }
+ }
+
+ ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L3L4_CTRL, value);
+ if (ret)
+ return ret;
+
+ if (sa) {
+ ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L3_ADDR0, match);
+ if (ret)
+ return ret;
+ } else {
+ ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L3_ADDR1, match);
+ if (ret)
+ return ret;
+ }
+
+ if (!en)
+ return dwxgmac2_filter_write(hw, filter_no, XGMAC_L3L4_CTRL, 0);
+
+ return 0;
+}
+
+static int dwxgmac2_config_l4_filter(struct mac_device_info *hw, u32 filter_no,
+ bool en, bool udp, bool sa, bool inv,
+ u32 match)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+ int ret;
+
+ value = readl(ioaddr + XGMAC_PACKET_FILTER);
+ value |= XGMAC_FILTER_IPFE;
+ writel(value, ioaddr + XGMAC_PACKET_FILTER);
+
+ ret = dwxgmac2_filter_read(hw, filter_no, XGMAC_L3L4_CTRL, &value);
+ if (ret)
+ return ret;
+
+ if (udp) {
+ value |= XGMAC_L4PEN0;
+ } else {
+ value &= ~XGMAC_L4PEN0;
+ }
+
+ value &= ~(XGMAC_L4SPM0 | XGMAC_L4SPIM0);
+ value &= ~(XGMAC_L4DPM0 | XGMAC_L4DPIM0);
+ if (sa) {
+ value |= XGMAC_L4SPM0;
+ if (inv)
+ value |= XGMAC_L4SPIM0;
+ } else {
+ value |= XGMAC_L4DPM0;
+ if (inv)
+ value |= XGMAC_L4DPIM0;
+ }
+
+ ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L3L4_CTRL, value);
+ if (ret)
+ return ret;
+
+ if (sa) {
+ value = match & XGMAC_L4SP0;
+
+ ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L4_ADDR, value);
+ if (ret)
+ return ret;
+ } else {
+ value = (match << XGMAC_L4DP0_SHIFT) & XGMAC_L4DP0;
+
+ ret = dwxgmac2_filter_write(hw, filter_no, XGMAC_L4_ADDR, value);
+ if (ret)
+ return ret;
+ }
+
+ if (!en)
+ return dwxgmac2_filter_write(hw, filter_no, XGMAC_L3L4_CTRL, 0);
+
+ return 0;
+}
+
+static void dwxgmac2_set_arp_offload(struct mac_device_info *hw, bool en,
+ u32 addr)
+{
+ void __iomem *ioaddr = hw->pcsr;
+ u32 value;
+
+ writel(addr, ioaddr + XGMAC_ARP_ADDR);
+
+ value = readl(ioaddr + XGMAC_RX_CONFIG);
+ if (en)
+ value |= XGMAC_CONFIG_ARPEN;
+ else
+ value &= ~XGMAC_CONFIG_ARPEN;
+ writel(value, ioaddr + XGMAC_RX_CONFIG);
+}
+
const struct stmmac_ops dwxgmac210_ops = {
.core_init = dwxgmac2_core_init,
.set_mac = dwxgmac2_set_mac,
@@ -1203,6 +1383,9 @@ const struct stmmac_ops dwxgmac210_ops = {
.flex_pps_config = dwxgmac2_flex_pps_config,
.sarc_configure = dwxgmac2_sarc_configure,
.enable_vlan = dwxgmac2_enable_vlan,
+ .config_l3_filter = dwxgmac2_config_l3_filter,
+ .config_l4_filter = dwxgmac2_config_l4_filter,
+ .set_arp_offload = dwxgmac2_set_arp_offload,
};
int dwxgmac2_setup(struct stmmac_priv *priv)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c
index 6495646..53c4a40 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c
@@ -322,6 +322,10 @@ static int dwxgmac2_dma_interrupt(void __iomem *ioaddr,
/* ABNORMAL interrupts */
if (unlikely(intr_status & XGMAC_AIS)) {
+ if (unlikely(intr_status & XGMAC_RBU)) {
+ x->rx_buf_unav_irq++;
+ ret |= handle_rx;
+ }
if (unlikely(intr_status & XGMAC_TPS)) {
x->tx_process_stopped_irq++;
ret |= tx_hard_error;
@@ -365,7 +369,8 @@ static void dwxgmac2_get_hw_feature(void __iomem *ioaddr,
dma_cap->eee = (hw_cap & XGMAC_HWFEAT_EEESEL) >> 13;
dma_cap->atime_stamp = (hw_cap & XGMAC_HWFEAT_TSSEL) >> 12;
dma_cap->av = (hw_cap & XGMAC_HWFEAT_AVSEL) >> 11;
- dma_cap->av &= (hw_cap & XGMAC_HWFEAT_RAVSEL) >> 10;
+ dma_cap->av &= !(hw_cap & XGMAC_HWFEAT_RAVSEL) >> 10;
+ dma_cap->arpoffsel = (hw_cap & XGMAC_HWFEAT_ARPOFFSEL) >> 9;
dma_cap->rmon = (hw_cap & XGMAC_HWFEAT_MMCSEL) >> 8;
dma_cap->pmt_magic_frame = (hw_cap & XGMAC_HWFEAT_MGKSEL) >> 7;
dma_cap->pmt_remote_wake_up = (hw_cap & XGMAC_HWFEAT_RWKSEL) >> 6;
@@ -374,6 +379,7 @@ static void dwxgmac2_get_hw_feature(void __iomem *ioaddr,
/* MAC HW feature 1 */
hw_cap = readl(ioaddr + XGMAC_HW_FEATURE1);
+ dma_cap->l3l4fnum = (hw_cap & XGMAC_HWFEAT_L3L4FNUM) >> 27;
dma_cap->rssen = (hw_cap & XGMAC_HWFEAT_RSSEN) >> 20;
dma_cap->tsoen = (hw_cap & XGMAC_HWFEAT_TSOEN) >> 18;
dma_cap->sphen = (hw_cap & XGMAC_HWFEAT_SPHEN) >> 17;
diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h
index 9435b31..ddb851d 100644
--- a/drivers/net/ethernet/stmicro/stmmac/hwif.h
+++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h
@@ -363,6 +363,14 @@ struct stmmac_ops {
int (*get_mac_tx_timestamp)(struct mac_device_info *hw, u64 *ts);
/* Source Address Insertion / Replacement */
void (*sarc_configure)(void __iomem *ioaddr, int val);
+ /* Filtering */
+ int (*config_l3_filter)(struct mac_device_info *hw, u32 filter_no,
+ bool en, bool ipv6, bool sa, bool inv,
+ u32 match);
+ int (*config_l4_filter)(struct mac_device_info *hw, u32 filter_no,
+ bool en, bool udp, bool sa, bool inv,
+ u32 match);
+ void (*set_arp_offload)(struct mac_device_info *hw, bool en, u32 addr);
};
#define stmmac_core_init(__priv, __args...) \
@@ -443,6 +451,12 @@ struct stmmac_ops {
stmmac_do_callback(__priv, mac, get_mac_tx_timestamp, __args)
#define stmmac_sarc_configure(__priv, __args...) \
stmmac_do_void_callback(__priv, mac, sarc_configure, __args)
+#define stmmac_config_l3_filter(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, config_l3_filter, __args)
+#define stmmac_config_l4_filter(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, config_l4_filter, __args)
+#define stmmac_set_arp_offload(__priv, __args...) \
+ stmmac_do_void_callback(__priv, mac, set_arp_offload, __args)
/* PTP and HW Timer helpers */
struct stmmac_hwtimestamp {
@@ -499,6 +513,7 @@ struct stmmac_mode_ops {
struct stmmac_priv;
struct tc_cls_u32_offload;
struct tc_cbs_qopt_offload;
+struct flow_cls_offload;
struct stmmac_tc_ops {
int (*init)(struct stmmac_priv *priv);
@@ -506,6 +521,8 @@ struct stmmac_tc_ops {
struct tc_cls_u32_offload *cls);
int (*setup_cbs)(struct stmmac_priv *priv,
struct tc_cbs_qopt_offload *qopt);
+ int (*setup_cls)(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls);
};
#define stmmac_tc_init(__priv, __args...) \
@@ -514,6 +531,8 @@ struct stmmac_tc_ops {
stmmac_do_callback(__priv, tc, setup_cls_u32, __args)
#define stmmac_tc_setup_cbs(__priv, __args...) \
stmmac_do_callback(__priv, tc, setup_cbs, __args)
+#define stmmac_tc_setup_cls(__priv, __args...) \
+ stmmac_do_callback(__priv, tc, setup_cls, __args)
struct stmmac_counters;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
index dcb2e29..d993fc7 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
@@ -128,6 +128,16 @@ struct stmmac_rss {
u32 table[STMMAC_RSS_MAX_TABLE_SIZE];
};
+#define STMMAC_FLOW_ACTION_DROP BIT(0)
+struct stmmac_flow_entry {
+ unsigned long cookie;
+ unsigned long action;
+ u8 ip_proto;
+ int in_use;
+ int idx;
+ int is_l4;
+};
+
struct stmmac_priv {
/* Frequently used values are kept adjacent for cache effect */
u32 tx_coal_frames;
@@ -216,6 +226,8 @@ struct stmmac_priv {
unsigned int tc_entries_max;
unsigned int tc_off_max;
struct stmmac_tc_entry *tc_entries;
+ unsigned int flow_entries_max;
+ struct stmmac_flow_entry *flow_entries;
/* Pulse Per Second output */
struct stmmac_pps_cfg pps[STMMAC_PPS_MAX];
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
index 1c45010..1a76883 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
@@ -746,8 +746,15 @@ static int stmmac_set_coalesce(struct net_device *dev,
(ec->tx_max_coalesced_frames_high) || (ec->rate_sample_interval))
return -EOPNOTSUPP;
- if (ec->rx_coalesce_usecs == 0)
- return -EINVAL;
+ if (priv->use_riwt && (ec->rx_coalesce_usecs > 0)) {
+ rx_riwt = stmmac_usec2riwt(ec->rx_coalesce_usecs, priv);
+
+ if ((rx_riwt > MAX_DMA_RIWT) || (rx_riwt < MIN_DMA_RIWT))
+ return -EINVAL;
+
+ priv->rx_riwt = rx_riwt;
+ stmmac_rx_watchdog(priv, priv->ioaddr, priv->rx_riwt, rx_cnt);
+ }
if ((ec->tx_coalesce_usecs == 0) &&
(ec->tx_max_coalesced_frames == 0))
@@ -757,20 +764,10 @@ static int stmmac_set_coalesce(struct net_device *dev,
(ec->tx_max_coalesced_frames > STMMAC_TX_MAX_FRAMES))
return -EINVAL;
- rx_riwt = stmmac_usec2riwt(ec->rx_coalesce_usecs, priv);
-
- if ((rx_riwt > MAX_DMA_RIWT) || (rx_riwt < MIN_DMA_RIWT))
- return -EINVAL;
- else if (!priv->use_riwt)
- return -EOPNOTSUPP;
-
/* Only copy relevant parameters, ignore all others. */
priv->tx_coal_frames = ec->tx_max_coalesced_frames;
priv->tx_coal_timer = ec->tx_coalesce_usecs;
priv->rx_coal_frames = ec->rx_max_coalesced_frames;
- priv->rx_riwt = rx_riwt;
- stmmac_rx_watchdog(priv, priv->ioaddr, priv->rx_riwt, rx_cnt);
-
return 0;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index bd10784..6e44013 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -831,15 +831,22 @@ static void stmmac_validate(struct phylink_config *config,
phylink_set(mask, 1000baseT_Full);
phylink_set(mask, 1000baseX_Full);
} else if (priv->plat->has_xgmac) {
- phylink_set(mac_supported, 2500baseT_Full);
- phylink_set(mac_supported, 5000baseT_Full);
- phylink_set(mac_supported, 10000baseSR_Full);
- phylink_set(mac_supported, 10000baseLR_Full);
- phylink_set(mac_supported, 10000baseER_Full);
- phylink_set(mac_supported, 10000baseLRM_Full);
- phylink_set(mac_supported, 10000baseT_Full);
- phylink_set(mac_supported, 10000baseKX4_Full);
- phylink_set(mac_supported, 10000baseKR_Full);
+ if (!max_speed || (max_speed >= 2500)) {
+ phylink_set(mac_supported, 2500baseT_Full);
+ phylink_set(mac_supported, 2500baseX_Full);
+ }
+ if (!max_speed || (max_speed >= 5000)) {
+ phylink_set(mac_supported, 5000baseT_Full);
+ }
+ if (!max_speed || (max_speed >= 10000)) {
+ phylink_set(mac_supported, 10000baseSR_Full);
+ phylink_set(mac_supported, 10000baseLR_Full);
+ phylink_set(mac_supported, 10000baseER_Full);
+ phylink_set(mac_supported, 10000baseLRM_Full);
+ phylink_set(mac_supported, 10000baseT_Full);
+ phylink_set(mac_supported, 10000baseKX4_Full);
+ phylink_set(mac_supported, 10000baseKR_Full);
+ }
}
/* Half-Duplex can only work with single queue */
@@ -3411,7 +3418,9 @@ static inline void stmmac_rx_refill(struct stmmac_priv *priv, u32 queue)
stmmac_refill_desc3(priv, rx_q, p);
rx_q->rx_count_frames++;
- rx_q->rx_count_frames %= priv->rx_coal_frames;
+ rx_q->rx_count_frames += priv->rx_coal_frames;
+ if (rx_q->rx_count_frames > priv->rx_coal_frames)
+ rx_q->rx_count_frames = 0;
use_rx_wd = priv->use_riwt && rx_q->rx_count_frames;
dma_wmb();
@@ -3511,16 +3520,16 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
&priv->xstats, rx_q->dma_erx + entry);
if (unlikely(status == discard_frame)) {
page_pool_recycle_direct(rx_q->page_pool, buf->page);
- priv->dev->stats.rx_errors++;
buf->page = NULL;
error = 1;
+ if (!priv->hwts_rx_en)
+ priv->dev->stats.rx_errors++;
}
if (unlikely(error && (status & rx_not_ls)))
goto read_again;
if (unlikely(error)) {
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
continue;
}
@@ -3932,12 +3941,17 @@ static int stmmac_setup_tc_block_cb(enum tc_setup_type type, void *type_data,
struct stmmac_priv *priv = cb_priv;
int ret = -EOPNOTSUPP;
+ if (!tc_cls_can_offload_and_chain0(priv->dev, type_data))
+ return ret;
+
stmmac_disable_all_queues(priv);
switch (type) {
case TC_SETUP_CLSU32:
- if (tc_cls_can_offload_and_chain0(priv->dev, type_data))
- ret = stmmac_tc_setup_cls_u32(priv, priv, type_data);
+ ret = stmmac_tc_setup_cls_u32(priv, priv, type_data);
+ break;
+ case TC_SETUP_CLSFLOWER:
+ ret = stmmac_tc_setup_cls(priv, priv, type_data);
break;
default:
break;
@@ -4537,10 +4551,10 @@ int stmmac_dvr_probe(struct device *device,
/* MTU range: 46 - hw-specific max */
ndev->min_mtu = ETH_ZLEN - ETH_HLEN;
- if ((priv->plat->enh_desc) || (priv->synopsys_id >= DWMAC_CORE_4_00))
- ndev->max_mtu = JUMBO_LEN;
- else if (priv->plat->has_xgmac)
+ if (priv->plat->has_xgmac)
ndev->max_mtu = XGMAC_JUMBO_LEN;
+ else if ((priv->plat->enh_desc) || (priv->synopsys_id >= DWMAC_CORE_4_00))
+ ndev->max_mtu = JUMBO_LEN;
else
ndev->max_mtu = SKB_MAX_HEAD(NET_SKB_PAD + NET_IP_ALIGN);
/* Will not overwrite ndev->max_mtu if plat->maxmtu > ndev->max_mtu
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_pci.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_pci.c
index d5d08e1..2090628 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_pci.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_pci.c
@@ -9,6 +9,7 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#include <linux/clk-provider.h>
#include <linux/pci.h>
#include <linux/dmi.h>
@@ -108,6 +109,166 @@ static const struct stmmac_pci_info stmmac_pci_info = {
.setup = stmmac_default_data,
};
+static int intel_mgbe_common_data(struct pci_dev *pdev,
+ struct plat_stmmacenet_data *plat)
+{
+ int i;
+
+ plat->clk_csr = 5;
+ plat->has_gmac = 0;
+ plat->has_gmac4 = 1;
+ plat->force_sf_dma_mode = 0;
+ plat->tso_en = 1;
+
+ plat->rx_sched_algorithm = MTL_RX_ALGORITHM_SP;
+
+ for (i = 0; i < plat->rx_queues_to_use; i++) {
+ plat->rx_queues_cfg[i].mode_to_use = MTL_QUEUE_DCB;
+ plat->rx_queues_cfg[i].chan = i;
+
+ /* Disable Priority config by default */
+ plat->rx_queues_cfg[i].use_prio = false;
+
+ /* Disable RX queues routing by default */
+ plat->rx_queues_cfg[i].pkt_route = 0x0;
+ }
+
+ for (i = 0; i < plat->tx_queues_to_use; i++) {
+ plat->tx_queues_cfg[i].mode_to_use = MTL_QUEUE_DCB;
+
+ /* Disable Priority config by default */
+ plat->tx_queues_cfg[i].use_prio = false;
+ }
+
+ /* FIFO size is 4096 bytes for 1 tx/rx queue */
+ plat->tx_fifo_size = plat->tx_queues_to_use * 4096;
+ plat->rx_fifo_size = plat->rx_queues_to_use * 4096;
+
+ plat->tx_sched_algorithm = MTL_TX_ALGORITHM_WRR;
+ plat->tx_queues_cfg[0].weight = 0x09;
+ plat->tx_queues_cfg[1].weight = 0x0A;
+ plat->tx_queues_cfg[2].weight = 0x0B;
+ plat->tx_queues_cfg[3].weight = 0x0C;
+ plat->tx_queues_cfg[4].weight = 0x0D;
+ plat->tx_queues_cfg[5].weight = 0x0E;
+ plat->tx_queues_cfg[6].weight = 0x0F;
+ plat->tx_queues_cfg[7].weight = 0x10;
+
+ plat->mdio_bus_data->phy_mask = 0;
+
+ plat->dma_cfg->pbl = 32;
+ plat->dma_cfg->pblx8 = true;
+ plat->dma_cfg->fixed_burst = 0;
+ plat->dma_cfg->mixed_burst = 0;
+ plat->dma_cfg->aal = 0;
+
+ plat->axi = devm_kzalloc(&pdev->dev, sizeof(*plat->axi),
+ GFP_KERNEL);
+ if (!plat->axi)
+ return -ENOMEM;
+
+ plat->axi->axi_lpi_en = 0;
+ plat->axi->axi_xit_frm = 0;
+ plat->axi->axi_wr_osr_lmt = 1;
+ plat->axi->axi_rd_osr_lmt = 1;
+ plat->axi->axi_blen[0] = 4;
+ plat->axi->axi_blen[1] = 8;
+ plat->axi->axi_blen[2] = 16;
+
+ plat->ptp_max_adj = plat->clk_ptp_rate;
+
+ /* Set system clock */
+ plat->stmmac_clk = clk_register_fixed_rate(&pdev->dev,
+ "stmmac-clk", NULL, 0,
+ plat->clk_ptp_rate);
+
+ if (IS_ERR(plat->stmmac_clk)) {
+ dev_warn(&pdev->dev, "Fail to register stmmac-clk\n");
+ plat->stmmac_clk = NULL;
+ }
+ clk_prepare_enable(plat->stmmac_clk);
+
+ /* Set default value for multicast hash bins */
+ plat->multicast_filter_bins = HASH_TABLE_SIZE;
+
+ /* Set default value for unicast filter entries */
+ plat->unicast_filter_entries = 1;
+
+ /* Set the maxmtu to a default of JUMBO_LEN */
+ plat->maxmtu = JUMBO_LEN;
+
+ return 0;
+}
+
+static int ehl_common_data(struct pci_dev *pdev,
+ struct plat_stmmacenet_data *plat)
+{
+ int ret;
+
+ plat->rx_queues_to_use = 8;
+ plat->tx_queues_to_use = 8;
+ plat->clk_ptp_rate = 200000000;
+ ret = intel_mgbe_common_data(pdev, plat);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int ehl_sgmii_data(struct pci_dev *pdev,
+ struct plat_stmmacenet_data *plat)
+{
+ plat->bus_id = 1;
+ plat->phy_addr = 0;
+ plat->interface = PHY_INTERFACE_MODE_SGMII;
+ return ehl_common_data(pdev, plat);
+}
+
+static struct stmmac_pci_info ehl_sgmii1g_pci_info = {
+ .setup = ehl_sgmii_data,
+};
+
+static int ehl_rgmii_data(struct pci_dev *pdev,
+ struct plat_stmmacenet_data *plat)
+{
+ plat->bus_id = 1;
+ plat->phy_addr = 0;
+ plat->interface = PHY_INTERFACE_MODE_RGMII;
+ return ehl_common_data(pdev, plat);
+}
+
+static struct stmmac_pci_info ehl_rgmii1g_pci_info = {
+ .setup = ehl_rgmii_data,
+};
+
+static int tgl_common_data(struct pci_dev *pdev,
+ struct plat_stmmacenet_data *plat)
+{
+ int ret;
+
+ plat->rx_queues_to_use = 6;
+ plat->tx_queues_to_use = 4;
+ plat->clk_ptp_rate = 200000000;
+ ret = intel_mgbe_common_data(pdev, plat);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int tgl_sgmii_data(struct pci_dev *pdev,
+ struct plat_stmmacenet_data *plat)
+{
+ plat->bus_id = 1;
+ plat->phy_addr = 0;
+ plat->interface = PHY_INTERFACE_MODE_SGMII;
+ return tgl_common_data(pdev, plat);
+}
+
+static struct stmmac_pci_info tgl_sgmii1g_pci_info = {
+ .setup = tgl_sgmii_data,
+};
+
static const struct stmmac_pci_func_data galileo_stmmac_func_data[] = {
{
.func = 6,
@@ -293,10 +454,15 @@ static int stmmac_pci_probe(struct pci_dev *pdev,
*/
static void stmmac_pci_remove(struct pci_dev *pdev)
{
+ struct net_device *ndev = dev_get_drvdata(&pdev->dev);
+ struct stmmac_priv *priv = netdev_priv(ndev);
int i;
stmmac_dvr_remove(&pdev->dev);
+ if (priv->plat->stmmac_clk)
+ clk_unregister_fixed_rate(priv->plat->stmmac_clk);
+
for (i = 0; i <= PCI_STD_RESOURCE_END; i++) {
if (pci_resource_len(pdev, i) == 0)
continue;
@@ -349,6 +515,9 @@ static SIMPLE_DEV_PM_OPS(stmmac_pm_ops, stmmac_pci_suspend, stmmac_pci_resume);
#define STMMAC_QUARK_ID 0x0937
#define STMMAC_DEVICE_ID 0x1108
+#define STMMAC_EHL_RGMII1G_ID 0x4b30
+#define STMMAC_EHL_SGMII1G_ID 0x4b31
+#define STMMAC_TGL_SGMII1G_ID 0xa0ac
#define STMMAC_DEVICE(vendor_id, dev_id, info) { \
PCI_VDEVICE(vendor_id, dev_id), \
@@ -359,6 +528,9 @@ static const struct pci_device_id stmmac_id_table[] = {
STMMAC_DEVICE(STMMAC, STMMAC_DEVICE_ID, stmmac_pci_info),
STMMAC_DEVICE(STMICRO, PCI_DEVICE_ID_STMICRO_MAC, stmmac_pci_info),
STMMAC_DEVICE(INTEL, STMMAC_QUARK_ID, quark_pci_info),
+ STMMAC_DEVICE(INTEL, STMMAC_EHL_RGMII1G_ID, ehl_rgmii1g_pci_info),
+ STMMAC_DEVICE(INTEL, STMMAC_EHL_SGMII1G_ID, ehl_sgmii1g_pci_info),
+ STMMAC_DEVICE(INTEL, STMMAC_TGL_SGMII1G_ID, tgl_sgmii1g_pci_info),
{}
};
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c
index eaf8f08..5de754a9 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c
@@ -23,6 +23,7 @@
/**
* dwmac1000_validate_mcast_bins - validates the number of Multicast filter bins
+ * @dev: struct device of the platform device
* @mcast_bins: Multicast filtering bins
* Description:
* this function validates the number of Multicast filtering bins specified
@@ -33,7 +34,7 @@
* invalid and will cause the filtering algorithm to use Multicast
* promiscuous mode.
*/
-static int dwmac1000_validate_mcast_bins(int mcast_bins)
+static int dwmac1000_validate_mcast_bins(struct device *dev, int mcast_bins)
{
int x = mcast_bins;
@@ -44,8 +45,8 @@ static int dwmac1000_validate_mcast_bins(int mcast_bins)
break;
default:
x = 0;
- pr_info("Hash table entries set to unexpected value %d",
- mcast_bins);
+ dev_info(dev, "Hash table entries set to unexpected value %d\n",
+ mcast_bins);
break;
}
return x;
@@ -53,6 +54,7 @@ static int dwmac1000_validate_mcast_bins(int mcast_bins)
/**
* dwmac1000_validate_ucast_entries - validate the Unicast address entries
+ * @dev: struct device of the platform device
* @ucast_entries: number of Unicast address entries
* Description:
* This function validates the number of Unicast address entries supported
@@ -62,7 +64,8 @@ static int dwmac1000_validate_mcast_bins(int mcast_bins)
* selected, and defaults to 1 Unicast address if an unsupported
* configuration is selected.
*/
-static int dwmac1000_validate_ucast_entries(int ucast_entries)
+static int dwmac1000_validate_ucast_entries(struct device *dev,
+ int ucast_entries)
{
int x = ucast_entries;
@@ -73,8 +76,8 @@ static int dwmac1000_validate_ucast_entries(int ucast_entries)
break;
default:
x = 1;
- pr_info("Unicast table entries set to unexpected value %d\n",
- ucast_entries);
+ dev_info(dev, "Unicast table entries set to unexpected value %d\n",
+ ucast_entries);
break;
}
return x;
@@ -463,9 +466,9 @@ stmmac_probe_config_dt(struct platform_device *pdev, const char **mac)
of_property_read_u32(np, "snps,perfect-filter-entries",
&plat->unicast_filter_entries);
plat->unicast_filter_entries = dwmac1000_validate_ucast_entries(
- plat->unicast_filter_entries);
+ &pdev->dev, plat->unicast_filter_entries);
plat->multicast_filter_bins = dwmac1000_validate_mcast_bins(
- plat->multicast_filter_bins);
+ &pdev->dev, plat->multicast_filter_bins);
plat->has_gmac = 1;
plat->pmt = 1;
}
@@ -514,7 +517,8 @@ stmmac_probe_config_dt(struct platform_device *pdev, const char **mac)
plat->force_thresh_dma_mode = of_property_read_bool(np, "snps,force_thresh_dma_mode");
if (plat->force_thresh_dma_mode) {
plat->force_sf_dma_mode = 0;
- pr_warn("force_sf_dma_mode is ignored if force_thresh_dma_mode is set.");
+ dev_warn(&pdev->dev,
+ "force_sf_dma_mode is ignored if force_thresh_dma_mode is set.\n");
}
of_property_read_u32(np, "snps,ps-speed", &plat->mac_port_sel_speed);
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
index c482249..173493d 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
@@ -194,6 +194,9 @@ void stmmac_ptp_register(struct stmmac_priv *priv)
priv->pps[i].available = true;
}
+ if (priv->plat->ptp_max_adj)
+ stmmac_ptp_clock_ops.max_adj = priv->plat->ptp_max_adj;
+
stmmac_ptp_clock_ops.n_per_out = priv->dma_cap.pps_out_num;
spin_lock_init(&priv->ptp_lock);
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
index ecc8602..c56e89e 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
@@ -43,9 +43,11 @@ struct stmmac_packet_attrs {
int dont_wait;
int timeout;
int size;
+ int max_size;
int remove_sa;
u8 id;
int sarc;
+ u16 queue_mapping;
};
static u8 stmmac_test_next_id;
@@ -73,12 +75,14 @@ static struct sk_buff *stmmac_test_get_udp_skb(struct stmmac_priv *priv,
else
size += sizeof(struct udphdr);
- skb = netdev_alloc_skb(priv->dev, size);
+ if (attr->max_size && (attr->max_size > size))
+ size = attr->max_size;
+
+ skb = netdev_alloc_skb_ip_align(priv->dev, size);
if (!skb)
return NULL;
prefetchw(skb->data);
- skb_reserve(skb, NET_IP_ALIGN);
if (attr->vlan > 1)
ehdr = skb_push(skb, ETH_HLEN + 8);
@@ -147,6 +151,9 @@ static struct sk_buff *stmmac_test_get_udp_skb(struct stmmac_priv *priv,
uhdr->source = htons(attr->sport);
uhdr->dest = htons(attr->dport);
uhdr->len = htons(sizeof(*shdr) + sizeof(*uhdr) + attr->size);
+ if (attr->max_size)
+ uhdr->len = htons(attr->max_size -
+ (sizeof(*ihdr) + sizeof(*ehdr)));
uhdr->check = 0;
}
@@ -162,9 +169,13 @@ static struct sk_buff *stmmac_test_get_udp_skb(struct stmmac_priv *priv,
iplen += sizeof(*thdr);
else
iplen += sizeof(*uhdr);
+
+ if (attr->max_size)
+ iplen = attr->max_size - sizeof(*ehdr);
+
ihdr->tot_len = htons(iplen);
ihdr->frag_off = 0;
- ihdr->saddr = 0;
+ ihdr->saddr = htonl(attr->ip_src);
ihdr->daddr = htonl(attr->ip_dst);
ihdr->tos = 0;
ihdr->id = 0;
@@ -178,6 +189,8 @@ static struct sk_buff *stmmac_test_get_udp_skb(struct stmmac_priv *priv,
if (attr->size)
skb_put(skb, attr->size);
+ if (attr->max_size && (attr->max_size > skb->len))
+ skb_put(skb, attr->max_size - skb->len);
skb->csum = 0;
skb->ip_summed = CHECKSUM_PARTIAL;
@@ -196,6 +209,24 @@ static struct sk_buff *stmmac_test_get_udp_skb(struct stmmac_priv *priv,
return skb;
}
+static struct sk_buff *stmmac_test_get_arp_skb(struct stmmac_priv *priv,
+ struct stmmac_packet_attrs *attr)
+{
+ __be32 ip_src = htonl(attr->ip_src);
+ __be32 ip_dst = htonl(attr->ip_dst);
+ struct sk_buff *skb = NULL;
+
+ skb = arp_create(ARPOP_REQUEST, ETH_P_ARP, ip_dst, priv->dev, ip_src,
+ NULL, attr->src, attr->dst);
+ if (!skb)
+ return NULL;
+
+ skb->pkt_type = PACKET_HOST;
+ skb->dev = priv->dev;
+
+ return skb;
+}
+
struct stmmac_test_priv {
struct stmmac_packet_attrs *packet;
struct packet_type pt;
@@ -306,7 +337,7 @@ static int __stmmac_test_loopback(struct stmmac_priv *priv,
goto cleanup;
}
- skb_set_queue_mapping(skb, 0);
+ skb_set_queue_mapping(skb, attr->queue_mapping);
ret = dev_queue_xmit(skb);
if (ret)
goto cleanup;
@@ -318,7 +349,7 @@ static int __stmmac_test_loopback(struct stmmac_priv *priv,
attr->timeout = STMMAC_LB_TIMEOUT;
wait_for_completion_timeout(&tpriv->comp, attr->timeout);
- ret = !tpriv->ok;
+ ret = tpriv->ok ? 0 : -ETIMEDOUT;
cleanup:
if (!attr->dont_wait)
@@ -480,7 +511,7 @@ static int stmmac_test_hfilt(struct stmmac_priv *priv)
/* Shall NOT receive packet */
ret = __stmmac_test_loopback(priv, &attr);
- ret = !ret;
+ ret = ret ? 0 : -EINVAL;
cleanup:
dev_mc_del(priv->dev, gd_addr);
@@ -512,7 +543,7 @@ static int stmmac_test_pfilt(struct stmmac_priv *priv)
/* Shall NOT receive packet */
ret = __stmmac_test_loopback(priv, &attr);
- ret = !ret;
+ ret = ret ? 0 : -EINVAL;
cleanup:
dev_uc_del(priv->dev, gd_addr);
@@ -562,7 +593,7 @@ static int stmmac_test_mcfilt(struct stmmac_priv *priv)
/* Shall NOT receive packet */
ret = __stmmac_test_loopback(priv, &attr);
- ret = !ret;
+ ret = ret ? 0 : -EINVAL;
cleanup:
dev_uc_del(priv->dev, uc_addr);
@@ -600,7 +631,7 @@ static int stmmac_test_ucfilt(struct stmmac_priv *priv)
/* Shall NOT receive packet */
ret = __stmmac_test_loopback(priv, &attr);
- ret = !ret;
+ ret = ret ? 0 : -EINVAL;
cleanup:
dev_mc_del(priv->dev, mc_addr);
@@ -691,15 +722,21 @@ static int stmmac_test_flowctrl(struct stmmac_priv *priv)
for (i = 0; i < rx_cnt; i++) {
struct stmmac_channel *ch = &priv->channel[i];
+ u32 tail;
+ tail = priv->rx_queue[i].dma_rx_phy +
+ (DMA_RX_SIZE * sizeof(struct dma_desc));
+
+ stmmac_set_rx_tail_ptr(priv, priv->ioaddr, tail, i);
stmmac_start_rx(priv, priv->ioaddr, i);
+
local_bh_disable();
napi_reschedule(&ch->rx_napi);
local_bh_enable();
}
wait_for_completion_timeout(&tpriv->comp, STMMAC_LB_TIMEOUT);
- ret = !tpriv->ok;
+ ret = tpriv->ok ? 0 : -ETIMEDOUT;
cleanup:
dev_mc_del(priv->dev, paddr);
@@ -833,11 +870,11 @@ static int stmmac_test_vlanfilt(struct stmmac_priv *priv)
goto vlan_del;
wait_for_completion_timeout(&tpriv->comp, STMMAC_LB_TIMEOUT);
- ret = !tpriv->ok;
+ ret = tpriv->ok ? 0 : -ETIMEDOUT;
if (ret && !i) {
goto vlan_del;
} else if (!ret && i) {
- ret = -1;
+ ret = -EINVAL;
goto vlan_del;
} else {
ret = 0;
@@ -909,11 +946,11 @@ static int stmmac_test_dvlanfilt(struct stmmac_priv *priv)
goto vlan_del;
wait_for_completion_timeout(&tpriv->comp, STMMAC_LB_TIMEOUT);
- ret = !tpriv->ok;
+ ret = tpriv->ok ? 0 : -ETIMEDOUT;
if (ret && !i) {
goto vlan_del;
} else if (!ret && i) {
- ret = -1;
+ ret = -EINVAL;
goto vlan_del;
} else {
ret = 0;
@@ -998,7 +1035,7 @@ static int stmmac_test_rxp(struct stmmac_priv *priv)
attr.src = addr;
ret = __stmmac_test_loopback(priv, &attr);
- ret = !ret; /* Shall NOT receive packet */
+ ret = ret ? 0 : -EINVAL; /* Shall NOT receive packet */
cls_u32.command = TC_CLSU32_DELETE_KNODE;
stmmac_tc_setup_cls_u32(priv, priv, &cls_u32);
@@ -1026,6 +1063,9 @@ static int stmmac_test_desc_sai(struct stmmac_priv *priv)
struct stmmac_packet_attrs attr = { };
int ret;
+ if (!priv->dma_cap.vlins)
+ return -EOPNOTSUPP;
+
attr.remove_sa = true;
attr.sarc = true;
attr.src = src;
@@ -1045,6 +1085,9 @@ static int stmmac_test_desc_sar(struct stmmac_priv *priv)
struct stmmac_packet_attrs attr = { };
int ret;
+ if (!priv->dma_cap.vlins)
+ return -EOPNOTSUPP;
+
attr.sarc = true;
attr.src = src;
attr.dst = priv->dev->dev_addr;
@@ -1063,6 +1106,9 @@ static int stmmac_test_reg_sai(struct stmmac_priv *priv)
struct stmmac_packet_attrs attr = { };
int ret;
+ if (!priv->dma_cap.vlins)
+ return -EOPNOTSUPP;
+
attr.remove_sa = true;
attr.sarc = true;
attr.src = src;
@@ -1083,6 +1129,9 @@ static int stmmac_test_reg_sar(struct stmmac_priv *priv)
struct stmmac_packet_attrs attr = { };
int ret;
+ if (!priv->dma_cap.vlins)
+ return -EOPNOTSUPP;
+
attr.sarc = true;
attr.src = src;
attr.dst = priv->dev->dev_addr;
@@ -1168,6 +1217,430 @@ static int stmmac_test_svlanoff(struct stmmac_priv *priv)
return stmmac_test_vlanoff_common(priv, true);
}
+#ifdef CONFIG_NET_CLS_ACT
+static int __stmmac_test_l3filt(struct stmmac_priv *priv, u32 dst, u32 src,
+ u32 dst_mask, u32 src_mask)
+{
+ struct flow_dissector_key_ipv4_addrs key, mask;
+ unsigned long dummy_cookie = 0xdeadbeef;
+ struct stmmac_packet_attrs attr = { };
+ struct flow_dissector *dissector;
+ struct flow_cls_offload *cls;
+ struct flow_rule *rule;
+ int ret;
+
+ if (!tc_can_offload(priv->dev))
+ return -EOPNOTSUPP;
+ if (!priv->dma_cap.l3l4fnum)
+ return -EOPNOTSUPP;
+ if (priv->rss.enable) {
+ struct stmmac_rss rss = { .enable = false, };
+
+ stmmac_rss_configure(priv, priv->hw, &rss,
+ priv->plat->rx_queues_to_use);
+ }
+
+ dissector = kzalloc(sizeof(*dissector), GFP_KERNEL);
+ if (!dissector) {
+ ret = -ENOMEM;
+ goto cleanup_rss;
+ }
+
+ dissector->used_keys |= (1 << FLOW_DISSECTOR_KEY_IPV4_ADDRS);
+ dissector->offset[FLOW_DISSECTOR_KEY_IPV4_ADDRS] = 0;
+
+ cls = kzalloc(sizeof(*cls), GFP_KERNEL);
+ if (!cls) {
+ ret = -ENOMEM;
+ goto cleanup_dissector;
+ }
+
+ cls->common.chain_index = 0;
+ cls->command = FLOW_CLS_REPLACE;
+ cls->cookie = dummy_cookie;
+
+ rule = kzalloc(struct_size(rule, action.entries, 1), GFP_KERNEL);
+ if (!rule) {
+ ret = -ENOMEM;
+ goto cleanup_cls;
+ }
+
+ rule->match.dissector = dissector;
+ rule->match.key = (void *)&key;
+ rule->match.mask = (void *)&mask;
+
+ key.src = htonl(src);
+ key.dst = htonl(dst);
+ mask.src = src_mask;
+ mask.dst = dst_mask;
+
+ cls->rule = rule;
+
+ rule->action.entries[0].id = FLOW_ACTION_DROP;
+ rule->action.num_entries = 1;
+
+ attr.dst = priv->dev->dev_addr;
+ attr.ip_dst = dst;
+ attr.ip_src = src;
+
+ /* Shall receive packet */
+ ret = __stmmac_test_loopback(priv, &attr);
+ if (ret)
+ goto cleanup_rule;
+
+ ret = stmmac_tc_setup_cls(priv, priv, cls);
+ if (ret)
+ goto cleanup_rule;
+
+ /* Shall NOT receive packet */
+ ret = __stmmac_test_loopback(priv, &attr);
+ ret = ret ? 0 : -EINVAL;
+
+ cls->command = FLOW_CLS_DESTROY;
+ stmmac_tc_setup_cls(priv, priv, cls);
+cleanup_rule:
+ kfree(rule);
+cleanup_cls:
+ kfree(cls);
+cleanup_dissector:
+ kfree(dissector);
+cleanup_rss:
+ if (priv->rss.enable) {
+ stmmac_rss_configure(priv, priv->hw, &priv->rss,
+ priv->plat->rx_queues_to_use);
+ }
+
+ return ret;
+}
+#else
+static int __stmmac_test_l3filt(struct stmmac_priv *priv, u32 dst, u32 src,
+ u32 dst_mask, u32 src_mask)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+
+static int stmmac_test_l3filt_da(struct stmmac_priv *priv)
+{
+ u32 addr = 0x10203040;
+
+ return __stmmac_test_l3filt(priv, addr, 0, ~0, 0);
+}
+
+static int stmmac_test_l3filt_sa(struct stmmac_priv *priv)
+{
+ u32 addr = 0x10203040;
+
+ return __stmmac_test_l3filt(priv, 0, addr, 0, ~0);
+}
+
+#ifdef CONFIG_NET_CLS_ACT
+static int __stmmac_test_l4filt(struct stmmac_priv *priv, u32 dst, u32 src,
+ u32 dst_mask, u32 src_mask, bool udp)
+{
+ struct {
+ struct flow_dissector_key_basic bkey;
+ struct flow_dissector_key_ports key;
+ } __aligned(BITS_PER_LONG / 8) keys;
+ struct {
+ struct flow_dissector_key_basic bmask;
+ struct flow_dissector_key_ports mask;
+ } __aligned(BITS_PER_LONG / 8) masks;
+ unsigned long dummy_cookie = 0xdeadbeef;
+ struct stmmac_packet_attrs attr = { };
+ struct flow_dissector *dissector;
+ struct flow_cls_offload *cls;
+ struct flow_rule *rule;
+ int ret;
+
+ if (!tc_can_offload(priv->dev))
+ return -EOPNOTSUPP;
+ if (!priv->dma_cap.l3l4fnum)
+ return -EOPNOTSUPP;
+ if (priv->rss.enable) {
+ struct stmmac_rss rss = { .enable = false, };
+
+ stmmac_rss_configure(priv, priv->hw, &rss,
+ priv->plat->rx_queues_to_use);
+ }
+
+ dissector = kzalloc(sizeof(*dissector), GFP_KERNEL);
+ if (!dissector) {
+ ret = -ENOMEM;
+ goto cleanup_rss;
+ }
+
+ dissector->used_keys |= (1 << FLOW_DISSECTOR_KEY_BASIC);
+ dissector->used_keys |= (1 << FLOW_DISSECTOR_KEY_PORTS);
+ dissector->offset[FLOW_DISSECTOR_KEY_BASIC] = 0;
+ dissector->offset[FLOW_DISSECTOR_KEY_PORTS] = offsetof(typeof(keys), key);
+
+ cls = kzalloc(sizeof(*cls), GFP_KERNEL);
+ if (!cls) {
+ ret = -ENOMEM;
+ goto cleanup_dissector;
+ }
+
+ cls->common.chain_index = 0;
+ cls->command = FLOW_CLS_REPLACE;
+ cls->cookie = dummy_cookie;
+
+ rule = kzalloc(struct_size(rule, action.entries, 1), GFP_KERNEL);
+ if (!rule) {
+ ret = -ENOMEM;
+ goto cleanup_cls;
+ }
+
+ rule->match.dissector = dissector;
+ rule->match.key = (void *)&keys;
+ rule->match.mask = (void *)&masks;
+
+ keys.bkey.ip_proto = udp ? IPPROTO_UDP : IPPROTO_TCP;
+ keys.key.src = htons(src);
+ keys.key.dst = htons(dst);
+ masks.mask.src = src_mask;
+ masks.mask.dst = dst_mask;
+
+ cls->rule = rule;
+
+ rule->action.entries[0].id = FLOW_ACTION_DROP;
+ rule->action.num_entries = 1;
+
+ attr.dst = priv->dev->dev_addr;
+ attr.tcp = !udp;
+ attr.sport = src;
+ attr.dport = dst;
+ attr.ip_dst = 0;
+
+ /* Shall receive packet */
+ ret = __stmmac_test_loopback(priv, &attr);
+ if (ret)
+ goto cleanup_rule;
+
+ ret = stmmac_tc_setup_cls(priv, priv, cls);
+ if (ret)
+ goto cleanup_rule;
+
+ /* Shall NOT receive packet */
+ ret = __stmmac_test_loopback(priv, &attr);
+ ret = ret ? 0 : -EINVAL;
+
+ cls->command = FLOW_CLS_DESTROY;
+ stmmac_tc_setup_cls(priv, priv, cls);
+cleanup_rule:
+ kfree(rule);
+cleanup_cls:
+ kfree(cls);
+cleanup_dissector:
+ kfree(dissector);
+cleanup_rss:
+ if (priv->rss.enable) {
+ stmmac_rss_configure(priv, priv->hw, &priv->rss,
+ priv->plat->rx_queues_to_use);
+ }
+
+ return ret;
+}
+#else
+static int __stmmac_test_l4filt(struct stmmac_priv *priv, u32 dst, u32 src,
+ u32 dst_mask, u32 src_mask, bool udp)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+
+static int stmmac_test_l4filt_da_tcp(struct stmmac_priv *priv)
+{
+ u16 dummy_port = 0x123;
+
+ return __stmmac_test_l4filt(priv, dummy_port, 0, ~0, 0, false);
+}
+
+static int stmmac_test_l4filt_sa_tcp(struct stmmac_priv *priv)
+{
+ u16 dummy_port = 0x123;
+
+ return __stmmac_test_l4filt(priv, 0, dummy_port, 0, ~0, false);
+}
+
+static int stmmac_test_l4filt_da_udp(struct stmmac_priv *priv)
+{
+ u16 dummy_port = 0x123;
+
+ return __stmmac_test_l4filt(priv, dummy_port, 0, ~0, 0, true);
+}
+
+static int stmmac_test_l4filt_sa_udp(struct stmmac_priv *priv)
+{
+ u16 dummy_port = 0x123;
+
+ return __stmmac_test_l4filt(priv, 0, dummy_port, 0, ~0, true);
+}
+
+static int stmmac_test_arp_validate(struct sk_buff *skb,
+ struct net_device *ndev,
+ struct packet_type *pt,
+ struct net_device *orig_ndev)
+{
+ struct stmmac_test_priv *tpriv = pt->af_packet_priv;
+ struct ethhdr *ehdr;
+ struct arphdr *ahdr;
+
+ ehdr = (struct ethhdr *)skb_mac_header(skb);
+ if (!ether_addr_equal(ehdr->h_dest, tpriv->packet->src))
+ goto out;
+
+ ahdr = arp_hdr(skb);
+ if (ahdr->ar_op != htons(ARPOP_REPLY))
+ goto out;
+
+ tpriv->ok = true;
+ complete(&tpriv->comp);
+out:
+ kfree_skb(skb);
+ return 0;
+}
+
+static int stmmac_test_arpoffload(struct stmmac_priv *priv)
+{
+ unsigned char src[ETH_ALEN] = {0x01, 0x02, 0x03, 0x04, 0x05, 0x06};
+ unsigned char dst[ETH_ALEN] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
+ struct stmmac_packet_attrs attr = { };
+ struct stmmac_test_priv *tpriv;
+ struct sk_buff *skb = NULL;
+ u32 ip_addr = 0xdeadcafe;
+ u32 ip_src = 0xdeadbeef;
+ int ret;
+
+ if (!priv->dma_cap.arpoffsel)
+ return -EOPNOTSUPP;
+
+ tpriv = kzalloc(sizeof(*tpriv), GFP_KERNEL);
+ if (!tpriv)
+ return -ENOMEM;
+
+ tpriv->ok = false;
+ init_completion(&tpriv->comp);
+
+ tpriv->pt.type = htons(ETH_P_ARP);
+ tpriv->pt.func = stmmac_test_arp_validate;
+ tpriv->pt.dev = priv->dev;
+ tpriv->pt.af_packet_priv = tpriv;
+ tpriv->packet = &attr;
+ dev_add_pack(&tpriv->pt);
+
+ attr.src = src;
+ attr.ip_src = ip_src;
+ attr.dst = dst;
+ attr.ip_dst = ip_addr;
+
+ skb = stmmac_test_get_arp_skb(priv, &attr);
+ if (!skb) {
+ ret = -ENOMEM;
+ goto cleanup;
+ }
+
+ ret = stmmac_set_arp_offload(priv, priv->hw, true, ip_addr);
+ if (ret)
+ goto cleanup;
+
+ ret = dev_set_promiscuity(priv->dev, 1);
+ if (ret)
+ goto cleanup;
+
+ skb_set_queue_mapping(skb, 0);
+ ret = dev_queue_xmit(skb);
+ if (ret)
+ goto cleanup_promisc;
+
+ wait_for_completion_timeout(&tpriv->comp, STMMAC_LB_TIMEOUT);
+ ret = tpriv->ok ? 0 : -ETIMEDOUT;
+
+cleanup_promisc:
+ dev_set_promiscuity(priv->dev, -1);
+cleanup:
+ stmmac_set_arp_offload(priv, priv->hw, false, 0x0);
+ dev_remove_pack(&tpriv->pt);
+ kfree(tpriv);
+ return ret;
+}
+
+static int __stmmac_test_jumbo(struct stmmac_priv *priv, u16 queue)
+{
+ struct stmmac_packet_attrs attr = { };
+ int size = priv->dma_buf_sz;
+
+ /* Only XGMAC has SW support for multiple RX descs in same packet */
+ if (priv->plat->has_xgmac)
+ size = priv->dev->max_mtu;
+
+ attr.dst = priv->dev->dev_addr;
+ attr.max_size = size - ETH_FCS_LEN;
+ attr.queue_mapping = queue;
+
+ return __stmmac_test_loopback(priv, &attr);
+}
+
+static int stmmac_test_jumbo(struct stmmac_priv *priv)
+{
+ return __stmmac_test_jumbo(priv, 0);
+}
+
+static int stmmac_test_mjumbo(struct stmmac_priv *priv)
+{
+ u32 chan, tx_cnt = priv->plat->tx_queues_to_use;
+ int ret;
+
+ if (tx_cnt <= 1)
+ return -EOPNOTSUPP;
+
+ for (chan = 0; chan < tx_cnt; chan++) {
+ ret = __stmmac_test_jumbo(priv, chan);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int stmmac_test_sph(struct stmmac_priv *priv)
+{
+ unsigned long cnt_end, cnt_start = priv->xstats.rx_split_hdr_pkt_n;
+ struct stmmac_packet_attrs attr = { };
+ int ret;
+
+ if (!priv->sph)
+ return -EOPNOTSUPP;
+
+ /* Check for UDP first */
+ attr.dst = priv->dev->dev_addr;
+ attr.tcp = false;
+
+ ret = __stmmac_test_loopback(priv, &attr);
+ if (ret)
+ return ret;
+
+ cnt_end = priv->xstats.rx_split_hdr_pkt_n;
+ if (cnt_end <= cnt_start)
+ return -EINVAL;
+
+ /* Check for TCP now */
+ cnt_start = cnt_end;
+
+ attr.dst = priv->dev->dev_addr;
+ attr.tcp = true;
+
+ ret = __stmmac_test_loopback(priv, &attr);
+ if (ret)
+ return ret;
+
+ cnt_end = priv->xstats.rx_split_hdr_pkt_n;
+ if (cnt_end <= cnt_start)
+ return -EINVAL;
+
+ return 0;
+}
+
#define STMMAC_LOOPBACK_NONE 0
#define STMMAC_LOOPBACK_MAC 1
#define STMMAC_LOOPBACK_PHY 2
@@ -1253,6 +1726,46 @@ static const struct stmmac_test {
.name = "SVLAN TX Insertion ",
.lb = STMMAC_LOOPBACK_PHY,
.fn = stmmac_test_svlanoff,
+ }, {
+ .name = "L3 DA Filtering ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_l3filt_da,
+ }, {
+ .name = "L3 SA Filtering ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_l3filt_sa,
+ }, {
+ .name = "L4 DA TCP Filtering ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_l4filt_da_tcp,
+ }, {
+ .name = "L4 SA TCP Filtering ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_l4filt_sa_tcp,
+ }, {
+ .name = "L4 DA UDP Filtering ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_l4filt_da_udp,
+ }, {
+ .name = "L4 SA UDP Filtering ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_l4filt_sa_udp,
+ }, {
+ .name = "ARP Offload ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_arpoffload,
+ }, {
+ .name = "Jumbo Frame ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_jumbo,
+ }, {
+ .name = "Multichannel Jumbo ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_mjumbo,
+ }, {
+ .name = "Split Header ",
+ .lb = STMMAC_LOOPBACK_PHY,
+ .fn = stmmac_test_sph,
},
};
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
index 6c305b6..e231098 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
@@ -242,9 +242,27 @@ static int tc_init(struct stmmac_priv *priv)
{
struct dma_features *dma_cap = &priv->dma_cap;
unsigned int count;
+ int i;
+ if (dma_cap->l3l4fnum) {
+ priv->flow_entries_max = dma_cap->l3l4fnum;
+ priv->flow_entries = devm_kcalloc(priv->device,
+ dma_cap->l3l4fnum,
+ sizeof(*priv->flow_entries),
+ GFP_KERNEL);
+ if (!priv->flow_entries)
+ return -ENOMEM;
+
+ for (i = 0; i < priv->flow_entries_max; i++)
+ priv->flow_entries[i].idx = i;
+
+ dev_info(priv->device, "Enabled Flow TC (entries=%d)\n",
+ priv->flow_entries_max);
+ }
+
+ /* Fail silently as we can still use remaining features, e.g. CBS */
if (!dma_cap->frpsel)
- return -EINVAL;
+ return 0;
switch (dma_cap->frpbs) {
case 0x0:
@@ -349,8 +367,235 @@ static int tc_setup_cbs(struct stmmac_priv *priv,
return 0;
}
+static int tc_parse_flow_actions(struct stmmac_priv *priv,
+ struct flow_action *action,
+ struct stmmac_flow_entry *entry)
+{
+ struct flow_action_entry *act;
+ int i;
+
+ if (!flow_action_has_entries(action))
+ return -EINVAL;
+
+ flow_action_for_each(i, act, action) {
+ switch (act->id) {
+ case FLOW_ACTION_DROP:
+ entry->action |= STMMAC_FLOW_ACTION_DROP;
+ return 0;
+ default:
+ break;
+ }
+ }
+
+ /* Nothing to do, maybe inverse filter ? */
+ return 0;
+}
+
+static int tc_add_basic_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls,
+ struct stmmac_flow_entry *entry)
+{
+ struct flow_rule *rule = flow_cls_offload_flow_rule(cls);
+ struct flow_dissector *dissector = rule->match.dissector;
+ struct flow_match_basic match;
+
+ /* Nothing to do here */
+ if (!dissector_uses_key(dissector, FLOW_DISSECTOR_KEY_BASIC))
+ return -EINVAL;
+
+ flow_rule_match_basic(rule, &match);
+ entry->ip_proto = match.key->ip_proto;
+ return 0;
+}
+
+static int tc_add_ip4_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls,
+ struct stmmac_flow_entry *entry)
+{
+ struct flow_rule *rule = flow_cls_offload_flow_rule(cls);
+ struct flow_dissector *dissector = rule->match.dissector;
+ bool inv = entry->action & STMMAC_FLOW_ACTION_DROP;
+ struct flow_match_ipv4_addrs match;
+ u32 hw_match;
+ int ret;
+
+ /* Nothing to do here */
+ if (!dissector_uses_key(dissector, FLOW_DISSECTOR_KEY_IPV4_ADDRS))
+ return -EINVAL;
+
+ flow_rule_match_ipv4_addrs(rule, &match);
+ hw_match = ntohl(match.key->src) & ntohl(match.mask->src);
+ if (hw_match) {
+ ret = stmmac_config_l3_filter(priv, priv->hw, entry->idx, true,
+ false, true, inv, hw_match);
+ if (ret)
+ return ret;
+ }
+
+ hw_match = ntohl(match.key->dst) & ntohl(match.mask->dst);
+ if (hw_match) {
+ ret = stmmac_config_l3_filter(priv, priv->hw, entry->idx, true,
+ false, false, inv, hw_match);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int tc_add_ports_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls,
+ struct stmmac_flow_entry *entry)
+{
+ struct flow_rule *rule = flow_cls_offload_flow_rule(cls);
+ struct flow_dissector *dissector = rule->match.dissector;
+ bool inv = entry->action & STMMAC_FLOW_ACTION_DROP;
+ struct flow_match_ports match;
+ u32 hw_match;
+ bool is_udp;
+ int ret;
+
+ /* Nothing to do here */
+ if (!dissector_uses_key(dissector, FLOW_DISSECTOR_KEY_PORTS))
+ return -EINVAL;
+
+ switch (entry->ip_proto) {
+ case IPPROTO_TCP:
+ is_udp = false;
+ break;
+ case IPPROTO_UDP:
+ is_udp = true;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ flow_rule_match_ports(rule, &match);
+
+ hw_match = ntohs(match.key->src) & ntohs(match.mask->src);
+ if (hw_match) {
+ ret = stmmac_config_l4_filter(priv, priv->hw, entry->idx, true,
+ is_udp, true, inv, hw_match);
+ if (ret)
+ return ret;
+ }
+
+ hw_match = ntohs(match.key->dst) & ntohs(match.mask->dst);
+ if (hw_match) {
+ ret = stmmac_config_l4_filter(priv, priv->hw, entry->idx, true,
+ is_udp, false, inv, hw_match);
+ if (ret)
+ return ret;
+ }
+
+ entry->is_l4 = true;
+ return 0;
+}
+
+static struct stmmac_flow_entry *tc_find_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls,
+ bool get_free)
+{
+ int i;
+
+ for (i = 0; i < priv->flow_entries_max; i++) {
+ struct stmmac_flow_entry *entry = &priv->flow_entries[i];
+
+ if (entry->cookie == cls->cookie)
+ return entry;
+ if (get_free && (entry->in_use == false))
+ return entry;
+ }
+
+ return NULL;
+}
+
+struct {
+ int (*fn)(struct stmmac_priv *priv, struct flow_cls_offload *cls,
+ struct stmmac_flow_entry *entry);
+} tc_flow_parsers[] = {
+ { .fn = tc_add_basic_flow },
+ { .fn = tc_add_ip4_flow },
+ { .fn = tc_add_ports_flow },
+};
+
+static int tc_add_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls)
+{
+ struct stmmac_flow_entry *entry = tc_find_flow(priv, cls, false);
+ struct flow_rule *rule = flow_cls_offload_flow_rule(cls);
+ int i, ret;
+
+ if (!entry) {
+ entry = tc_find_flow(priv, cls, true);
+ if (!entry)
+ return -ENOENT;
+ }
+
+ ret = tc_parse_flow_actions(priv, &rule->action, entry);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < ARRAY_SIZE(tc_flow_parsers); i++) {
+ ret = tc_flow_parsers[i].fn(priv, cls, entry);
+ if (!ret) {
+ entry->in_use = true;
+ continue;
+ }
+ }
+
+ if (!entry->in_use)
+ return -EINVAL;
+
+ entry->cookie = cls->cookie;
+ return 0;
+}
+
+static int tc_del_flow(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls)
+{
+ struct stmmac_flow_entry *entry = tc_find_flow(priv, cls, false);
+ int ret;
+
+ if (!entry || !entry->in_use)
+ return -ENOENT;
+
+ if (entry->is_l4) {
+ ret = stmmac_config_l4_filter(priv, priv->hw, entry->idx, false,
+ false, false, false, 0);
+ } else {
+ ret = stmmac_config_l3_filter(priv, priv->hw, entry->idx, false,
+ false, false, false, 0);
+ }
+
+ entry->in_use = false;
+ entry->cookie = 0;
+ entry->is_l4 = false;
+ return ret;
+}
+
+static int tc_setup_cls(struct stmmac_priv *priv,
+ struct flow_cls_offload *cls)
+{
+ int ret = 0;
+
+ switch (cls->command) {
+ case FLOW_CLS_REPLACE:
+ ret = tc_add_flow(priv, cls);
+ break;
+ case FLOW_CLS_DESTROY:
+ ret = tc_del_flow(priv, cls);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return ret;
+}
+
const struct stmmac_tc_ops dwmac510_tc_ops = {
.init = tc_init,
.setup_cls_u32 = tc_setup_cls_u32,
.setup_cbs = tc_setup_cbs,
+ .setup_cls = tc_setup_cls,
};
diff --git a/drivers/net/ethernet/sun/sunvnet_common.c b/drivers/net/ethernet/sun/sunvnet_common.c
index 646e672..8b94d9a 100644
--- a/drivers/net/ethernet/sun/sunvnet_common.c
+++ b/drivers/net/ethernet/sun/sunvnet_common.c
@@ -1532,8 +1532,7 @@ sunvnet_start_xmit_common(struct sk_buff *skb, struct net_device *dev,
else if (port)
del_timer(&port->clean_timer);
rcu_read_unlock();
- if (skb)
- dev_kfree_skb(skb);
+ dev_kfree_skb(skb);
vnet_free_skbs(freeskbs);
dev->stats.tx_dropped++;
return NETDEV_TX_OK;
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 5401095..f298d71 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -2775,6 +2775,7 @@ static int cpsw_probe(struct platform_device *pdev)
if (!cpsw)
return -ENOMEM;
+ platform_set_drvdata(pdev, cpsw);
cpsw->dev = dev;
mode = devm_gpiod_get_array_optional(dev, "mode", GPIOD_OUT_LOW);
@@ -2878,7 +2879,6 @@ static int cpsw_probe(struct platform_device *pdev)
goto clean_cpts;
}
- platform_set_drvdata(pdev, cpsw);
priv = netdev_priv(ndev);
priv->cpsw = cpsw;
priv->ndev = ndev;
diff --git a/drivers/net/ethernet/toshiba/spider_net.c b/drivers/net/ethernet/toshiba/spider_net.c
index 0f34676..538e708 100644
--- a/drivers/net/ethernet/toshiba/spider_net.c
+++ b/drivers/net/ethernet/toshiba/spider_net.c
@@ -2311,11 +2311,9 @@ spider_net_alloc_card(void)
{
struct net_device *netdev;
struct spider_net_card *card;
- size_t alloc_size;
- alloc_size = sizeof(struct spider_net_card) +
- (tx_descriptors + rx_descriptors) * sizeof(struct spider_net_descr);
- netdev = alloc_etherdev(alloc_size);
+ netdev = alloc_etherdev(struct_size(card, darray,
+ tx_descriptors + rx_descriptors));
if (!netdev)
return NULL;
diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c
index cb2ea8f..3ab24fd 100644
--- a/drivers/net/geneve.c
+++ b/drivers/net/geneve.c
@@ -1345,7 +1345,7 @@ static int geneve_nl2info(struct nlattr *tb[], struct nlattr *data[],
info->key.u.ipv4.dst =
nla_get_in_addr(data[IFLA_GENEVE_REMOTE]);
- if (IN_MULTICAST(ntohl(info->key.u.ipv4.dst))) {
+ if (ipv4_is_multicast(info->key.u.ipv4.dst)) {
NL_SET_ERR_MSG_ATTR(extack, data[IFLA_GENEVE_REMOTE],
"Remote IPv4 address cannot be Multicast");
return -EINVAL;
diff --git a/drivers/net/hamradio/baycom_epp.c b/drivers/net/hamradio/baycom_epp.c
index 9303aeb..4476491 100644
--- a/drivers/net/hamradio/baycom_epp.c
+++ b/drivers/net/hamradio/baycom_epp.c
@@ -961,8 +961,7 @@ static int epp_close(struct net_device *dev)
parport_write_control(pp, 0); /* reset the adapter */
parport_release(bc->pdev);
parport_unregister_device(bc->pdev);
- if (bc->skb)
- dev_kfree_skb(bc->skb);
+ dev_kfree_skb(bc->skb);
bc->skb = NULL;
printk(KERN_INFO "%s: close epp at iobase 0x%lx irq %u\n",
bc_drvname, dev->base_addr, dev->irq);
diff --git a/drivers/net/hamradio/hdlcdrv.c b/drivers/net/hamradio/hdlcdrv.c
index c6f83e0..df495b5 100644
--- a/drivers/net/hamradio/hdlcdrv.c
+++ b/drivers/net/hamradio/hdlcdrv.c
@@ -475,8 +475,7 @@ static int hdlcdrv_close(struct net_device *dev)
if (s->ops && s->ops->close)
i = s->ops->close(dev);
- if (s->skb)
- dev_kfree_skb(s->skb);
+ dev_kfree_skb(s->skb);
s->skb = NULL;
s->opened = 0;
return i;
diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
index ecc9af0..670ef68 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -822,7 +822,7 @@ struct nvsp_message {
#define NETVSC_SUPPORTED_HW_FEATURES (NETIF_F_RXCSUM | NETIF_F_IP_CSUM | \
NETIF_F_TSO | NETIF_F_IPV6_CSUM | \
- NETIF_F_TSO6 | NETIF_F_LRO)
+ NETIF_F_TSO6 | NETIF_F_LRO | NETIF_F_SG)
#define VRSS_SEND_TAB_SIZE 16 /* must be power of 2 */
#define VRSS_CHANNEL_MAX 64
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 0a6cd2f..39dddcd 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -1785,13 +1785,15 @@ static int netvsc_set_features(struct net_device *ndev,
netdev_features_t change = features ^ ndev->features;
struct net_device_context *ndevctx = netdev_priv(ndev);
struct netvsc_device *nvdev = rtnl_dereference(ndevctx->nvdev);
+ struct net_device *vf_netdev = rtnl_dereference(ndevctx->vf_netdev);
struct ndis_offload_params offloads;
+ int ret = 0;
if (!nvdev || nvdev->destroy)
return -ENODEV;
if (!(change & NETIF_F_LRO))
- return 0;
+ goto syncvf;
memset(&offloads, 0, sizeof(struct ndis_offload_params));
@@ -1803,7 +1805,19 @@ static int netvsc_set_features(struct net_device *ndev,
offloads.rsc_ip_v6 = NDIS_OFFLOAD_PARAMETERS_RSC_DISABLED;
}
- return rndis_filter_set_offload_params(ndev, nvdev, &offloads);
+ ret = rndis_filter_set_offload_params(ndev, nvdev, &offloads);
+
+ if (ret)
+ features ^= NETIF_F_LRO;
+
+syncvf:
+ if (!vf_netdev)
+ return ret;
+
+ vf_netdev->wanted_features = features;
+ netdev_update_features(vf_netdev);
+
+ return ret;
}
static u32 netvsc_get_msglevel(struct net_device *ndev)
@@ -2181,6 +2195,10 @@ static int netvsc_register_vf(struct net_device *vf_netdev)
dev_hold(vf_netdev);
rcu_assign_pointer(net_device_ctx->vf_netdev, vf_netdev);
+
+ vf_netdev->wanted_features = ndev->features;
+ netdev_update_features(vf_netdev);
+
return NOTIFY_OK;
}
@@ -2313,8 +2331,8 @@ static int netvsc_probe(struct hv_device *dev,
/* hw_features computed in rndis_netdev_set_hwcaps() */
net->features = net->hw_features |
- NETIF_F_HIGHDMA | NETIF_F_SG |
- NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX;
+ NETIF_F_HIGHDMA | NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_CTAG_RX;
net->vlan_features = net->features;
netdev_lockdep_set_classes(net);
diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
index 317dbe9..abaf815 100644
--- a/drivers/net/hyperv/rndis_filter.c
+++ b/drivers/net/hyperv/rndis_filter.c
@@ -1207,6 +1207,7 @@ static int rndis_netdev_set_hwcaps(struct rndis_device *rndis_device,
/* Compute tx offload settings based on hw capabilities */
net->hw_features |= NETIF_F_RXCSUM;
+ net->hw_features |= NETIF_F_SG;
if ((hwcaps.csum.ip4_txcsum & NDIS_TXCSUM_ALL_TCP4) == NDIS_TXCSUM_ALL_TCP4) {
/* Can checksum TCP */
diff --git a/drivers/net/ieee802154/mac802154_hwsim.c b/drivers/net/ieee802154/mac802154_hwsim.c
index b41696e..c20e7ef 100644
--- a/drivers/net/ieee802154/mac802154_hwsim.c
+++ b/drivers/net/ieee802154/mac802154_hwsim.c
@@ -802,7 +802,7 @@ static int hwsim_add_one(struct genl_info *info, struct device *dev,
err = hwsim_subscribe_all_others(phy);
if (err < 0) {
mutex_unlock(&hwsim_phys_lock);
- goto err_reg;
+ goto err_subscribe;
}
}
list_add_tail(&phy->list, &hwsim_phys);
@@ -812,6 +812,8 @@ static int hwsim_add_one(struct genl_info *info, struct device *dev,
return idx;
+err_subscribe:
+ ieee802154_unregister_hw(phy->hw);
err_reg:
kfree(pib);
err_pib:
@@ -901,9 +903,9 @@ static __init int hwsim_init_module(void)
return 0;
platform_drv:
- genl_unregister_family(&hwsim_genl_family);
-platform_dev:
platform_device_unregister(mac802154hwsim_dev);
+platform_dev:
+ genl_unregister_family(&hwsim_genl_family);
return rc;
}
diff --git a/drivers/net/phy/mdio-bcm-iproc.c b/drivers/net/phy/mdio-bcm-iproc.c
index 7d0f388..7e9975d 100644
--- a/drivers/net/phy/mdio-bcm-iproc.c
+++ b/drivers/net/phy/mdio-bcm-iproc.c
@@ -123,15 +123,13 @@ static int iproc_mdio_probe(struct platform_device *pdev)
{
struct iproc_mdio_priv *priv;
struct mii_bus *bus;
- struct resource *res;
int rc;
priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- priv->base = devm_ioremap_resource(&pdev->dev, res);
+ priv->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(priv->base)) {
dev_err(&pdev->dev, "failed to ioremap register\n");
return PTR_ERR(priv->base);
diff --git a/drivers/net/phy/mdio-hisi-femac.c b/drivers/net/phy/mdio-hisi-femac.c
index 287f3cc..f231c2f 100644
--- a/drivers/net/phy/mdio-hisi-femac.c
+++ b/drivers/net/phy/mdio-hisi-femac.c
@@ -74,7 +74,6 @@ static int hisi_femac_mdio_probe(struct platform_device *pdev)
struct device_node *np = pdev->dev.of_node;
struct mii_bus *bus;
struct hisi_femac_mdio_data *data;
- struct resource *res;
int ret;
bus = mdiobus_alloc_size(sizeof(*data));
@@ -88,8 +87,7 @@ static int hisi_femac_mdio_probe(struct platform_device *pdev)
bus->parent = &pdev->dev;
data = bus->priv;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- data->membase = devm_ioremap_resource(&pdev->dev, res);
+ data->membase = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(data->membase)) {
ret = PTR_ERR(data->membase);
goto err_out_free_mdiobus;
diff --git a/drivers/net/phy/mdio-moxart.c b/drivers/net/phy/mdio-moxart.c
index af3910f..2d16fc4 100644
--- a/drivers/net/phy/mdio-moxart.c
+++ b/drivers/net/phy/mdio-moxart.c
@@ -113,7 +113,6 @@ static int moxart_mdio_probe(struct platform_device *pdev)
struct device_node *np = pdev->dev.of_node;
struct mii_bus *bus;
struct moxart_mdio_data *data;
- struct resource *res;
int ret, i;
bus = mdiobus_alloc_size(sizeof(*data));
@@ -138,8 +137,7 @@ static int moxart_mdio_probe(struct platform_device *pdev)
bus->irq[i] = PHY_IGNORE_INTERRUPT;
data = bus->priv;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- data->base = devm_ioremap_resource(&pdev->dev, res);
+ data->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(data->base)) {
ret = PTR_ERR(data->base);
goto err_out_free_mdiobus;
diff --git a/drivers/net/phy/mdio-mux-meson-g12a.c b/drivers/net/phy/mdio-mux-meson-g12a.c
index 6644762..7a9ad54 100644
--- a/drivers/net/phy/mdio-mux-meson-g12a.c
+++ b/drivers/net/phy/mdio-mux-meson-g12a.c
@@ -302,7 +302,6 @@ static int g12a_mdio_mux_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct g12a_mdio_mux *priv;
- struct resource *res;
int ret;
priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
@@ -311,8 +310,7 @@ static int g12a_mdio_mux_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, priv);
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- priv->regs = devm_ioremap_resource(dev, res);
+ priv->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(priv->regs))
return PTR_ERR(priv->regs);
diff --git a/drivers/net/phy/mdio-sun4i.c b/drivers/net/phy/mdio-sun4i.c
index 20ffd8f..58d6504 100644
--- a/drivers/net/phy/mdio-sun4i.c
+++ b/drivers/net/phy/mdio-sun4i.c
@@ -92,7 +92,6 @@ static int sun4i_mdio_probe(struct platform_device *pdev)
struct device_node *np = pdev->dev.of_node;
struct mii_bus *bus;
struct sun4i_mdio_data *data;
- struct resource *res;
int ret;
bus = mdiobus_alloc_size(sizeof(*data));
@@ -106,8 +105,7 @@ static int sun4i_mdio_probe(struct platform_device *pdev)
bus->parent = &pdev->dev;
data = bus->priv;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- data->membase = devm_ioremap_resource(&pdev->dev, res);
+ data->membase = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(data->membase)) {
ret = PTR_ERR(data->membase);
goto err_out_free_mdiobus;
diff --git a/drivers/net/phy/phy-c45.c b/drivers/net/phy/phy-c45.c
index 58bb25e..7935593 100644
--- a/drivers/net/phy/phy-c45.c
+++ b/drivers/net/phy/phy-c45.c
@@ -523,6 +523,32 @@ int genphy_c45_read_status(struct phy_device *phydev)
}
EXPORT_SYMBOL_GPL(genphy_c45_read_status);
+/**
+ * genphy_c45_config_aneg - restart auto-negotiation or forced setup
+ * @phydev: target phy_device struct
+ *
+ * Description: If auto-negotiation is enabled, we configure the
+ * advertising, and then restart auto-negotiation. If it is not
+ * enabled, then we force a configuration.
+ */
+int genphy_c45_config_aneg(struct phy_device *phydev)
+{
+ bool changed = false;
+ int ret;
+
+ if (phydev->autoneg == AUTONEG_DISABLE)
+ return genphy_c45_pma_setup_forced(phydev);
+
+ ret = genphy_c45_an_config_aneg(phydev);
+ if (ret < 0)
+ return ret;
+ if (ret > 0)
+ changed = true;
+
+ return genphy_c45_check_and_restart_aneg(phydev, changed);
+}
+EXPORT_SYMBOL_GPL(genphy_c45_config_aneg);
+
/* The gen10g_* functions are the old Clause 45 stub */
int gen10g_config_aneg(struct phy_device *phydev)
diff --git a/drivers/net/phy/phy.c b/drivers/net/phy/phy.c
index f3adea9..7c92afd 100644
--- a/drivers/net/phy/phy.c
+++ b/drivers/net/phy/phy.c
@@ -507,7 +507,7 @@ static int phy_config_aneg(struct phy_device *phydev)
* allowed to call genphy_config_aneg()
*/
if (phydev->is_c45 && !(phydev->c45_ids.devices_in_package & BIT(0)))
- return -EOPNOTSUPP;
+ return genphy_c45_config_aneg(phydev);
return genphy_config_aneg(phydev);
}
@@ -525,6 +525,12 @@ static int phy_check_link_status(struct phy_device *phydev)
WARN_ON(!mutex_is_locked(&phydev->lock));
+ /* Keep previous state if loopback is enabled because some PHYs
+ * report that Link is Down when loopback is enabled.
+ */
+ if (phydev->loopback_enabled)
+ return 0;
+
err = phy_read_status(phydev);
if (err)
return err;
@@ -911,8 +917,8 @@ void phy_state_machine(struct work_struct *work)
if (phydev->link) {
phydev->link = 0;
phy_link_down(phydev, true);
- do_suspend = true;
}
+ do_suspend = true;
break;
}
diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index e36c04c..272d577 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -429,6 +429,7 @@ static umode_t sfp_hwmon_is_visible(const void *data,
return 0;
/* fall through */
case hwmon_temp_input:
+ case hwmon_temp_label:
return 0444;
default:
return 0;
@@ -447,6 +448,7 @@ static umode_t sfp_hwmon_is_visible(const void *data,
return 0;
/* fall through */
case hwmon_in_input:
+ case hwmon_in_label:
return 0444;
default:
return 0;
@@ -465,6 +467,7 @@ static umode_t sfp_hwmon_is_visible(const void *data,
return 0;
/* fall through */
case hwmon_curr_input:
+ case hwmon_curr_label:
return 0444;
default:
return 0;
@@ -492,6 +495,7 @@ static umode_t sfp_hwmon_is_visible(const void *data,
return 0;
/* fall through */
case hwmon_power_input:
+ case hwmon_power_label:
return 0444;
default:
return 0;
@@ -987,9 +991,63 @@ static int sfp_hwmon_read(struct device *dev, enum hwmon_sensor_types type,
}
}
+static const char *const sfp_hwmon_power_labels[] = {
+ "TX_power",
+ "RX_power",
+};
+
+static int sfp_hwmon_read_string(struct device *dev,
+ enum hwmon_sensor_types type,
+ u32 attr, int channel, const char **str)
+{
+ switch (type) {
+ case hwmon_curr:
+ switch (attr) {
+ case hwmon_curr_label:
+ *str = "bias";
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+ break;
+ case hwmon_temp:
+ switch (attr) {
+ case hwmon_temp_label:
+ *str = "temperature";
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+ break;
+ case hwmon_in:
+ switch (attr) {
+ case hwmon_in_label:
+ *str = "VCC";
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+ break;
+ case hwmon_power:
+ switch (attr) {
+ case hwmon_power_label:
+ *str = sfp_hwmon_power_labels[channel];
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ return -EOPNOTSUPP;
+}
+
static const struct hwmon_ops sfp_hwmon_ops = {
.is_visible = sfp_hwmon_is_visible,
.read = sfp_hwmon_read,
+ .read_string = sfp_hwmon_read_string,
};
static u32 sfp_hwmon_chip_config[] = {
@@ -1007,7 +1065,8 @@ static u32 sfp_hwmon_temp_config[] = {
HWMON_T_MAX | HWMON_T_MIN |
HWMON_T_MAX_ALARM | HWMON_T_MIN_ALARM |
HWMON_T_CRIT | HWMON_T_LCRIT |
- HWMON_T_CRIT_ALARM | HWMON_T_LCRIT_ALARM,
+ HWMON_T_CRIT_ALARM | HWMON_T_LCRIT_ALARM |
+ HWMON_T_LABEL,
0,
};
@@ -1021,7 +1080,8 @@ static u32 sfp_hwmon_vcc_config[] = {
HWMON_I_MAX | HWMON_I_MIN |
HWMON_I_MAX_ALARM | HWMON_I_MIN_ALARM |
HWMON_I_CRIT | HWMON_I_LCRIT |
- HWMON_I_CRIT_ALARM | HWMON_I_LCRIT_ALARM,
+ HWMON_I_CRIT_ALARM | HWMON_I_LCRIT_ALARM |
+ HWMON_I_LABEL,
0,
};
@@ -1035,7 +1095,8 @@ static u32 sfp_hwmon_bias_config[] = {
HWMON_C_MAX | HWMON_C_MIN |
HWMON_C_MAX_ALARM | HWMON_C_MIN_ALARM |
HWMON_C_CRIT | HWMON_C_LCRIT |
- HWMON_C_CRIT_ALARM | HWMON_C_LCRIT_ALARM,
+ HWMON_C_CRIT_ALARM | HWMON_C_LCRIT_ALARM |
+ HWMON_C_LABEL,
0,
};
@@ -1050,13 +1111,15 @@ static u32 sfp_hwmon_power_config[] = {
HWMON_P_MAX | HWMON_P_MIN |
HWMON_P_MAX_ALARM | HWMON_P_MIN_ALARM |
HWMON_P_CRIT | HWMON_P_LCRIT |
- HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM,
+ HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM |
+ HWMON_P_LABEL,
/* Receive power */
HWMON_P_INPUT |
HWMON_P_MAX | HWMON_P_MIN |
HWMON_P_MAX_ALARM | HWMON_P_MIN_ALARM |
HWMON_P_CRIT | HWMON_P_LCRIT |
- HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM,
+ HWMON_P_CRIT_ALARM | HWMON_P_LCRIT_ALARM |
+ HWMON_P_LABEL,
0,
};
diff --git a/drivers/net/phy/xilinx_gmii2rgmii.c b/drivers/net/phy/xilinx_gmii2rgmii.c
index 2d14493..151c2a3 100644
--- a/drivers/net/phy/xilinx_gmii2rgmii.c
+++ b/drivers/net/phy/xilinx_gmii2rgmii.c
@@ -29,7 +29,7 @@ struct gmii2rgmii {
static int xgmiitorgmii_read_status(struct phy_device *phydev)
{
- struct gmii2rgmii *priv = phydev->priv;
+ struct gmii2rgmii *priv = mdiodev_get_drvdata(&phydev->mdio);
struct mii_bus *bus = priv->mdio->bus;
int addr = priv->mdio->addr;
u16 val = 0;
@@ -90,7 +90,7 @@ static int xgmiitorgmii_probe(struct mdio_device *mdiodev)
memcpy(&priv->conv_phy_drv, priv->phy_dev->drv,
sizeof(struct phy_driver));
priv->conv_phy_drv.read_status = xgmiitorgmii_read_status;
- priv->phy_dev->priv = priv;
+ mdiodev_set_drvdata(&priv->phy_dev->mdio, priv);
priv->phy_dev->drv = &priv->conv_phy_drv;
return 0;
diff --git a/drivers/net/usb/lg-vl600.c b/drivers/net/usb/lg-vl600.c
index 6c2b3e3..217a2d8 100644
--- a/drivers/net/usb/lg-vl600.c
+++ b/drivers/net/usb/lg-vl600.c
@@ -87,9 +87,7 @@ static void vl600_unbind(struct usbnet *dev, struct usb_interface *intf)
{
struct vl600_state *s = dev->driver_priv;
- if (s->current_rx_buf)
- dev_kfree_skb(s->current_rx_buf);
-
+ dev_kfree_skb(s->current_rx_buf);
kfree(s);
return usbnet_cdc_unbind(dev, intf);
diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index 1aa6161..0872609 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -445,18 +445,18 @@
#define UPS_FLAGS_250M_CKDIV BIT(2)
#define UPS_FLAGS_EN_ALDPS BIT(3)
#define UPS_FLAGS_CTAP_SHORT_DIS BIT(4)
-#define UPS_FLAGS_SPEED_MASK (0xf << 16)
#define ups_flags_speed(x) ((x) << 16)
#define UPS_FLAGS_EN_EEE BIT(20)
#define UPS_FLAGS_EN_500M_EEE BIT(21)
#define UPS_FLAGS_EN_EEE_CKDIV BIT(22)
+#define UPS_FLAGS_EEE_PLLOFF_100 BIT(23)
#define UPS_FLAGS_EEE_PLLOFF_GIGA BIT(24)
#define UPS_FLAGS_EEE_CMOD_LV_EN BIT(25)
#define UPS_FLAGS_EN_GREEN BIT(26)
#define UPS_FLAGS_EN_FLOW_CTR BIT(27)
enum spd_duplex {
- NWAY_10M_HALF = 1,
+ NWAY_10M_HALF,
NWAY_10M_FULL,
NWAY_100M_HALF,
NWAY_100M_FULL,
@@ -749,19 +749,39 @@ struct r8152 {
void (*autosuspend_en)(struct r8152 *tp, bool enable);
} rtl_ops;
+ struct ups_info {
+ u32 _10m_ckdiv:1;
+ u32 _250m_ckdiv:1;
+ u32 aldps:1;
+ u32 lite_mode:2;
+ u32 speed_duplex:4;
+ u32 eee:1;
+ u32 eee_lite:1;
+ u32 eee_ckdiv:1;
+ u32 eee_plloff_100:1;
+ u32 eee_plloff_giga:1;
+ u32 eee_cmod_lv:1;
+ u32 green:1;
+ u32 flow_control:1;
+ u32 ctap_short_off:1;
+ } ups_info;
+
atomic_t rx_count;
+ bool eee_en;
int intr_interval;
u32 saved_wolopts;
u32 msg_enable;
u32 tx_qlen;
u32 coalesce;
+ u32 advertising;
u32 rx_buf_sz;
u32 rx_copybreak;
u32 rx_pending;
u16 ocp_base;
u16 speed;
+ u16 eee_adv;
u8 *intr_buff;
u8 version;
u8 duplex;
@@ -788,6 +808,13 @@ enum tx_csum_stat {
TX_CSUM_NONE
};
+#define RTL_ADVERTISED_10_HALF BIT(0)
+#define RTL_ADVERTISED_10_FULL BIT(1)
+#define RTL_ADVERTISED_100_HALF BIT(2)
+#define RTL_ADVERTISED_100_FULL BIT(3)
+#define RTL_ADVERTISED_1000_HALF BIT(4)
+#define RTL_ADVERTISED_1000_FULL BIT(5)
+
/* Maximum number of multicast addresses to filter (vs. Rx-all-multicast).
* The RTL chips use a 64 element hash table based on the Ethernet CRC.
*/
@@ -810,8 +837,11 @@ int get_registers(struct r8152 *tp, u16 value, u16 index, u16 size, void *data)
ret = usb_control_msg(tp->udev, usb_rcvctrlpipe(tp->udev, 0),
RTL8152_REQ_GET_REGS, RTL8152_REQT_READ,
value, index, tmp, size, 500);
+ if (ret < 0)
+ memset(data, 0xff, size);
+ else
+ memcpy(data, tmp, size);
- memcpy(data, tmp, size);
kfree(tmp);
return ret;
@@ -2852,14 +2882,76 @@ static void r8153_u2p3en(struct r8152 *tp, bool enable)
ocp_write_word(tp, MCU_TYPE_USB, USB_U2P3_CTRL, ocp_data);
}
-static void r8153b_ups_flags_w1w0(struct r8152 *tp, u32 set, u32 clear)
+static void r8153b_ups_flags(struct r8152 *tp)
{
- u32 ocp_data;
+ u32 ups_flags = 0;
- ocp_data = ocp_read_dword(tp, MCU_TYPE_USB, USB_UPS_FLAGS);
- ocp_data &= ~clear;
- ocp_data |= set;
- ocp_write_dword(tp, MCU_TYPE_USB, USB_UPS_FLAGS, ocp_data);
+ if (tp->ups_info.green)
+ ups_flags |= UPS_FLAGS_EN_GREEN;
+
+ if (tp->ups_info.aldps)
+ ups_flags |= UPS_FLAGS_EN_ALDPS;
+
+ if (tp->ups_info.eee)
+ ups_flags |= UPS_FLAGS_EN_EEE;
+
+ if (tp->ups_info.flow_control)
+ ups_flags |= UPS_FLAGS_EN_FLOW_CTR;
+
+ if (tp->ups_info.eee_ckdiv)
+ ups_flags |= UPS_FLAGS_EN_EEE_CKDIV;
+
+ if (tp->ups_info.eee_cmod_lv)
+ ups_flags |= UPS_FLAGS_EEE_CMOD_LV_EN;
+
+ if (tp->ups_info._10m_ckdiv)
+ ups_flags |= UPS_FLAGS_EN_10M_CKDIV;
+
+ if (tp->ups_info.eee_plloff_100)
+ ups_flags |= UPS_FLAGS_EEE_PLLOFF_100;
+
+ if (tp->ups_info.eee_plloff_giga)
+ ups_flags |= UPS_FLAGS_EEE_PLLOFF_GIGA;
+
+ if (tp->ups_info._250m_ckdiv)
+ ups_flags |= UPS_FLAGS_250M_CKDIV;
+
+ if (tp->ups_info.ctap_short_off)
+ ups_flags |= UPS_FLAGS_CTAP_SHORT_DIS;
+
+ switch (tp->ups_info.speed_duplex) {
+ case NWAY_10M_HALF:
+ ups_flags |= ups_flags_speed(1);
+ break;
+ case NWAY_10M_FULL:
+ ups_flags |= ups_flags_speed(2);
+ break;
+ case NWAY_100M_HALF:
+ ups_flags |= ups_flags_speed(3);
+ break;
+ case NWAY_100M_FULL:
+ ups_flags |= ups_flags_speed(4);
+ break;
+ case NWAY_1000M_FULL:
+ ups_flags |= ups_flags_speed(5);
+ break;
+ case FORCE_10M_HALF:
+ ups_flags |= ups_flags_speed(6);
+ break;
+ case FORCE_10M_FULL:
+ ups_flags |= ups_flags_speed(7);
+ break;
+ case FORCE_100M_HALF:
+ ups_flags |= ups_flags_speed(8);
+ break;
+ case FORCE_100M_FULL:
+ ups_flags |= ups_flags_speed(9);
+ break;
+ default:
+ break;
+ }
+
+ ocp_write_dword(tp, MCU_TYPE_USB, USB_UPS_FLAGS, ups_flags);
}
static void r8153b_green_en(struct r8152 *tp, bool enable)
@@ -2880,7 +2972,7 @@ static void r8153b_green_en(struct r8152 *tp, bool enable)
data |= GREEN_ETH_EN;
sram_write(tp, SRAM_GREEN_CFG, data);
- r8153b_ups_flags_w1w0(tp, UPS_FLAGS_EN_GREEN, 0);
+ tp->ups_info.green = enable;
}
static u16 r8153_phy_status(struct r8152 *tp, u16 desired)
@@ -2910,6 +3002,8 @@ static void r8153b_ups_en(struct r8152 *tp, bool enable)
u32 ocp_data = ocp_read_byte(tp, MCU_TYPE_USB, USB_POWER_CUT);
if (enable) {
+ r8153b_ups_flags(tp);
+
ocp_data |= UPS_EN | USP_PREWAKE | PHASE2_EN;
ocp_write_byte(tp, MCU_TYPE_USB, USB_POWER_CUT, ocp_data);
@@ -3200,10 +3294,60 @@ static void r8152_eee_en(struct r8152 *tp, bool enable)
ocp_reg_write(tp, OCP_EEE_CONFIG3, config3);
}
-static void r8152b_enable_eee(struct r8152 *tp)
+static void r8153_eee_en(struct r8152 *tp, bool enable)
{
- r8152_eee_en(tp, true);
- r8152_mmd_write(tp, MDIO_MMD_AN, MDIO_AN_EEE_ADV, MDIO_EEE_100TX);
+ u32 ocp_data;
+ u16 config;
+
+ ocp_data = ocp_read_word(tp, MCU_TYPE_PLA, PLA_EEE_CR);
+ config = ocp_reg_read(tp, OCP_EEE_CFG);
+
+ if (enable) {
+ ocp_data |= EEE_RX_EN | EEE_TX_EN;
+ config |= EEE10_EN;
+ } else {
+ ocp_data &= ~(EEE_RX_EN | EEE_TX_EN);
+ config &= ~EEE10_EN;
+ }
+
+ ocp_write_word(tp, MCU_TYPE_PLA, PLA_EEE_CR, ocp_data);
+ ocp_reg_write(tp, OCP_EEE_CFG, config);
+
+ tp->ups_info.eee = enable;
+}
+
+static void rtl_eee_enable(struct r8152 *tp, bool enable)
+{
+ switch (tp->version) {
+ case RTL_VER_01:
+ case RTL_VER_02:
+ case RTL_VER_07:
+ if (enable) {
+ r8152_eee_en(tp, true);
+ r8152_mmd_write(tp, MDIO_MMD_AN, MDIO_AN_EEE_ADV,
+ tp->eee_adv);
+ } else {
+ r8152_eee_en(tp, false);
+ r8152_mmd_write(tp, MDIO_MMD_AN, MDIO_AN_EEE_ADV, 0);
+ }
+ break;
+ case RTL_VER_03:
+ case RTL_VER_04:
+ case RTL_VER_05:
+ case RTL_VER_06:
+ case RTL_VER_08:
+ case RTL_VER_09:
+ if (enable) {
+ r8153_eee_en(tp, true);
+ ocp_reg_write(tp, OCP_EEE_ADV, tp->eee_adv);
+ } else {
+ r8153_eee_en(tp, false);
+ ocp_reg_write(tp, OCP_EEE_ADV, 0);
+ }
+ break;
+ default:
+ break;
+ }
}
static void r8152b_enable_fc(struct r8152 *tp)
@@ -3213,6 +3357,8 @@ static void r8152b_enable_fc(struct r8152 *tp)
anar = r8152_mdio_read(tp, MII_ADVERTISE);
anar |= ADVERTISE_PAUSE_CAP | ADVERTISE_PAUSE_ASYM;
r8152_mdio_write(tp, MII_ADVERTISE, anar);
+
+ tp->ups_info.flow_control = true;
}
static void rtl8152_disable(struct r8152 *tp)
@@ -3224,7 +3370,7 @@ static void rtl8152_disable(struct r8152 *tp)
static void r8152b_hw_phy_cfg(struct r8152 *tp)
{
- r8152b_enable_eee(tp);
+ rtl_eee_enable(tp, tp->eee_en);
r8152_aldps_en(tp, true);
r8152b_enable_fc(tp);
@@ -3406,52 +3552,8 @@ static void r8153_aldps_en(struct r8152 *tp, bool enable)
break;
}
}
-}
-static void r8153b_aldps_en(struct r8152 *tp, bool enable)
-{
- r8153_aldps_en(tp, enable);
-
- if (enable)
- r8153b_ups_flags_w1w0(tp, UPS_FLAGS_EN_ALDPS, 0);
- else
- r8153b_ups_flags_w1w0(tp, 0, UPS_FLAGS_EN_ALDPS);
-}
-
-static void r8153_eee_en(struct r8152 *tp, bool enable)
-{
- u32 ocp_data;
- u16 config;
-
- ocp_data = ocp_read_word(tp, MCU_TYPE_PLA, PLA_EEE_CR);
- config = ocp_reg_read(tp, OCP_EEE_CFG);
-
- if (enable) {
- ocp_data |= EEE_RX_EN | EEE_TX_EN;
- config |= EEE10_EN;
- } else {
- ocp_data &= ~(EEE_RX_EN | EEE_TX_EN);
- config &= ~EEE10_EN;
- }
-
- ocp_write_word(tp, MCU_TYPE_PLA, PLA_EEE_CR, ocp_data);
- ocp_reg_write(tp, OCP_EEE_CFG, config);
-}
-
-static void r8153b_eee_en(struct r8152 *tp, bool enable)
-{
- r8153_eee_en(tp, enable);
-
- if (enable)
- r8153b_ups_flags_w1w0(tp, UPS_FLAGS_EN_EEE, 0);
- else
- r8153b_ups_flags_w1w0(tp, 0, UPS_FLAGS_EN_EEE);
-}
-
-static void r8153b_enable_fc(struct r8152 *tp)
-{
- r8152b_enable_fc(tp);
- r8153b_ups_flags_w1w0(tp, UPS_FLAGS_EN_FLOW_CTR, 0);
+ tp->ups_info.aldps = enable;
}
static void r8153_hw_phy_cfg(struct r8152 *tp)
@@ -3463,8 +3565,7 @@ static void r8153_hw_phy_cfg(struct r8152 *tp)
r8153_aldps_en(tp, false);
/* disable EEE before updating the PHY parameters */
- r8153_eee_en(tp, false);
- ocp_reg_write(tp, OCP_EEE_ADV, 0);
+ rtl_eee_enable(tp, false);
if (tp->version == RTL_VER_03) {
data = ocp_reg_read(tp, OCP_EEE_CFG);
@@ -3495,8 +3596,8 @@ static void r8153_hw_phy_cfg(struct r8152 *tp)
sram_write(tp, SRAM_10M_AMP1, 0x00af);
sram_write(tp, SRAM_10M_AMP2, 0x0208);
- r8153_eee_en(tp, true);
- ocp_reg_write(tp, OCP_EEE_ADV, MDIO_EEE_1000T | MDIO_EEE_100TX);
+ if (tp->eee_en)
+ rtl_eee_enable(tp, true);
r8153_aldps_en(tp, true);
r8152b_enable_fc(tp);
@@ -3529,15 +3630,14 @@ static u32 r8152_efuse_read(struct r8152 *tp, u8 addr)
static void r8153b_hw_phy_cfg(struct r8152 *tp)
{
- u32 ocp_data, ups_flags = 0;
+ u32 ocp_data;
u16 data;
/* disable ALDPS before updating the PHY parameters */
- r8153b_aldps_en(tp, false);
+ r8153_aldps_en(tp, false);
/* disable EEE before updating the PHY parameters */
- r8153b_eee_en(tp, false);
- ocp_reg_write(tp, OCP_EEE_ADV, 0);
+ rtl_eee_enable(tp, false);
r8153b_green_en(tp, test_bit(GREEN_ETHERNET, &tp->flags));
@@ -3582,28 +3682,27 @@ static void r8153b_hw_phy_cfg(struct r8152 *tp)
data = ocp_reg_read(tp, OCP_POWER_CFG);
data |= EEE_CLKDIV_EN;
ocp_reg_write(tp, OCP_POWER_CFG, data);
+ tp->ups_info.eee_ckdiv = true;
data = ocp_reg_read(tp, OCP_DOWN_SPEED);
data |= EN_EEE_CMODE | EN_EEE_1000 | EN_10M_CLKDIV;
ocp_reg_write(tp, OCP_DOWN_SPEED, data);
+ tp->ups_info.eee_cmod_lv = true;
+ tp->ups_info._10m_ckdiv = true;
+ tp->ups_info.eee_plloff_giga = true;
ocp_reg_write(tp, OCP_SYSCLK_CFG, 0);
ocp_reg_write(tp, OCP_SYSCLK_CFG, clk_div_expo(5));
-
- ups_flags |= UPS_FLAGS_EN_10M_CKDIV | UPS_FLAGS_250M_CKDIV |
- UPS_FLAGS_EN_EEE_CKDIV | UPS_FLAGS_EEE_CMOD_LV_EN |
- UPS_FLAGS_EEE_PLLOFF_GIGA;
+ tp->ups_info._250m_ckdiv = true;
r8153_patch_request(tp, false);
}
- r8153b_ups_flags_w1w0(tp, ups_flags, 0);
+ if (tp->eee_en)
+ rtl_eee_enable(tp, true);
- r8153b_eee_en(tp, true);
- ocp_reg_write(tp, OCP_EEE_ADV, MDIO_EEE_1000T | MDIO_EEE_100TX);
-
- r8153b_aldps_en(tp, true);
- r8153b_enable_fc(tp);
+ r8153_aldps_en(tp, true);
+ r8152b_enable_fc(tp);
r8153_u2p3en(tp, true);
set_bit(PHY_RESET, &tp->flags);
@@ -3754,111 +3853,118 @@ static void rtl8153_disable(struct r8152 *tp)
r8153_aldps_en(tp, true);
}
-static void rtl8153b_disable(struct r8152 *tp)
+static int rtl8152_set_speed(struct r8152 *tp, u8 autoneg, u32 speed, u8 duplex,
+ u32 advertising)
{
- r8153b_aldps_en(tp, false);
- rtl_disable(tp);
- rtl_reset_bmu(tp);
- r8153b_aldps_en(tp, true);
-}
-
-static int rtl8152_set_speed(struct r8152 *tp, u8 autoneg, u16 speed, u8 duplex)
-{
- u16 bmcr, anar, gbcr;
- enum spd_duplex speed_duplex;
+ u16 bmcr;
int ret = 0;
- anar = r8152_mdio_read(tp, MII_ADVERTISE);
- anar &= ~(ADVERTISE_10HALF | ADVERTISE_10FULL |
- ADVERTISE_100HALF | ADVERTISE_100FULL);
- if (tp->mii.supports_gmii) {
- gbcr = r8152_mdio_read(tp, MII_CTRL1000);
- gbcr &= ~(ADVERTISE_1000FULL | ADVERTISE_1000HALF);
- } else {
- gbcr = 0;
- }
-
if (autoneg == AUTONEG_DISABLE) {
- if (speed == SPEED_10) {
- bmcr = 0;
- anar |= ADVERTISE_10HALF | ADVERTISE_10FULL;
- speed_duplex = FORCE_10M_HALF;
- } else if (speed == SPEED_100) {
+ if (duplex != DUPLEX_HALF && duplex != DUPLEX_FULL)
+ return -EINVAL;
+
+ switch (speed) {
+ case SPEED_10:
+ bmcr = BMCR_SPEED10;
+ if (duplex == DUPLEX_FULL) {
+ bmcr |= BMCR_FULLDPLX;
+ tp->ups_info.speed_duplex = FORCE_10M_FULL;
+ } else {
+ tp->ups_info.speed_duplex = FORCE_10M_HALF;
+ }
+ break;
+ case SPEED_100:
bmcr = BMCR_SPEED100;
- anar |= ADVERTISE_100HALF | ADVERTISE_100FULL;
- speed_duplex = FORCE_100M_HALF;
- } else if (speed == SPEED_1000 && tp->mii.supports_gmii) {
- bmcr = BMCR_SPEED1000;
- gbcr |= ADVERTISE_1000FULL | ADVERTISE_1000HALF;
- speed_duplex = NWAY_1000M_FULL;
- } else {
+ if (duplex == DUPLEX_FULL) {
+ bmcr |= BMCR_FULLDPLX;
+ tp->ups_info.speed_duplex = FORCE_100M_FULL;
+ } else {
+ tp->ups_info.speed_duplex = FORCE_100M_HALF;
+ }
+ break;
+ case SPEED_1000:
+ if (tp->mii.supports_gmii) {
+ bmcr = BMCR_SPEED1000 | BMCR_FULLDPLX;
+ tp->ups_info.speed_duplex = NWAY_1000M_FULL;
+ break;
+ }
+ /* fall through */
+ default:
ret = -EINVAL;
goto out;
}
- if (duplex == DUPLEX_FULL) {
- bmcr |= BMCR_FULLDPLX;
- if (speed != SPEED_1000)
- speed_duplex++;
- }
+ if (duplex == DUPLEX_FULL)
+ tp->mii.full_duplex = 1;
+ else
+ tp->mii.full_duplex = 0;
+
+ tp->mii.force_media = 1;
} else {
- if (speed == SPEED_10) {
- if (duplex == DUPLEX_FULL) {
- anar |= ADVERTISE_10HALF | ADVERTISE_10FULL;
- speed_duplex = NWAY_10M_FULL;
- } else {
- anar |= ADVERTISE_10HALF;
- speed_duplex = NWAY_10M_HALF;
+ u16 anar, tmp1;
+ u32 support;
+
+ support = RTL_ADVERTISED_10_HALF | RTL_ADVERTISED_10_FULL |
+ RTL_ADVERTISED_100_HALF | RTL_ADVERTISED_100_FULL;
+
+ if (tp->mii.supports_gmii)
+ support |= RTL_ADVERTISED_1000_FULL;
+
+ if (!(advertising & support))
+ return -EINVAL;
+
+ anar = r8152_mdio_read(tp, MII_ADVERTISE);
+ tmp1 = anar & ~(ADVERTISE_10HALF | ADVERTISE_10FULL |
+ ADVERTISE_100HALF | ADVERTISE_100FULL);
+ if (advertising & RTL_ADVERTISED_10_HALF) {
+ tmp1 |= ADVERTISE_10HALF;
+ tp->ups_info.speed_duplex = NWAY_10M_HALF;
+ }
+ if (advertising & RTL_ADVERTISED_10_FULL) {
+ tmp1 |= ADVERTISE_10FULL;
+ tp->ups_info.speed_duplex = NWAY_10M_FULL;
+ }
+
+ if (advertising & RTL_ADVERTISED_100_HALF) {
+ tmp1 |= ADVERTISE_100HALF;
+ tp->ups_info.speed_duplex = NWAY_100M_HALF;
+ }
+ if (advertising & RTL_ADVERTISED_100_FULL) {
+ tmp1 |= ADVERTISE_100FULL;
+ tp->ups_info.speed_duplex = NWAY_100M_FULL;
+ }
+
+ if (anar != tmp1) {
+ r8152_mdio_write(tp, MII_ADVERTISE, tmp1);
+ tp->mii.advertising = tmp1;
+ }
+
+ if (tp->mii.supports_gmii) {
+ u16 gbcr;
+
+ gbcr = r8152_mdio_read(tp, MII_CTRL1000);
+ tmp1 = gbcr & ~(ADVERTISE_1000FULL |
+ ADVERTISE_1000HALF);
+
+ if (advertising & RTL_ADVERTISED_1000_FULL) {
+ tmp1 |= ADVERTISE_1000FULL;
+ tp->ups_info.speed_duplex = NWAY_1000M_FULL;
}
- } else if (speed == SPEED_100) {
- if (duplex == DUPLEX_FULL) {
- anar |= ADVERTISE_10HALF | ADVERTISE_10FULL;
- anar |= ADVERTISE_100HALF | ADVERTISE_100FULL;
- speed_duplex = NWAY_100M_FULL;
- } else {
- anar |= ADVERTISE_10HALF;
- anar |= ADVERTISE_100HALF;
- speed_duplex = NWAY_100M_HALF;
- }
- } else if (speed == SPEED_1000 && tp->mii.supports_gmii) {
- if (duplex == DUPLEX_FULL) {
- anar |= ADVERTISE_10HALF | ADVERTISE_10FULL;
- anar |= ADVERTISE_100HALF | ADVERTISE_100FULL;
- gbcr |= ADVERTISE_1000FULL | ADVERTISE_1000HALF;
- } else {
- anar |= ADVERTISE_10HALF;
- anar |= ADVERTISE_100HALF;
- gbcr |= ADVERTISE_1000HALF;
- }
- speed_duplex = NWAY_1000M_FULL;
- } else {
- ret = -EINVAL;
- goto out;
+
+ if (gbcr != tmp1)
+ r8152_mdio_write(tp, MII_CTRL1000, tmp1);
}
bmcr = BMCR_ANENABLE | BMCR_ANRESTART;
+
+ tp->mii.force_media = 0;
}
if (test_and_clear_bit(PHY_RESET, &tp->flags))
bmcr |= BMCR_RESET;
- if (tp->mii.supports_gmii)
- r8152_mdio_write(tp, MII_CTRL1000, gbcr);
-
- r8152_mdio_write(tp, MII_ADVERTISE, anar);
r8152_mdio_write(tp, MII_BMCR, bmcr);
- switch (tp->version) {
- case RTL_VER_08:
- case RTL_VER_09:
- r8153b_ups_flags_w1w0(tp, ups_flags_speed(speed_duplex),
- UPS_FLAGS_SPEED_MASK);
- break;
-
- default:
- break;
- }
-
if (bmcr & BMCR_RESET) {
int i;
@@ -3943,12 +4049,12 @@ static void rtl8153b_up(struct r8152 *tp)
r8153b_u1u2en(tp, false);
r8153_u2p3en(tp, false);
- r8153b_aldps_en(tp, false);
+ r8153_aldps_en(tp, false);
r8153_first_init(tp);
ocp_write_dword(tp, MCU_TYPE_USB, USB_RX_BUF_TH, RX_THR_B);
- r8153b_aldps_en(tp, true);
+ r8153_aldps_en(tp, true);
r8153_u2p3en(tp, true);
r8153b_u1u2en(tp, true);
}
@@ -3963,9 +4069,9 @@ static void rtl8153b_down(struct r8152 *tp)
r8153b_u1u2en(tp, false);
r8153_u2p3en(tp, false);
r8153b_power_cut_en(tp, false);
- r8153b_aldps_en(tp, false);
+ r8153_aldps_en(tp, false);
r8153_enter_oob(tp);
- r8153b_aldps_en(tp, true);
+ r8153_aldps_en(tp, true);
}
static bool rtl8152_in_nway(struct r8152 *tp)
@@ -4083,7 +4189,8 @@ static void rtl_hw_phy_work_func_t(struct work_struct *work)
tp->rtl_ops.hw_phy_cfg(tp);
- rtl8152_set_speed(tp, tp->autoneg, tp->speed, tp->duplex);
+ rtl8152_set_speed(tp, tp->autoneg, tp->speed, tp->duplex,
+ tp->advertising);
mutex_unlock(&tp->control);
@@ -4176,8 +4283,7 @@ static int rtl8152_close(struct net_device *netdev)
unregister_pm_notifier(&tp->pm_notifier);
#endif
tasklet_disable(&tp->tx_tl);
- if (!test_bit(RTL8152_UNPLUG, &tp->flags))
- napi_disable(&tp->napi);
+ napi_disable(&tp->napi);
clear_bit(WORK_ENABLE, &tp->flags);
usb_kill_urb(tp->intr_urb);
cancel_delayed_work_sync(&tp->schedule);
@@ -4802,20 +4908,46 @@ static int rtl8152_set_link_ksettings(struct net_device *dev,
const struct ethtool_link_ksettings *cmd)
{
struct r8152 *tp = netdev_priv(dev);
+ u32 advertising = 0;
int ret;
ret = usb_autopm_get_interface(tp->intf);
if (ret < 0)
goto out;
+ if (test_bit(ETHTOOL_LINK_MODE_10baseT_Half_BIT,
+ cmd->link_modes.advertising))
+ advertising |= RTL_ADVERTISED_10_HALF;
+
+ if (test_bit(ETHTOOL_LINK_MODE_10baseT_Full_BIT,
+ cmd->link_modes.advertising))
+ advertising |= RTL_ADVERTISED_10_FULL;
+
+ if (test_bit(ETHTOOL_LINK_MODE_100baseT_Half_BIT,
+ cmd->link_modes.advertising))
+ advertising |= RTL_ADVERTISED_100_HALF;
+
+ if (test_bit(ETHTOOL_LINK_MODE_100baseT_Full_BIT,
+ cmd->link_modes.advertising))
+ advertising |= RTL_ADVERTISED_100_FULL;
+
+ if (test_bit(ETHTOOL_LINK_MODE_1000baseT_Half_BIT,
+ cmd->link_modes.advertising))
+ advertising |= RTL_ADVERTISED_1000_HALF;
+
+ if (test_bit(ETHTOOL_LINK_MODE_1000baseT_Full_BIT,
+ cmd->link_modes.advertising))
+ advertising |= RTL_ADVERTISED_1000_FULL;
+
mutex_lock(&tp->control);
ret = rtl8152_set_speed(tp, cmd->base.autoneg, cmd->base.speed,
- cmd->base.duplex);
+ cmd->base.duplex, advertising);
if (!ret) {
tp->autoneg = cmd->base.autoneg;
tp->speed = cmd->base.speed;
tp->duplex = cmd->base.duplex;
+ tp->advertising = advertising;
}
mutex_unlock(&tp->control);
@@ -4891,7 +5023,7 @@ static void rtl8152_get_strings(struct net_device *dev, u32 stringset, u8 *data)
static int r8152_get_eee(struct r8152 *tp, struct ethtool_eee *eee)
{
- u32 ocp_data, lp, adv, supported = 0;
+ u32 lp, adv, supported = 0;
u16 val;
val = r8152_mmd_read(tp, MDIO_MMD_PCS, MDIO_PCS_EEE_ABLE);
@@ -4903,13 +5035,10 @@ static int r8152_get_eee(struct r8152 *tp, struct ethtool_eee *eee)
val = r8152_mmd_read(tp, MDIO_MMD_AN, MDIO_AN_EEE_LPABLE);
lp = mmd_eee_adv_to_ethtool_adv_t(val);
- ocp_data = ocp_read_word(tp, MCU_TYPE_PLA, PLA_EEE_CR);
- ocp_data &= EEE_RX_EN | EEE_TX_EN;
-
- eee->eee_enabled = !!ocp_data;
+ eee->eee_enabled = tp->eee_en;
eee->eee_active = !!(supported & adv & lp);
eee->supported = supported;
- eee->advertised = adv;
+ eee->advertised = tp->eee_adv;
eee->lp_advertised = lp;
return 0;
@@ -4919,19 +5048,17 @@ static int r8152_set_eee(struct r8152 *tp, struct ethtool_eee *eee)
{
u16 val = ethtool_adv_to_mmd_eee_adv_t(eee->advertised);
- r8152_eee_en(tp, eee->eee_enabled);
+ tp->eee_en = eee->eee_enabled;
+ tp->eee_adv = val;
- if (!eee->eee_enabled)
- val = 0;
-
- r8152_mmd_write(tp, MDIO_MMD_AN, MDIO_AN_EEE_ADV, val);
+ rtl_eee_enable(tp, tp->eee_en);
return 0;
}
static int r8153_get_eee(struct r8152 *tp, struct ethtool_eee *eee)
{
- u32 ocp_data, lp, adv, supported = 0;
+ u32 lp, adv, supported = 0;
u16 val;
val = ocp_reg_read(tp, OCP_EEE_ABLE);
@@ -4943,46 +5070,15 @@ static int r8153_get_eee(struct r8152 *tp, struct ethtool_eee *eee)
val = ocp_reg_read(tp, OCP_EEE_LPABLE);
lp = mmd_eee_adv_to_ethtool_adv_t(val);
- ocp_data = ocp_read_word(tp, MCU_TYPE_PLA, PLA_EEE_CR);
- ocp_data &= EEE_RX_EN | EEE_TX_EN;
-
- eee->eee_enabled = !!ocp_data;
+ eee->eee_enabled = tp->eee_en;
eee->eee_active = !!(supported & adv & lp);
eee->supported = supported;
- eee->advertised = adv;
+ eee->advertised = tp->eee_adv;
eee->lp_advertised = lp;
return 0;
}
-static int r8153_set_eee(struct r8152 *tp, struct ethtool_eee *eee)
-{
- u16 val = ethtool_adv_to_mmd_eee_adv_t(eee->advertised);
-
- r8153_eee_en(tp, eee->eee_enabled);
-
- if (!eee->eee_enabled)
- val = 0;
-
- ocp_reg_write(tp, OCP_EEE_ADV, val);
-
- return 0;
-}
-
-static int r8153b_set_eee(struct r8152 *tp, struct ethtool_eee *eee)
-{
- u16 val = ethtool_adv_to_mmd_eee_adv_t(eee->advertised);
-
- r8153b_eee_en(tp, eee->eee_enabled);
-
- if (!eee->eee_enabled)
- val = 0;
-
- ocp_reg_write(tp, OCP_EEE_ADV, val);
-
- return 0;
-}
-
static int
rtl_ethtool_get_eee(struct net_device *net, struct ethtool_eee *edata)
{
@@ -5353,6 +5449,8 @@ static int rtl_ops_init(struct r8152 *tp)
ops->hw_phy_cfg = r8152b_hw_phy_cfg;
ops->autosuspend_en = rtl_runtime_suspend_enable;
tp->rx_buf_sz = 16 * 1024;
+ tp->eee_en = true;
+ tp->eee_adv = MDIO_EEE_100TX;
break;
case RTL_VER_03:
@@ -5366,27 +5464,31 @@ static int rtl_ops_init(struct r8152 *tp)
ops->down = rtl8153_down;
ops->unload = rtl8153_unload;
ops->eee_get = r8153_get_eee;
- ops->eee_set = r8153_set_eee;
+ ops->eee_set = r8152_set_eee;
ops->in_nway = rtl8153_in_nway;
ops->hw_phy_cfg = r8153_hw_phy_cfg;
ops->autosuspend_en = rtl8153_runtime_enable;
tp->rx_buf_sz = 32 * 1024;
+ tp->eee_en = true;
+ tp->eee_adv = MDIO_EEE_1000T | MDIO_EEE_100TX;
break;
case RTL_VER_08:
case RTL_VER_09:
ops->init = r8153b_init;
ops->enable = rtl8153_enable;
- ops->disable = rtl8153b_disable;
+ ops->disable = rtl8153_disable;
ops->up = rtl8153b_up;
ops->down = rtl8153b_down;
ops->unload = rtl8153b_unload;
ops->eee_get = r8153_get_eee;
- ops->eee_set = r8153b_set_eee;
+ ops->eee_set = r8152_set_eee;
ops->in_nway = rtl8153_in_nway;
ops->hw_phy_cfg = r8153b_hw_phy_cfg;
ops->autosuspend_en = rtl8153b_runtime_enable;
tp->rx_buf_sz = 32 * 1024;
+ tp->eee_en = true;
+ tp->eee_adv = MDIO_EEE_1000T | MDIO_EEE_100TX;
break;
default:
@@ -5560,7 +5662,13 @@ static int rtl8152_probe(struct usb_interface *intf,
tp->mii.phy_id = R8152_PHY_ID;
tp->autoneg = AUTONEG_ENABLE;
- tp->speed = tp->mii.supports_gmii ? SPEED_1000 : SPEED_100;
+ tp->speed = SPEED_100;
+ tp->advertising = RTL_ADVERTISED_10_HALF | RTL_ADVERTISED_10_FULL |
+ RTL_ADVERTISED_100_HALF | RTL_ADVERTISED_100_FULL;
+ if (tp->mii.supports_gmii) {
+ tp->speed = SPEED_1000;
+ tp->advertising |= RTL_ADVERTISED_1000_FULL;
+ }
tp->duplex = DUPLEX_FULL;
tp->rx_copybreak = RTL8152_RXFG_HEADSZ;
@@ -5595,7 +5703,6 @@ static int rtl8152_probe(struct usb_interface *intf,
return 0;
out1:
- netif_napi_del(&tp->napi);
tasklet_kill(&tp->tx_tl);
usb_set_intfdata(intf, NULL);
out:
@@ -5611,7 +5718,6 @@ static void rtl8152_disconnect(struct usb_interface *intf)
if (tp) {
rtl_set_unplug(tp);
- netif_napi_del(&tp->napi);
unregister_netdev(tp->netdev);
tasklet_kill(&tp->tx_tl);
cancel_delayed_work_sync(&tp->hw_phy_work);
diff --git a/drivers/net/usb/rtl8150.c b/drivers/net/usb/rtl8150.c
index 98f33e2..13e51cc 100644
--- a/drivers/net/usb/rtl8150.c
+++ b/drivers/net/usb/rtl8150.c
@@ -586,8 +586,7 @@ static void free_skb_pool(rtl8150_t *dev)
int i;
for (i = 0; i < RX_SKB_POOL_SIZE; i++)
- if (dev->rx_skb_pool[i])
- dev_kfree_skb(dev->rx_skb_pool[i]);
+ dev_kfree_skb(dev->rx_skb_pool[i]);
}
static void rx_fixup(unsigned long data)
@@ -946,8 +945,7 @@ static void rtl8150_disconnect(struct usb_interface *intf)
unlink_all_urbs(dev);
free_all_urbs(dev);
free_skb_pool(dev);
- if (dev->rx_skb)
- dev_kfree_skb(dev->rx_skb);
+ dev_kfree_skb(dev->rx_skb);
kfree(dev->intr_buff);
free_netdev(dev->netdev);
}
diff --git a/drivers/net/wimax/i2400m/fw.c b/drivers/net/wimax/i2400m/fw.c
index 489cba9..6c9a41b 100644
--- a/drivers/net/wimax/i2400m/fw.c
+++ b/drivers/net/wimax/i2400m/fw.c
@@ -397,14 +397,9 @@ int i2400m_is_boot_barker(struct i2400m *i2400m,
/* Short circuit if we have already discovered the barker
* associated with the device. */
- if (i2400m->barker
- && !memcmp(buf, i2400m->barker, sizeof(i2400m->barker->data))) {
- unsigned index = (i2400m->barker - i2400m_barker_db)
- / sizeof(*i2400m->barker);
- d_printf(2, dev, "boot barker cache-confirmed #%u/%08x\n",
- index, le32_to_cpu(i2400m->barker->data[0]));
+ if (i2400m->barker &&
+ !memcmp(buf, i2400m->barker, sizeof(i2400m->barker->data)))
return 0;
- }
for (i = 0; i < i2400m_barker_db_used; i++) {
barker = &i2400m_barker_db[i];
diff --git a/drivers/net/wimax/i2400m/rx.c b/drivers/net/wimax/i2400m/rx.c
index d28b96d..c9fb619 100644
--- a/drivers/net/wimax/i2400m/rx.c
+++ b/drivers/net/wimax/i2400m/rx.c
@@ -1253,7 +1253,6 @@ int i2400m_rx(struct i2400m *i2400m, struct sk_buff *skb)
skb_len = skb->len;
d_fnstart(4, dev, "(i2400m %p skb %p [size %u])\n",
i2400m, skb, skb_len);
- result = -EIO;
msg_hdr = (void *) skb->data;
result = i2400m_rx_msg_hdr_check(i2400m, msg_hdr, skb_len);
if (result < 0)
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
index 581d001..dd6303f 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
@@ -4222,10 +4222,8 @@ brcmf_parse_vndr_ies(const u8 *vndr_ie_buf, u32 vndr_ie_len,
vndr_ies->count++;
- brcmf_dbg(TRACE, "** OUI %02x %02x %02x, type 0x%02x\n",
- parsed_info->vndrie.oui[0],
- parsed_info->vndrie.oui[1],
- parsed_info->vndrie.oui[2],
+ brcmf_dbg(TRACE, "** OUI %3ph, type 0x%02x\n",
+ parsed_info->vndrie.oui,
parsed_info->vndrie.oui_type);
if (vndr_ies->count >= VNDR_IE_PARSE_LIMIT)
@@ -4244,9 +4242,7 @@ brcmf_parse_vndr_ies(const u8 *vndr_ie_buf, u32 vndr_ie_len,
static u32
brcmf_vndr_ie(u8 *iebuf, s32 pktflag, u8 *ie_ptr, u32 ie_len, s8 *add_del_cmd)
{
-
- strncpy(iebuf, add_del_cmd, VNDR_IE_CMD_LEN - 1);
- iebuf[VNDR_IE_CMD_LEN - 1] = '\0';
+ strscpy(iebuf, add_del_cmd, VNDR_IE_CMD_LEN);
put_unaligned_le32(1, &iebuf[VNDR_IE_COUNT_OFFSET]);
@@ -4351,12 +4347,10 @@ s32 brcmf_vif_set_mgmt_ie(struct brcmf_cfg80211_vif *vif, s32 pktflag,
for (i = 0; i < old_vndr_ies.count; i++) {
vndrie_info = &old_vndr_ies.ie_info[i];
- brcmf_dbg(TRACE, "DEL ID : %d, Len: %d , OUI:%02x:%02x:%02x\n",
+ brcmf_dbg(TRACE, "DEL ID : %d, Len: %d , OUI:%3ph\n",
vndrie_info->vndrie.id,
vndrie_info->vndrie.len,
- vndrie_info->vndrie.oui[0],
- vndrie_info->vndrie.oui[1],
- vndrie_info->vndrie.oui[2]);
+ vndrie_info->vndrie.oui);
del_add_ie_buf_len = brcmf_vndr_ie(curr_ie_buf, pktflag,
vndrie_info->ie_ptr,
@@ -4388,12 +4382,10 @@ s32 brcmf_vif_set_mgmt_ie(struct brcmf_cfg80211_vif *vif, s32 pktflag,
remained_buf_len -= (vndrie_info->ie_len +
VNDR_IE_VSIE_OFFSET);
- brcmf_dbg(TRACE, "ADDED ID : %d, Len: %d, OUI:%02x:%02x:%02x\n",
+ brcmf_dbg(TRACE, "ADDED ID : %d, Len: %d, OUI:%3ph\n",
vndrie_info->vndrie.id,
vndrie_info->vndrie.len,
- vndrie_info->vndrie.oui[0],
- vndrie_info->vndrie.oui[1],
- vndrie_info->vndrie.oui[2]);
+ vndrie_info->vndrie.oui);
del_add_ie_buf_len = brcmf_vndr_ie(curr_ie_buf, pktflag,
vndrie_info->ie_ptr,
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
index 1ec48c4..dd586a9 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
@@ -696,8 +696,10 @@ static u32 brcmf_chip_tcm_rambase(struct brcmf_chip_priv *ci)
return 0;
}
-static int brcmf_chip_get_raminfo(struct brcmf_chip_priv *ci)
+int brcmf_chip_get_raminfo(struct brcmf_chip *pub)
{
+ struct brcmf_chip_priv *ci = container_of(pub, struct brcmf_chip_priv,
+ pub);
struct brcmf_core_priv *mem_core;
struct brcmf_core *mem;
@@ -979,7 +981,7 @@ static int brcmf_chip_recognition(struct brcmf_chip_priv *ci)
brcmf_chip_set_passive(&ci->pub);
}
- return brcmf_chip_get_raminfo(ci);
+ return brcmf_chip_get_raminfo(&ci->pub);
}
static void brcmf_chip_disable_arm(struct brcmf_chip_priv *chip, u16 id)
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.h
index 206d769..7b00f6a 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.h
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.h
@@ -69,6 +69,7 @@ struct brcmf_buscore_ops {
void (*activate)(void *ctx, struct brcmf_chip *chip, u32 rstvec);
};
+int brcmf_chip_get_raminfo(struct brcmf_chip *pub);
struct brcmf_chip *brcmf_chip_attach(void *ctx,
const struct brcmf_buscore_ops *ops);
void brcmf_chip_detach(struct brcmf_chip *chip);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c
index 705b8cc..21e07d1 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c
@@ -1086,6 +1086,29 @@ static void brcmf_core_bus_reset(struct work_struct *work)
brcmf_bus_reset(drvr->bus_if);
}
+static ssize_t bus_reset_write(struct file *file, const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct brcmf_pub *drvr = file->private_data;
+ u8 value;
+
+ if (kstrtou8_from_user(user_buf, count, 0, &value))
+ return -EINVAL;
+
+ if (value != 1)
+ return -EINVAL;
+
+ schedule_work(&drvr->bus_reset);
+
+ return count;
+}
+
+static const struct file_operations bus_reset_fops = {
+ .open = simple_open,
+ .llseek = no_llseek,
+ .write = bus_reset_write,
+};
+
static int brcmf_bus_started(struct brcmf_pub *drvr, struct cfg80211_ops *ops)
{
int ret = -1;
@@ -1161,6 +1184,8 @@ static int brcmf_bus_started(struct brcmf_pub *drvr, struct cfg80211_ops *ops)
/* populate debugfs */
brcmf_debugfs_add_entry(drvr, "revinfo", brcmf_revinfo_read);
+ debugfs_create_file("reset", 0600, brcmf_debugfs_get_devdir(drvr), drvr,
+ &bus_reset_fops);
brcmf_feat_debugfs_create(drvr);
brcmf_proto_debugfs_create(drvr);
brcmf_bus_debugfs_create(bus_if);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/debug.h b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/debug.h
index ea6e8e8..9b221b5 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/debug.h
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/debug.h
@@ -121,6 +121,10 @@ int brcmf_debugfs_add_entry(struct brcmf_pub *drvr, const char *fn,
int brcmf_debug_create_memdump(struct brcmf_bus *bus, const void *data,
size_t len);
#else
+static inline struct dentry *brcmf_debugfs_get_devdir(struct brcmf_pub *drvr)
+{
+ return ERR_PTR(-ENOENT);
+}
static inline
int brcmf_debugfs_add_entry(struct brcmf_pub *drvr, const char *fn,
int (*read_fn)(struct seq_file *seq, void *data))
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
index 8428be8..e3dd862 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
@@ -1468,7 +1468,6 @@ static int brcmf_msgbuf_stats_read(struct seq_file *seq, void *data)
seq_printf(seq, "\nh2d_flowrings: depth %u\n",
BRCMF_H2D_TXFLOWRING_MAX_ITEM);
seq_puts(seq, "Active flowrings:\n");
- hash = msgbuf->flow->hash;
for (i = 0; i < msgbuf->flow->nrofrings; i++) {
if (!msgbuf->flow->rings[i])
continue;
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c
index 8d0e744..7ac9453 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c
@@ -1770,6 +1770,12 @@ static void brcmf_pcie_setup(struct device *dev, int ret,
nvram_len = fwreq->items[BRCMF_PCIE_FW_NVRAM].nv_data.len;
kfree(fwreq);
+ ret = brcmf_chip_get_raminfo(devinfo->ci);
+ if (ret) {
+ brcmf_err(bus, "Failed to get RAM info\n");
+ goto fail;
+ }
+
/* Some of the firmwares have the size of the memory of the device
* defined inside the firmware. This is because part of the memory in
* the device is shared and the devision is determined by FW. Parse
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
index 07f61d6..0c57d48 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
@@ -20035,7 +20035,7 @@ static void wlc_phy_radio_init_2056(struct brcms_phy *pi)
break;
default:
- break;
+ return;
}
}
diff --git a/drivers/net/wireless/cisco/airo.c b/drivers/net/wireless/cisco/airo.c
index 9342ffb..f43c065 100644
--- a/drivers/net/wireless/cisco/airo.c
+++ b/drivers/net/wireless/cisco/airo.c
@@ -5441,11 +5441,18 @@ static int proc_BSSList_open( struct inode *inode, struct file *file ) {
Cmd cmd;
Resp rsp;
- if (ai->flags & FLAG_RADIO_MASK) return -ENETDOWN;
+ if (ai->flags & FLAG_RADIO_MASK) {
+ kfree(data->rbuffer);
+ kfree(file->private_data);
+ return -ENETDOWN;
+ }
memset(&cmd, 0, sizeof(cmd));
cmd.cmd=CMD_LISTBSS;
- if (down_interruptible(&ai->sem))
+ if (down_interruptible(&ai->sem)) {
+ kfree(data->rbuffer);
+ kfree(file->private_data);
return -ERESTARTSYS;
+ }
issuecommand(ai, &cmd, &rsp);
up(&ai->sem);
data->readlen = 0;
diff --git a/drivers/net/wireless/intel/ipw2x00/ipw2200.c b/drivers/net/wireless/intel/ipw2x00/ipw2200.c
index fa55d2c..ed0f065 100644
--- a/drivers/net/wireless/intel/ipw2x00/ipw2200.c
+++ b/drivers/net/wireless/intel/ipw2x00/ipw2200.c
@@ -2721,7 +2721,7 @@ static void ipw_eeprom_init_sram(struct ipw_priv *priv)
/* Do not load eeprom data on fatal error or suspend */
ipw_write32(priv, IPW_EEPROM_LOAD_DISABLE, 0);
} else {
- IPW_DEBUG_INFO("Enabling FW initializationg of SRAM\n");
+ IPW_DEBUG_INFO("Enabling FW initialization of SRAM\n");
/* Load eeprom data on fatal error or suspend */
ipw_write32(priv, IPW_EEPROM_LOAD_DISABLE, 1);
diff --git a/drivers/net/wireless/intel/iwlwifi/Kconfig b/drivers/net/wireless/intel/iwlwifi/Kconfig
index 235349a..7dbc0d3 100644
--- a/drivers/net/wireless/intel/iwlwifi/Kconfig
+++ b/drivers/net/wireless/intel/iwlwifi/Kconfig
@@ -92,20 +92,6 @@
If unsure, don't enable this option, as some programs might
expect incoming broadcasts for their normal operations.
-config IWLWIFI_PCIE_RTPM
- bool "Enable runtime power management mode for PCIe devices"
- depends on IWLMVM && PM && EXPERT
- help
- Say Y here to enable runtime power management for PCIe
- devices. If enabled, the device will go into low power mode
- when idle for a short period of time, allowing for improved
- power saving during runtime. Note that this feature requires
- a tight integration with the platform. It is not recommended
- to enable this feature without proper validation with the
- specific target platform.
-
- If unsure, say N.
-
menu "Debugging Options"
config IWLWIFI_DEBUG
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/1000.c b/drivers/net/wireless/intel/iwlwifi/cfg/1000.c
index a1aa295..b92255b 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/1000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/1000.c
@@ -2,7 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* Contact Information:
* Intel Linux Wireless <linuxwifi@intel.com>
@@ -69,16 +69,16 @@ static const struct iwl_eeprom_params iwl1000_eeprom_params = {
.fw_name_pre = IWL1000_FW_PRE, \
.ucode_api_max = IWL1000_UCODE_API_MAX, \
.ucode_api_min = IWL1000_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_1000, \
+ .trans.device_family = IWL_DEVICE_FAMILY_1000, \
.max_inst_size = IWLAGN_RTC_INST_SIZE, \
.max_data_size = IWLAGN_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_1000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_1000_TX_POWER_VERSION, \
- .base_params = &iwl1000_base_params, \
+ .trans.base_params = &iwl1000_base_params, \
.eeprom_params = &iwl1000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl1000_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 1000 BGN",
@@ -95,17 +95,17 @@ const struct iwl_cfg iwl1000_bg_cfg = {
.fw_name_pre = IWL100_FW_PRE, \
.ucode_api_max = IWL100_UCODE_API_MAX, \
.ucode_api_min = IWL100_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_100, \
+ .trans.device_family = IWL_DEVICE_FAMILY_100, \
.max_inst_size = IWLAGN_RTC_INST_SIZE, \
.max_data_size = IWLAGN_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_1000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_1000_TX_POWER_VERSION, \
- .base_params = &iwl1000_base_params, \
+ .trans.base_params = &iwl1000_base_params, \
.eeprom_params = &iwl1000_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.rx_with_siso_diversity = true, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl100_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 100 BGN",
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/2000.c b/drivers/net/wireless/intel/iwlwifi/cfg/2000.c
index 4a988b6..2b1ae0c 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/2000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/2000.c
@@ -2,7 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* Contact Information:
* Intel Linux Wireless <linuxwifi@intel.com>
@@ -95,16 +95,16 @@ static const struct iwl_eeprom_params iwl20x0_eeprom_params = {
.fw_name_pre = IWL2000_FW_PRE, \
.ucode_api_max = IWL2000_UCODE_API_MAX, \
.ucode_api_min = IWL2000_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_2000, \
+ .trans.device_family = IWL_DEVICE_FAMILY_2000, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_2000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_2000_TX_POWER_VERSION, \
- .base_params = &iwl2000_base_params, \
+ .trans.base_params = &iwl2000_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl2000_2bgn_cfg = {
@@ -123,16 +123,16 @@ const struct iwl_cfg iwl2000_2bgn_d_cfg = {
.fw_name_pre = IWL2030_FW_PRE, \
.ucode_api_max = IWL2030_UCODE_API_MAX, \
.ucode_api_min = IWL2030_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_2030, \
+ .trans.device_family = IWL_DEVICE_FAMILY_2030, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_2000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_2000_TX_POWER_VERSION, \
- .base_params = &iwl2030_base_params, \
+ .trans.base_params = &iwl2030_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl2030_2bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 2230 BGN",
@@ -144,17 +144,17 @@ const struct iwl_cfg iwl2030_2bgn_cfg = {
.fw_name_pre = IWL105_FW_PRE, \
.ucode_api_max = IWL105_UCODE_API_MAX, \
.ucode_api_min = IWL105_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_105, \
+ .trans.device_family = IWL_DEVICE_FAMILY_105, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_2000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_2000_TX_POWER_VERSION, \
- .base_params = &iwl2000_base_params, \
+ .trans.base_params = &iwl2000_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.rx_with_siso_diversity = true, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl105_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 105 BGN",
@@ -172,17 +172,17 @@ const struct iwl_cfg iwl105_bgn_d_cfg = {
.fw_name_pre = IWL135_FW_PRE, \
.ucode_api_max = IWL135_UCODE_API_MAX, \
.ucode_api_min = IWL135_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_135, \
+ .trans.device_family = IWL_DEVICE_FAMILY_135, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_2000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_2000_TX_POWER_VERSION, \
- .base_params = &iwl2030_base_params, \
+ .trans.base_params = &iwl2030_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.rx_with_siso_diversity = true, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl135_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 135 BGN",
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
index 1f500cd..5e355c4 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/22000.c
@@ -56,7 +56,7 @@
#include "iwl-config.h"
/* Highest firmware API version supported */
-#define IWL_22000_UCODE_API_MAX 48
+#define IWL_22000_UCODE_API_MAX 50
/* Lowest firmware API version supported */
#define IWL_22000_UCODE_API_MIN 39
@@ -76,7 +76,6 @@
#define IWL_22000_HR_FW_PRE "iwlwifi-Qu-a0-hr-a0-"
#define IWL_22000_HR_CDB_FW_PRE "iwlwifi-QuIcp-z0-hrcdb-a0-"
#define IWL_22000_HR_A_F0_FW_PRE "iwlwifi-QuQnj-f0-hr-a0-"
-#define IWL_22000_HR_B_F0_FW_PRE "iwlwifi-Qu-b0-hr-b0-"
#define IWL_22000_QU_B_HR_B_FW_PRE "iwlwifi-Qu-b0-hr-b0-"
#define IWL_22000_HR_B_FW_PRE "iwlwifi-QuQnj-b0-hr-b0-"
#define IWL_22000_HR_A0_FW_PRE "iwlwifi-QuQnj-a0-hr-a0-"
@@ -99,8 +98,6 @@
IWL_22000_JF_FW_PRE __stringify(api) ".ucode"
#define IWL_22000_HR_A_F0_QNJ_MODULE_FIRMWARE(api) \
IWL_22000_HR_A_F0_FW_PRE __stringify(api) ".ucode"
-#define IWL_22000_HR_B_F0_QNJ_MODULE_FIRMWARE(api) \
- IWL_22000_HR_B_F0_FW_PRE __stringify(api) ".ucode"
#define IWL_22000_QU_B_HR_B_MODULE_FIRMWARE(api) \
IWL_22000_QU_B_HR_B_FW_PRE __stringify(api) ".ucode"
#define IWL_22000_HR_B_QNJ_MODULE_FIRMWARE(api) \
@@ -172,15 +169,15 @@ static const struct iwl_ht_params iwl_22000_ht_params = {
.smem_len = IWL_22000_SMEM_LEN, \
.features = IWL_TX_CSUM_NETIF_FLAGS | NETIF_F_RXCSUM, \
.apmg_not_supported = true, \
- .mq_rx_supported = true, \
+ .trans.mq_rx_supported = true, \
.vht_mu_mimo_supported = true, \
.mac_addr_from_csr = true, \
.ht_params = &iwl_22000_ht_params, \
.nvm_ver = IWL_22000_NVM_VERSION, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .use_tfh = true, \
- .rf_id = true, \
- .gen2 = true, \
+ .trans.use_tfh = true, \
+ .trans.rf_id = true, \
+ .trans.gen2 = true, \
.nvm_type = IWL_NVM_EXT, \
.dbgc_supported = true, \
.min_umac_error_event_table = 0x400000, \
@@ -191,28 +188,25 @@ static const struct iwl_ht_params iwl_22000_ht_params = {
.fw_mon_smem_cycle_cnt_ptr_addr = 0xa0c174, \
.fw_mon_smem_cycle_cnt_ptr_msk = 0xfffff
-#define IWL_DEVICE_AX200_COMMON \
- IWL_DEVICE_22000_COMMON, \
- .umac_prph_offset = 0x300000
-
#define IWL_DEVICE_22500 \
IWL_DEVICE_22000_COMMON, \
- .device_family = IWL_DEVICE_FAMILY_22000, \
- .base_params = &iwl_22000_base_params, \
- .csr = &iwl_csr_v1, \
+ .trans.device_family = IWL_DEVICE_FAMILY_22000, \
+ .trans.base_params = &iwl_22000_base_params, \
+ .trans.csr = &iwl_csr_v1, \
.gp2_reg_addr = 0xa02c68
#define IWL_DEVICE_22560 \
IWL_DEVICE_22000_COMMON, \
- .device_family = IWL_DEVICE_FAMILY_22560, \
- .base_params = &iwl_22560_base_params, \
- .csr = &iwl_csr_v2
+ .trans.device_family = IWL_DEVICE_FAMILY_22560, \
+ .trans.base_params = &iwl_22560_base_params, \
+ .trans.csr = &iwl_csr_v2
#define IWL_DEVICE_AX210 \
- IWL_DEVICE_AX200_COMMON, \
- .device_family = IWL_DEVICE_FAMILY_AX210, \
- .base_params = &iwl_22560_base_params, \
- .csr = &iwl_csr_v1, \
+ IWL_DEVICE_22000_COMMON, \
+ .trans.umac_prph_offset = 0x300000, \
+ .trans.device_family = IWL_DEVICE_FAMILY_AX210, \
+ .trans.base_params = &iwl_22560_base_params, \
+ .trans.csr = &iwl_csr_v1, \
.min_txq_size = 128, \
.gp2_reg_addr = 0xd02c68, \
.min_256_ba_txq_size = 512
@@ -246,6 +240,7 @@ const struct iwl_cfg iwl_ax101_cfg_qu_hr = {
* HT size; mac80211 would otherwise pick the HE max (256) by default.
*/
.max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
+ .tx_with_siso_diversity = true,
};
const struct iwl_cfg iwl_ax201_cfg_qu_hr = {
@@ -342,7 +337,7 @@ const struct iwl_cfg iwl_ax200_cfg_cc = {
* HT size; mac80211 would otherwise pick the HE max (256) by default.
*/
.max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
- .bisr_workaround = 1,
+ .trans.bisr_workaround = 1,
};
const struct iwl_cfg killer1650x_2ax_cfg = {
@@ -355,7 +350,7 @@ const struct iwl_cfg killer1650x_2ax_cfg = {
* HT size; mac80211 would otherwise pick the HE max (256) by default.
*/
.max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
- .bisr_workaround = 1,
+ .trans.bisr_workaround = 1,
};
const struct iwl_cfg killer1650w_2ax_cfg = {
@@ -368,7 +363,7 @@ const struct iwl_cfg killer1650w_2ax_cfg = {
* HT size; mac80211 would otherwise pick the HE max (256) by default.
*/
.max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
- .bisr_workaround = 1,
+ .trans.bisr_workaround = 1,
};
/*
@@ -556,6 +551,30 @@ const struct iwl_cfg killer1650i_2ax_cfg_qu_b0_hr_b0 = {
.max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
};
+const struct iwl_cfg killer1650s_2ax_cfg_qu_c0_hr_b0 = {
+ .name = "Killer(R) Wi-Fi 6 AX1650i 160MHz Wireless Network Adapter (201NGW)",
+ .fw_name_pre = IWL_QU_C_HR_B_FW_PRE,
+ IWL_DEVICE_22500,
+ /*
+ * This device doesn't support receiving BlockAck with a large bitmap
+ * so we need to restrict the size of transmitted aggregation to the
+ * HT size; mac80211 would otherwise pick the HE max (256) by default.
+ */
+ .max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
+};
+
+const struct iwl_cfg killer1650i_2ax_cfg_qu_c0_hr_b0 = {
+ .name = "Killer(R) Wi-Fi 6 AX1650s 160MHz Wireless Network Adapter (201D2W)",
+ .fw_name_pre = IWL_QU_C_HR_B_FW_PRE,
+ IWL_DEVICE_22500,
+ /*
+ * This device doesn't support receiving BlockAck with a large bitmap
+ * so we need to restrict the size of transmitted aggregation to the
+ * HT size; mac80211 would otherwise pick the HE max (256) by default.
+ */
+ .max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
+};
+
const struct iwl_cfg iwl22000_2ax_cfg_jf = {
.name = "Intel(R) Dual Band Wireless AX 22000",
.fw_name_pre = IWL_QU_B_JF_B_FW_PRE,
@@ -639,7 +658,6 @@ const struct iwl_cfg iwlax411_2ax_cfg_so_gf4_a0 = {
MODULE_FIRMWARE(IWL_22000_HR_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_22000_JF_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_22000_HR_A_F0_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
-MODULE_FIRMWARE(IWL_22000_HR_B_F0_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_22000_HR_B_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_22000_HR_A0_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_QU_C_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/5000.c b/drivers/net/wireless/intel/iwlwifi/cfg/5000.c
index ce25c69..aab4495 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/5000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/5000.c
@@ -67,16 +67,16 @@ static const struct iwl_eeprom_params iwl5000_eeprom_params = {
.fw_name_pre = IWL5000_FW_PRE, \
.ucode_api_max = IWL5000_UCODE_API_MAX, \
.ucode_api_min = IWL5000_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_5000, \
+ .trans.device_family = IWL_DEVICE_FAMILY_5000, \
.max_inst_size = IWLAGN_RTC_INST_SIZE, \
.max_data_size = IWLAGN_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_5000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_5000_TX_POWER_VERSION, \
- .base_params = &iwl5000_base_params, \
+ .trans.base_params = &iwl5000_base_params, \
.eeprom_params = &iwl5000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl5300_agn_cfg = {
.name = "Intel(R) Ultimate N WiFi Link 5300 AGN",
@@ -115,34 +115,34 @@ const struct iwl_cfg iwl5350_agn_cfg = {
.fw_name_pre = IWL5000_FW_PRE,
.ucode_api_max = IWL5000_UCODE_API_MAX,
.ucode_api_min = IWL5000_UCODE_API_MIN,
- .device_family = IWL_DEVICE_FAMILY_5000,
+ .trans.device_family = IWL_DEVICE_FAMILY_5000,
.max_inst_size = IWLAGN_RTC_INST_SIZE,
.max_data_size = IWLAGN_RTC_DATA_SIZE,
.nvm_ver = EEPROM_5050_EEPROM_VERSION,
.nvm_calib_ver = EEPROM_5050_TX_POWER_VERSION,
- .base_params = &iwl5000_base_params,
+ .trans.base_params = &iwl5000_base_params,
.eeprom_params = &iwl5000_eeprom_params,
.ht_params = &iwl5000_ht_params,
.led_mode = IWL_LED_BLINK,
.internal_wimax_coex = true,
- .csr = &iwl_csr_v1,
+ .trans.csr = &iwl_csr_v1,
};
#define IWL_DEVICE_5150 \
.fw_name_pre = IWL5150_FW_PRE, \
.ucode_api_max = IWL5150_UCODE_API_MAX, \
.ucode_api_min = IWL5150_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_5150, \
+ .trans.device_family = IWL_DEVICE_FAMILY_5150, \
.max_inst_size = IWLAGN_RTC_INST_SIZE, \
.max_data_size = IWLAGN_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_5050_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_5050_TX_POWER_VERSION, \
- .base_params = &iwl5000_base_params, \
+ .trans.base_params = &iwl5000_base_params, \
.eeprom_params = &iwl5000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
.internal_wimax_coex = true, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl5150_agn_cfg = {
.name = "Intel(R) WiMAX/WiFi Link 5150 AGN",
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/6000.c b/drivers/net/wireless/intel/iwlwifi/cfg/6000.c
index 67d61a1..39ea819 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/6000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/6000.c
@@ -2,7 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* Contact Information:
* Intel Linux Wireless <linuxwifi@intel.com>
@@ -116,16 +116,16 @@ static const struct iwl_eeprom_params iwl6000_eeprom_params = {
.fw_name_pre = IWL6005_FW_PRE, \
.ucode_api_max = IWL6000G2_UCODE_API_MAX, \
.ucode_api_min = IWL6000G2_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_6005, \
+ .trans.device_family = IWL_DEVICE_FAMILY_6005, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_6005_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_6005_TX_POWER_VERSION, \
- .base_params = &iwl6000_g2_base_params, \
+ .trans.base_params = &iwl6000_g2_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl6005_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6205 AGN",
@@ -171,16 +171,16 @@ const struct iwl_cfg iwl6005_2agn_mow2_cfg = {
.fw_name_pre = IWL6030_FW_PRE, \
.ucode_api_max = IWL6000G2_UCODE_API_MAX, \
.ucode_api_min = IWL6000G2_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_6030, \
+ .trans.device_family = IWL_DEVICE_FAMILY_6030, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_6030_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_6030_TX_POWER_VERSION, \
- .base_params = &iwl6000_g2_base_params, \
+ .trans.base_params = &iwl6000_g2_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl6030_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6230 AGN",
@@ -208,16 +208,16 @@ const struct iwl_cfg iwl6030_2bg_cfg = {
.fw_name_pre = IWL6030_FW_PRE, \
.ucode_api_max = IWL6035_UCODE_API_MAX, \
.ucode_api_min = IWL6035_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_6030, \
+ .trans.device_family = IWL_DEVICE_FAMILY_6030, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_6030_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_6030_TX_POWER_VERSION, \
- .base_params = &iwl6000_g2_base_params, \
+ .trans.base_params = &iwl6000_g2_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl6035_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6235 AGN",
@@ -262,18 +262,18 @@ const struct iwl_cfg iwl130_bg_cfg = {
.fw_name_pre = IWL6000_FW_PRE, \
.ucode_api_max = IWL6000_UCODE_API_MAX, \
.ucode_api_min = IWL6000_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_6000i, \
+ .trans.device_family = IWL_DEVICE_FAMILY_6000i, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.valid_tx_ant = ANT_BC, /* .cfg overwrite */ \
.valid_rx_ant = ANT_BC, /* .cfg overwrite */ \
.nvm_ver = EEPROM_6000_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_6000_TX_POWER_VERSION, \
- .base_params = &iwl6000_base_params, \
+ .trans.base_params = &iwl6000_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl6000i_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6200 AGN",
@@ -295,19 +295,19 @@ const struct iwl_cfg iwl6000i_2bg_cfg = {
.fw_name_pre = IWL6050_FW_PRE, \
.ucode_api_max = IWL6050_UCODE_API_MAX, \
.ucode_api_min = IWL6050_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_6050, \
+ .trans.device_family = IWL_DEVICE_FAMILY_6050, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.valid_tx_ant = ANT_AB, /* .cfg overwrite */ \
.valid_rx_ant = ANT_AB, /* .cfg overwrite */ \
.nvm_ver = EEPROM_6050_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_6050_TX_POWER_VERSION, \
- .base_params = &iwl6050_base_params, \
+ .trans.base_params = &iwl6050_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
.internal_wimax_coex = true, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl6050_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N + WiMAX 6250 AGN",
@@ -324,17 +324,17 @@ const struct iwl_cfg iwl6050_2abg_cfg = {
.fw_name_pre = IWL6050_FW_PRE, \
.ucode_api_max = IWL6050_UCODE_API_MAX, \
.ucode_api_min = IWL6050_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_6150, \
+ .trans.device_family = IWL_DEVICE_FAMILY_6150, \
.max_inst_size = IWL60_RTC_INST_SIZE, \
.max_data_size = IWL60_RTC_DATA_SIZE, \
.nvm_ver = EEPROM_6150_EEPROM_VERSION, \
.nvm_calib_ver = EEPROM_6150_TX_POWER_VERSION, \
- .base_params = &iwl6050_base_params, \
+ .trans.base_params = &iwl6050_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
.internal_wimax_coex = true, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
const struct iwl_cfg iwl6150_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N + WiMAX 6150 BGN",
@@ -352,16 +352,16 @@ const struct iwl_cfg iwl6000_3agn_cfg = {
.fw_name_pre = IWL6000_FW_PRE,
.ucode_api_max = IWL6000_UCODE_API_MAX,
.ucode_api_min = IWL6000_UCODE_API_MIN,
- .device_family = IWL_DEVICE_FAMILY_6000,
+ .trans.device_family = IWL_DEVICE_FAMILY_6000,
.max_inst_size = IWL60_RTC_INST_SIZE,
.max_data_size = IWL60_RTC_DATA_SIZE,
.nvm_ver = EEPROM_6000_EEPROM_VERSION,
.nvm_calib_ver = EEPROM_6000_TX_POWER_VERSION,
- .base_params = &iwl6000_base_params,
+ .trans.base_params = &iwl6000_base_params,
.eeprom_params = &iwl6000_eeprom_params,
.ht_params = &iwl6000_ht_params,
.led_mode = IWL_LED_BLINK,
- .csr = &iwl_csr_v1,
+ .trans.csr = &iwl_csr_v1,
};
MODULE_FIRMWARE(IWL6000_MODULE_FIRMWARE(IWL6000_UCODE_API_MAX));
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/7000.c b/drivers/net/wireless/intel/iwlwifi/cfg/7000.c
index 289e3c39..deb520ae 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/7000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/7000.c
@@ -8,7 +8,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2015 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -31,7 +31,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2015 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -148,14 +148,14 @@ static const struct iwl_ht_params iwl7000_ht_params = {
};
#define IWL_DEVICE_7000_COMMON \
- .device_family = IWL_DEVICE_FAMILY_7000, \
- .base_params = &iwl7000_base_params, \
+ .trans.device_family = IWL_DEVICE_FAMILY_7000, \
+ .trans.base_params = &iwl7000_base_params, \
.led_mode = IWL_LED_RF_STATE, \
.nvm_hw_section_num = 0, \
.non_shared_ant = ANT_A, \
.max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K, \
.dccm_offset = IWL7000_DCCM_OFFSET, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
#define IWL_DEVICE_7000 \
IWL_DEVICE_7000_COMMON, \
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/8000.c b/drivers/net/wireless/intel/iwlwifi/cfg/8000.c
index d7d17c1..b3cc477 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/8000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/8000.c
@@ -8,7 +8,7 @@
* Copyright(c) 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2014 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -30,7 +30,7 @@
*
* Copyright(c) 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2014 - 2015 Intel Mobile Communications GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -134,8 +134,8 @@ static const struct iwl_tt_params iwl8000_tt_params = {
};
#define IWL_DEVICE_8000_COMMON \
- .device_family = IWL_DEVICE_FAMILY_8000, \
- .base_params = &iwl8000_base_params, \
+ .trans.device_family = IWL_DEVICE_FAMILY_8000, \
+ .trans.base_params = &iwl8000_base_params, \
.led_mode = IWL_LED_RF_STATE, \
.nvm_hw_section_num = 10, \
.features = NETIF_F_RXCSUM, \
@@ -152,7 +152,7 @@ static const struct iwl_tt_params iwl8000_tt_params = {
.nvm_type = IWL_NVM_EXT, \
.dbgc_supported = true, \
.min_umac_error_event_table = 0x800000, \
- .csr = &iwl_csr_v1
+ .trans.csr = &iwl_csr_v1
#define IWL_DEVICE_8000 \
IWL_DEVICE_8000_COMMON, \
diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/9000.c b/drivers/net/wireless/intel/iwlwifi/cfg/9000.c
index 41bdd0e..e8372b6 100644
--- a/drivers/net/wireless/intel/iwlwifi/cfg/9000.c
+++ b/drivers/net/wireless/intel/iwlwifi/cfg/9000.c
@@ -122,8 +122,8 @@ static const struct iwl_tt_params iwl9000_tt_params = {
#define IWL_DEVICE_9000 \
.ucode_api_max = IWL9000_UCODE_API_MAX, \
.ucode_api_min = IWL9000_UCODE_API_MIN, \
- .device_family = IWL_DEVICE_FAMILY_9000, \
- .base_params = &iwl9000_base_params, \
+ .trans.device_family = IWL_DEVICE_FAMILY_9000, \
+ .trans.base_params = &iwl9000_base_params, \
.led_mode = IWL_LED_RF_STATE, \
.nvm_hw_section_num = 10, \
.non_shared_ant = ANT_B, \
@@ -136,14 +136,14 @@ static const struct iwl_tt_params iwl9000_tt_params = {
.features = IWL_TX_CSUM_NETIF_FLAGS | NETIF_F_RXCSUM, \
.thermal_params = &iwl9000_tt_params, \
.apmg_not_supported = true, \
- .mq_rx_supported = true, \
+ .trans.mq_rx_supported = true, \
.vht_mu_mimo_supported = true, \
.mac_addr_from_csr = true, \
- .rf_id = true, \
+ .trans.rf_id = true, \
.nvm_type = IWL_NVM_EXT, \
.dbgc_supported = true, \
.min_umac_error_event_table = 0x800000, \
- .csr = &iwl_csr_v1, \
+ .trans.csr = &iwl_csr_v1, \
.d3_debug_data_base_addr = 0x401000, \
.d3_debug_data_length = 92 * 1024, \
.ht_params = &iwl9000_ht_params, \
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/devices.c b/drivers/net/wireless/intel/iwlwifi/dvm/devices.c
index b39f8b1..dc3f197 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/devices.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/devices.c
@@ -2,6 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
+ * Copyright (C) 2019 Intel Corporation
*
* Contact Information:
* Intel Linux Wireless <linuxwifi@intel.com>
@@ -483,7 +484,7 @@ static void iwl6000_set_ct_threshold(struct iwl_priv *priv)
/* NIC configuration for 6000 series */
static void iwl6000_nic_config(struct iwl_priv *priv)
{
- switch (priv->cfg->device_family) {
+ switch (priv->trans->trans_cfg->device_family) {
case IWL_DEVICE_FAMILY_6005:
case IWL_DEVICE_FAMILY_6030:
case IWL_DEVICE_FAMILY_6000:
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/led.c b/drivers/net/wireless/intel/iwlwifi/dvm/led.c
index 38fd41f..dd387ab 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/led.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/led.c
@@ -2,6 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2003 - 2014 Intel Corporation. All rights reserved.
+ * Copyright (C) 2019 Intel Corporation
*
* Contact Information:
* Intel Linux Wireless <linuxwifi@intel.com>
@@ -120,9 +121,9 @@ static int iwl_led_cmd(struct iwl_priv *priv,
}
led_cmd.on = iwl_blink_compensation(priv, on,
- priv->cfg->base_params->led_compensation);
+ priv->trans->trans_cfg->base_params->led_compensation);
led_cmd.off = iwl_blink_compensation(priv, off,
- priv->cfg->base_params->led_compensation);
+ priv->trans->trans_cfg->base_params->led_compensation);
ret = iwl_send_led_cmd(priv, &led_cmd);
if (!ret) {
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/dvm/mac80211.c
index 6c17063..6512d25 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/mac80211.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/mac80211.c
@@ -2,7 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2003 - 2014 Intel Corporation. All rights reserved.
- * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 - 2019 Intel Corporation
*
* Portions of this file are derived from the ipw3945 project, as well
* as portions of the ieee80211 subsystem header files.
@@ -1099,7 +1099,7 @@ static void iwlagn_mac_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
goto done;
}
- scd_queues = BIT(priv->cfg->base_params->num_of_queues) - 1;
+ scd_queues = BIT(priv->trans->trans_cfg->base_params->num_of_queues) - 1;
scd_queues &= ~(BIT(IWL_IPAN_CMD_QUEUE_NUM) |
BIT(IWL_DEFAULT_CMD_QUEUE_NUM));
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/main.c b/drivers/net/wireless/intel/iwlwifi/dvm/main.c
index ae5e457..4f2789b 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/main.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/main.c
@@ -3,7 +3,7 @@
*
* Copyright(c) 2003 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2015 Intel Deutschland GmbH
- * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 - 2019 Intel Corporation
*
* Portions of this file are derived from the ipw3945 project, as well
* as portions of the ieee80211 subsystem header files.
@@ -1267,7 +1267,7 @@ static struct iwl_op_mode *iwl_op_mode_dvm_start(struct iwl_trans *trans,
priv->cfg = cfg;
priv->fw = fw;
- switch (priv->cfg->device_family) {
+ switch (priv->trans->trans_cfg->device_family) {
case IWL_DEVICE_FAMILY_1000:
case IWL_DEVICE_FAMILY_100:
priv->lib = &iwl_dvm_1000_cfg;
@@ -1342,7 +1342,7 @@ static struct iwl_op_mode *iwl_op_mode_dvm_start(struct iwl_trans *trans,
driver_data[2]);
WARN_ON(sizeof(priv->transport_queue_stop) * BITS_PER_BYTE <
- priv->cfg->base_params->num_of_queues);
+ priv->trans->trans_cfg->base_params->num_of_queues);
ucode_flags = fw->ucode_capa.flags;
@@ -1405,9 +1405,9 @@ static struct iwl_op_mode *iwl_op_mode_dvm_start(struct iwl_trans *trans,
/* Reset chip to save power until we load uCode during "up". */
iwl_trans_stop_device(priv->trans);
- priv->nvm_data = iwl_parse_eeprom_data(priv->trans->dev, priv->cfg,
- priv->eeprom_blob,
- priv->eeprom_blob_size);
+ priv->nvm_data = iwl_parse_eeprom_data(priv->trans, priv->cfg,
+ priv->eeprom_blob,
+ priv->eeprom_blob_size);
if (!priv->nvm_data)
goto out_free_eeprom_blob;
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/power.c b/drivers/net/wireless/intel/iwlwifi/dvm/power.c
index dcb9480..93ef023 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/power.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/power.c
@@ -2,6 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2007 - 2014 Intel Corporation. All rights reserved.
+ * Copyright (C) 2019 Intel Corporation
*
* Portions of this file are derived from the ipw3945 project, as well
* as portions of the ieee80211 subsystem header files.
@@ -199,7 +200,7 @@ static void iwl_static_sleep_cmd(struct iwl_priv *priv,
else
cmd->flags &= ~IWL_POWER_SLEEP_OVER_DTIM_MSK;
- if (priv->cfg->base_params->shadow_reg_enable)
+ if (priv->trans->trans_cfg->base_params->shadow_reg_enable)
cmd->flags |= IWL_POWER_SHADOW_REG_ENA;
else
cmd->flags &= ~IWL_POWER_SHADOW_REG_ENA;
diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/tx.c b/drivers/net/wireless/intel/iwlwifi/dvm/tx.c
index 247f417..3029e3f 100644
--- a/drivers/net/wireless/intel/iwlwifi/dvm/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/dvm/tx.c
@@ -2,6 +2,7 @@
/******************************************************************************
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
+ * Copyright (C) 2019 Intel Corporation
*
* Contact Information:
* Intel Linux Wireless <linuxwifi@intel.com>
@@ -467,7 +468,7 @@ static int iwlagn_alloc_agg_txq(struct iwl_priv *priv, int mq)
int q;
for (q = IWLAGN_FIRST_AMPDU_QUEUE;
- q < priv->cfg->base_params->num_of_queues; q++) {
+ q < priv->trans->trans_cfg->base_params->num_of_queues; q++) {
if (!test_and_set_bit(q, priv->agg_q_alloc)) {
priv->queue_to_mac80211[q] = mq;
return q;
@@ -1281,7 +1282,7 @@ void iwlagn_rx_reply_compressed_ba(struct iwl_priv *priv,
* (in Tx queue's circular buffer) of first TFD/frame in window */
u16 ba_resp_scd_ssn = le16_to_cpu(ba_resp->scd_ssn);
- if (scd_flow >= priv->cfg->base_params->num_of_queues) {
+ if (scd_flow >= priv->trans->trans_cfg->base_params->num_of_queues) {
IWL_ERR(priv,
"BUG_ON scd_flow is bigger than number of queues\n");
return;
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/acpi.h b/drivers/net/wireless/intel/iwlwifi/fw/acpi.h
index 991a234..6cb2d1f 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/acpi.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/acpi.h
@@ -68,6 +68,7 @@
#define ACPI_WRDD_METHOD "WRDD"
#define ACPI_SPLC_METHOD "SPLC"
#define ACPI_ECKV_METHOD "ECKV"
+#define ACPI_PPAG_METHOD "PPAG"
#define ACPI_WIFI_DOMAIN (0x07)
@@ -92,6 +93,17 @@
#define ACPI_WGDS_NUM_BANDS 2
#define ACPI_WGDS_TABLE_SIZE 3
+#define ACPI_PPAG_NUM_CHAINS 2
+#define ACPI_PPAG_NUM_SUB_BANDS 5
+#define ACPI_PPAG_WIFI_DATA_SIZE ((ACPI_PPAG_NUM_CHAINS * \
+ ACPI_PPAG_NUM_SUB_BANDS) + 3)
+
+/* PPAG gain value bounds in 1/8 dBm */
+#define ACPI_PPAG_MIN_LB -16
+#define ACPI_PPAG_MAX_LB 24
+#define ACPI_PPAG_MIN_HB -16
+#define ACPI_PPAG_MAX_HB 40
+
#ifdef CONFIG_ACPI
void *iwl_acpi_get_object(struct device *dev, acpi_string method);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h b/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h
index 4d2274b..22dff2c 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/commands.h
@@ -475,6 +475,13 @@ enum iwl_legacy_cmds {
REPLY_RX_MPDU_CMD = 0xc1,
/**
+ * @BAR_FRAME_RELEASE: Frame release from BAR notification, used for
+ * multi-TID BAR (previously, the BAR frame itself was reported
+ * instead). Uses &struct iwl_bar_frame_release.
+ */
+ BAR_FRAME_RELEASE = 0xc2,
+
+ /**
* @FRAME_RELEASE:
* Frame release (reorder helper) notification, uses
* &struct iwl_frame_release
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
index 31231b2..4c3219e 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/d3.h
@@ -396,6 +396,7 @@ enum iwl_wowlan_flags {
* @is_11n_connection: indicates HT connection
* @offloading_tid: TID reserved for firmware use
* @flags: extra flags, see &enum iwl_wowlan_flags
+ * @sta_id: station ID for wowlan.
* @reserved: reserved
*/
struct iwl_wowlan_config_cmd {
@@ -406,8 +407,9 @@ struct iwl_wowlan_config_cmd {
u8 is_11n_connection;
u8 offloading_tid;
u8 flags;
- u8 reserved[2];
-} __packed; /* WOWLAN_CONFIG_API_S_VER_4 */
+ u8 sta_id;
+ u8 reserved;
+} __packed; /* WOWLAN_CONFIG_API_S_VER_5 */
/*
* WOWLAN_TSC_RSC_PARAMS
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h b/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h
index aaf3974..ba586f1 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h
@@ -140,17 +140,6 @@ struct iwl_fw_ini_hcmd_tlv {
struct iwl_fw_ini_hcmd hcmd;
} __packed; /* FW_DEBUG_TLV_HCMD_API_S_VER_1 */
-/**
- * struct iwl_fw_ini_debug_flow_tlv - (IWL_UCODE_TLV_TYPE_DEBUG_FLOW)
- *
- * @header: header
- * @debug_flow_cfg: &enum iwl_fw_ini_debug_flow
- */
-struct iwl_fw_ini_debug_flow_tlv {
- struct iwl_fw_ini_header header;
- __le32 debug_flow_cfg;
-} __packed; /* FW_DEBUG_TLV_FLOW_TLV_S_VER_1 */
-
#define IWL_FW_INI_MAX_REGION_ID 64
#define IWL_FW_INI_MAX_NAME 32
@@ -409,27 +398,6 @@ enum iwl_fw_ini_trigger_id {
}; /* FW_DEBUG_TLV_TRIGGER_ID_E_VER_1 */
/**
- * enum iwl_fw_ini_apply_point
- *
- * @IWL_FW_INI_APPLY_INVALID: invalid
- * @IWL_FW_INI_APPLY_EARLY: pre loading FW
- * @IWL_FW_INI_APPLY_AFTER_ALIVE: first cmd from host after alive
- * @IWL_FW_INI_APPLY_POST_INIT: last cmd in initialization sequence
- * @IWL_FW_INI_APPLY_MISSED_BEACONS: missed beacons notification
- * @IWL_FW_INI_APPLY_SCAN_COMPLETE: scan completed
- * @IWL_FW_INI_APPLY_NUM: number of apply points
-*/
-enum iwl_fw_ini_apply_point {
- IWL_FW_INI_APPLY_INVALID,
- IWL_FW_INI_APPLY_EARLY,
- IWL_FW_INI_APPLY_AFTER_ALIVE,
- IWL_FW_INI_APPLY_POST_INIT,
- IWL_FW_INI_APPLY_MISSED_BEACONS,
- IWL_FW_INI_APPLY_SCAN_COMPLETE,
- IWL_FW_INI_APPLY_NUM,
-}; /* FW_DEBUG_TLV_APPLY_POINT_E_VER_1 */
-
-/**
* enum iwl_fw_ini_allocation_id
*
* @IWL_FW_INI_ALLOCATION_INVALID: invalid
@@ -439,6 +407,7 @@ enum iwl_fw_ini_apply_point {
* @IWL_FW_INI_ALLOCATION_ID_SDFX: for SDFX module
* @IWL_FW_INI_ALLOCATION_ID_FW_DUMP: used for crash and runtime dumps
* @IWL_FW_INI_ALLOCATION_ID_USER_DEFINED: for future user scenarios
+ * @IWL_FW_INI_ALLOCATION_NUM: number of allocation ids
*/
enum iwl_fw_ini_allocation_id {
IWL_FW_INI_ALLOCATION_INVALID,
@@ -448,6 +417,7 @@ enum iwl_fw_ini_allocation_id {
IWL_FW_INI_ALLOCATION_ID_SDFX,
IWL_FW_INI_ALLOCATION_ID_FW_DUMP,
IWL_FW_INI_ALLOCATION_ID_USER_DEFINED,
+ IWL_FW_INI_ALLOCATION_NUM,
}; /* FW_DEBUG_TLV_ALLOCATION_ID_E_VER_1 */
/**
@@ -519,4 +489,72 @@ enum iwl_fw_ini_region_type {
IWL_FW_INI_REGION_NUM
}; /* FW_DEBUG_TLV_REGION_TYPE_E_VER_1 */
+/**
+ * enum iwl_fw_ini_time_point
+ *
+ * Hard coded time points in which the driver can send hcmd or perform dump
+ * collection
+ *
+ * @IWL_FW_INI_TIME_POINT_EARLY: pre loading the FW
+ * @IWL_FW_INI_TIME_POINT_AFTER_ALIVE: first cmd from host after alive notif
+ * @IWL_FW_INI_TIME_POINT_POST_INIT: last cmd in series of init sequence
+ * @IWL_FW_INI_TIME_POINT_FW_ASSERT: FW assert
+ * @IWL_FW_INI_TIME_POINT_FW_HW_ERROR: FW HW error
+ * @IWL_FW_INI_TIME_POINT_FW_TFD_Q_HANG: TFD queue hang
+ * @IWL_FW_INI_TIME_POINT_FW_DHC_NOTIFOCATION: DHC cmd response and notif
+ * @IWL_FW_INI_TIME_POINT_FW_RSP_OR_NOTIF: FW response or notification.
+ * data field holds id and group
+ * @IWL_FW_INI_TIME_POINT_USER_TRIGGER: user trigger time point
+ * @IWL_FW_INI_TIME_POINT_PERIODIC: periodic timepoint that fires in constant
+ * intervals. data field holds the interval time in msec
+ * @IWL_FW_INI_TIME_POINT_WDG_TIMEOUT: watchdog timeout
+ * @IWL_FW_INI_TIME_POINT_HOST_ASSERT: Unused
+ * @IWL_FW_INI_TIME_POINT_HOST_ALIVE_TIMEOUT: alive timeout
+ * @IWL_FW_INI_TIME_POINT_HOST_DEVICE_ENABLE: device enable
+ * @IWL_FW_INI_TIME_POINT_HOST_DEVICE_DISABLE: device disable
+ * @IWL_FW_INI_TIME_POINT_HOST_D3_START: D3 start
+ * @IWL_FW_INI_TIME_POINT_HOST_D3_END: D3 end
+ * @IWL_FW_INI_TIME_POINT_MISSED_BEACONS: missed beacons
+ * @IWL_FW_INI_TIME_POINT_ASSOC_FAILED: association failure
+ * @IWL_FW_INI_TIME_POINT_TX_FAILED: Tx frame failed
+ * @IWL_FW_INI_TIME_POINT_TX_WFD_ACTION_FRAME_FAILED: wifi direct action
+ * frame failed
+ * @IWL_FW_INI_TIME_POINT_TX_LATENCY_THRESHOLD: Tx latency threshold
+ * @IWL_FW_INI_TIME_POINT_HANG_OCCURRED: hang occurred
+ * @IWL_FW_INI_TIME_POINT_EAPOL_FAILED: EAPOL failed
+ * @IWL_FW_INI_TIME_POINT_FAKE_TX: fake Tx
+ * @IWL_FW_INI_TIME_POINT_DEASSOC: de association
+ * @IWL_FW_INI_TIME_POINT_NUM: number of time points
+ */
+enum iwl_fw_ini_time_point {
+ IWL_FW_INI_TIME_POINT_INVALID,
+ IWL_FW_INI_TIME_POINT_EARLY,
+ IWL_FW_INI_TIME_POINT_AFTER_ALIVE,
+ IWL_FW_INI_TIME_POINT_POST_INIT,
+ IWL_FW_INI_TIME_POINT_FW_ASSERT,
+ IWL_FW_INI_TIME_POINT_FW_HW_ERROR,
+ IWL_FW_INI_TIME_POINT_FW_TFD_Q_HANG,
+ IWL_FW_INI_TIME_POINT_FW_DHC_NOTIFOCATION,
+ IWL_FW_INI_TIME_POINT_FW_RSP_OR_NOTIF,
+ IWL_FW_INI_TIME_POINT_USER_TRIGGER,
+ IWL_FW_INI_TIME_POINT_PERIODIC,
+ IWL_FW_INI_TIME_POINT_WDG_TIMEOUT,
+ IWL_FW_INI_TIME_POINT_HOST_ASSERT,
+ IWL_FW_INI_TIME_POINT_HOST_ALIVE_TIMEOUT,
+ IWL_FW_INI_TIME_POINT_HOST_DEVICE_ENABLE,
+ IWL_FW_INI_TIME_POINT_HOST_DEVICE_DISABLE,
+ IWL_FW_INI_TIME_POINT_HOST_D3_START,
+ IWL_FW_INI_TIME_POINT_HOST_D3_END,
+ IWL_FW_INI_TIME_POINT_MISSED_BEACONS,
+ IWL_FW_INI_TIME_POINT_ASSOC_FAILED,
+ IWL_FW_INI_TIME_POINT_TX_FAILED,
+ IWL_FW_INI_TIME_POINT_TX_WFD_ACTION_FRAME_FAILED,
+ IWL_FW_INI_TIME_POINT_TX_LATENCY_THRESHOLD,
+ IWL_FW_INI_TIME_POINT_HANG_OCCURRED,
+ IWL_FW_INI_TIME_POINT_EAPOL_FAILED,
+ IWL_FW_INI_TIME_POINT_FAKE_TX,
+ IWL_FW_INI_TIME_POINT_DEASSOC,
+ IWL_FW_INI_TIME_POINT_NUM,
+}; /* FW_TLV_DEBUG_TIME_POINT_API_E */
+
#endif
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h b/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h
index 9885849..98e957e 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/debug.h
@@ -8,7 +8,7 @@
* Copyright(c) 2007 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -31,7 +31,7 @@
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -81,6 +81,19 @@ enum iwl_debug_cmds {
*/
UMAC_RD_WR = 0x1,
/**
+ * @DBGC_SUSPEND_RESUME:
+ * DBGC suspend/resume commad. Uses a single dword as data:
+ * 0 - resume DBGC recording
+ * 1 - suspend DBGC recording
+ */
+ DBGC_SUSPEND_RESUME = 0x7,
+ /**
+ * @BUFFER_ALLOCATION:
+ * passes DRAM buffers to a DBGC
+ * &struct iwl_buf_alloc_cmd
+ */
+ BUFFER_ALLOCATION = 0x8,
+ /**
* @MFU_ASSERT_DUMP_NTF:
* &struct iwl_mfu_assert_dump_notif
*/
@@ -102,6 +115,16 @@ enum {
FW_ERR_FATAL = 0xFF
};
+/** enum iwl_dbg_suspend_resume_cmds - dbgc suspend resume operations
+ * dbgc suspend resume command operations
+ * @DBGC_RESUME_CMD: resume dbgc recording
+ * @DBGC_SUSPEND_CMD: stop dbgc recording
+ */
+enum iwl_dbg_suspend_resume_cmds {
+ DBGC_RESUME_CMD,
+ DBGC_SUSPEND_CMD,
+};
+
/**
* struct iwl_error_resp - FW error indication
* ( REPLY_ERROR = 0x2 )
@@ -335,49 +358,39 @@ struct iwl_dbg_mem_access_rsp {
__le32 data[];
} __packed; /* DEBUG_(U|L)MAC_RD_WR_RSP_API_S_VER_1 */
-#define LDBG_CFG_COMMAND_SIZE 80
-#define BUFFER_ALLOCATION 0x27
-#define START_DEBUG_RECORDING 0x29
-#define STOP_DEBUG_RECORDING 0x2A
+/**
+ * struct iwl_dbg_suspend_resume_cmd - dbgc suspend resume command
+ * @operation: suspend or resume operation, uses
+ * &enum iwl_dbg_suspend_resume_cmds
+ */
+struct iwl_dbg_suspend_resume_cmd {
+ __le32 operation;
+} __packed;
-/* maximum fragments to be allocated per target of allocationId */
-#define IWL_BUFFER_LOCATION_MAX_FRAGS 2
+#define BUF_ALLOC_MAX_NUM_FRAGS 16
/**
- * struct iwl_fragment_data single fragment structure
- * @address: 64bit start address
- * @size: size in bytes
+ * struct iwl_buf_alloc_frag - a DBGC fragment
+ * @addr: base address of the fragment
+ * @size: size of the fragment
*/
-struct iwl_fragment_data {
- __le64 address;
+struct iwl_buf_alloc_frag {
+ __le64 addr;
__le32 size;
} __packed; /* FRAGMENT_STRUCTURE_API_S_VER_1 */
/**
- * struct iwl_buffer_allocation_cmd - buffer allocation command structure
- * @allocation_id: id of the allocation
- * @buffer_location: location of the buffer
+ * struct iwl_buf_alloc_cmd - buffer allocation command
+ * @alloc_id: &enum iwl_fw_ini_allocation_id
+ * @buf_location: &enum iwl_fw_ini_buffer_location
* @num_frags: number of fragments
- * @fragments: memory fragments
+ * @frags: fragments array
*/
-struct iwl_buffer_allocation_cmd {
- __le32 allocation_id;
- __le32 buffer_location;
+struct iwl_buf_alloc_cmd {
+ __le32 alloc_id;
+ __le32 buf_location;
__le32 num_frags;
- struct iwl_fragment_data fragments[IWL_BUFFER_LOCATION_MAX_FRAGS];
-} __packed; /* BUFFER_ALLOCATION_CMD_API_S_VER_1 */
-
-/**
- * struct iwl_ldbg_config_cmd - LDBG config command
- * @type: configuration type
- * @pad: reserved space for type-dependent data
- */
-struct iwl_ldbg_config_cmd {
- __le32 type;
- union {
- u8 pad[LDBG_CFG_COMMAND_SIZE - sizeof(__le32)];
- struct iwl_buffer_allocation_cmd buffer_allocation;
- }; /* LDBG_CFG_BODY_API_U_VER_2 (partially) */
-} __packed; /* LDBG_CFG_CMD_API_S_VER_2 */
+ struct iwl_buf_alloc_frag frags[BUF_ALLOC_MAX_NUM_FRAGS];
+} __packed; /* BUFFER_ALLOCATION_CMD_API_S_VER_2 */
#endif /* __iwl_fw_api_debug_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/location.h b/drivers/net/wireless/intel/iwlwifi/fw/api/location.h
index ec864c7..7a0fe5a 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/location.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/location.h
@@ -82,7 +82,7 @@ enum iwl_location_subcmd_ids {
TOF_RANGE_ABORT_CMD = 0x2,
/**
* @TOF_RANGE_REQ_EXT_CMD: TOF extended ranging config,
- * uses &struct iwl_tof_range_request_ext_cmd
+ * uses &struct iwl_tof_range_req_ext_cmd
*/
TOF_RANGE_REQ_EXT_CMD = 0x3,
/**
@@ -292,7 +292,7 @@ struct iwl_tof_responder_dyn_config_cmd {
} __packed; /* TOF_RESPONDER_DYN_CONFIG_CMD_API_S_VER_2 */
/**
- * struct iwl_tof_range_request_ext_cmd - extended range req for WLS
+ * struct iwl_tof_range_req_ext_cmd - extended range req for WLS
* @tsf_timer_offset_msec: the recommended time offset (mSec) from the AP's TSF
* @reserved: reserved
* @min_delta_ftm: Minimal time between two consecutive measurements,
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/mac.h b/drivers/net/wireless/intel/iwlwifi/fw/api/mac.h
index 85c5e36..73fb003 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/mac.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/mac.h
@@ -500,6 +500,9 @@ struct iwl_he_pkt_ext {
* enabled AGG, i.e. both BACK and non-BACK frames in a single AGG
* @STA_CTXT_HE_MU_EDCA_CW: indicates that there is an element of MU EDCA
* parameter set, i.e. the backoff counters for trig-based ACs
+ * @STA_CTXT_HE_RU_2MHZ_BLOCK: indicates that 26-tone RU OFDMA transmission are
+ * not allowed (as there are OBSS that might classify such transmissions as
+ * radar pulses).
*/
enum iwl_he_sta_ctxt_flags {
STA_CTXT_HE_REF_BSSID_VALID = BIT(4),
@@ -511,6 +514,7 @@ enum iwl_he_sta_ctxt_flags {
STA_CTXT_HE_CONST_TRIG_RND_ALLOC = BIT(10),
STA_CTXT_HE_ACK_ENABLED = BIT(11),
STA_CTXT_HE_MU_EDCA_CW = BIT(12),
+ STA_CTXT_HE_RU_2MHZ_BLOCK = BIT(14),
};
/**
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/phy.h b/drivers/net/wireless/intel/iwlwifi/fw/api/phy.h
index 9cc59e0..8991ddf 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/phy.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/phy.h
@@ -8,6 +8,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -30,6 +31,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -90,6 +92,11 @@ enum iwl_phy_ops_subcmd_ids {
GEO_TX_POWER_LIMIT = 0x05,
/**
+ * @PER_PLATFORM_ANT_GAIN_CMD: &struct iwl_ppag_table_cmd
+ */
+ PER_PLATFORM_ANT_GAIN_CMD = 0x07,
+
+ /**
* @CT_KILL_NOTIFICATION: &struct ct_kill_notif
*/
CT_KILL_NOTIFICATION = 0xFE,
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/power.h b/drivers/net/wireless/intel/iwlwifi/fw/api/power.h
index f195db3..6e1b9b2 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/power.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/power.h
@@ -450,6 +450,18 @@ struct iwl_geo_tx_power_profiles_resp {
} __packed; /* GEO_TX_POWER_LIMIT_RESP */
/**
+ * struct iwl_ppag_table_cmd - struct for PER_PLATFORM_ANT_GAIN_CMD cmd.
+ * @enabled: 1 if PPAG is enabled, 0 otherwise
+ * @gain: table of antenna gain values per chain and sub-band
+ * @reserved: reserved
+ */
+struct iwl_ppag_table_cmd {
+ __le32 enabled;
+ s8 gain[IWL_NUM_CHAIN_LIMITS][IWL_NUM_SUB_BANDS];
+ s8 reserved[2];
+} __packed; /* PER_PLATFORM_ANT_GAIN_CMD */
+
+/**
* struct iwl_beacon_filter_cmd
* REPLY_BEACON_FILTERING_CMD = 0xd2 (command)
* @bf_energy_delta: Used for RSSI filtering, if in 'normal' state. Send beacon
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h b/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h
index 9eddc4d..4347be6 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/rs.h
@@ -7,7 +7,7 @@
*
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -29,7 +29,7 @@
*
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -166,8 +166,16 @@ enum iwl_tlc_mng_ht_rates {
IWL_TLC_MNG_HT_RATE_MAX = IWL_TLC_MNG_HT_RATE_MCS11,
};
-/* Maximum supported tx antennas number */
-#define MAX_NSS 2
+enum IWL_TLC_MNG_NSS {
+ IWL_TLC_NSS_1,
+ IWL_TLC_NSS_2,
+ IWL_TLC_NSS_MAX
+};
+
+enum IWL_TLC_HT_BW_RATES {
+ IWL_TLC_HT_BW_NONE_160,
+ IWL_TLC_HT_BW_160,
+};
/**
* struct tlc_config_cmd - TLC configuration
@@ -195,7 +203,7 @@ struct iwl_tlc_config_cmd {
u8 amsdu;
__le16 flags;
__le16 non_ht_rates;
- __le16 ht_rates[MAX_NSS][2];
+ __le16 ht_rates[IWL_TLC_NSS_MAX][2];
__le16 max_mpdu_len;
u8 sgi_ch_width_supp;
u8 reserved2[1];
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h b/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h
index 9b0bb89..a93449d 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/rx.h
@@ -746,6 +746,38 @@ struct iwl_frame_release {
__le16 nssn;
};
+/**
+ * enum iwl_bar_frame_release_sta_tid - STA/TID information for BAR release
+ * @IWL_BAR_FRAME_RELEASE_TID_MASK: TID mask
+ * @IWL_BAR_FRAME_RELEASE_STA_MASK: STA mask
+ */
+enum iwl_bar_frame_release_sta_tid {
+ IWL_BAR_FRAME_RELEASE_TID_MASK = 0x0000000f,
+ IWL_BAR_FRAME_RELEASE_STA_MASK = 0x000001f0,
+};
+
+/**
+ * enum iwl_bar_frame_release_ba_info - BA information for BAR release
+ * @IWL_BAR_FRAME_RELEASE_NSSN_MASK: NSSN mask
+ * @IWL_BAR_FRAME_RELEASE_SN_MASK: SN mask (ignored by driver)
+ * @IWL_BAR_FRAME_RELEASE_BAID_MASK: BAID mask
+ */
+enum iwl_bar_frame_release_ba_info {
+ IWL_BAR_FRAME_RELEASE_NSSN_MASK = 0x00000fff,
+ IWL_BAR_FRAME_RELEASE_SN_MASK = 0x00fff000,
+ IWL_BAR_FRAME_RELEASE_BAID_MASK = 0x3f000000,
+};
+
+/**
+ * struct iwl_bar_frame_release - frame release from BAR info
+ * @sta_tid: STA & TID information, see &enum iwl_bar_frame_release_sta_tid.
+ * @ba_info: BA information, see &enum iwl_bar_frame_release_ba_info.
+ */
+struct iwl_bar_frame_release {
+ __le32 sta_tid;
+ __le32 ba_info;
+} __packed; /* RX_BAR_TO_FRAME_RELEASE_API_S_VER_1 */
+
enum iwl_rss_hash_func_en {
IWL_RSS_HASH_TYPE_IPV4_TCP,
IWL_RSS_HASH_TYPE_IPV4_UDP,
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
index c4960f0..39c64850 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/api/scan.h
@@ -93,6 +93,8 @@ struct iwl_ssid_ie {
#define IWL_SCAN_SHORT_BLACKLIST_LEN 16
#define IWL_SCAN_MAX_PROFILES 11
#define SCAN_OFFLOAD_PROBE_REQ_SIZE 512
+#define SCAN_NUM_BAND_PROBE_DATA_V_1 2
+#define SCAN_NUM_BAND_PROBE_DATA_V_2 3
/* Default watchdog (in MS) for scheduled scan iteration */
#define IWL_SCHED_SCAN_WATCHDOG cpu_to_le16(15000)
@@ -251,9 +253,22 @@ struct iwl_scan_probe_segment {
* @common_data: last (and common) part of the probe
* @buf: raw data block
*/
+struct iwl_scan_probe_req_v1 {
+ struct iwl_scan_probe_segment mac_header;
+ struct iwl_scan_probe_segment band_data[SCAN_NUM_BAND_PROBE_DATA_V_1];
+ struct iwl_scan_probe_segment common_data;
+ u8 buf[SCAN_OFFLOAD_PROBE_REQ_SIZE];
+} __packed;
+
+/* iwl_scan_probe_req - PROBE_REQUEST_FRAME_API_S_VER_v2
+ * @mac_header: first (and common) part of the probe
+ * @band_data: band specific data
+ * @common_data: last (and common) part of the probe
+ * @buf: raw data block
+ */
struct iwl_scan_probe_req {
struct iwl_scan_probe_segment mac_header;
- struct iwl_scan_probe_segment band_data[2];
+ struct iwl_scan_probe_segment band_data[SCAN_NUM_BAND_PROBE_DATA_V_2];
struct iwl_scan_probe_segment common_data;
u8 buf[SCAN_OFFLOAD_PROBE_REQ_SIZE];
} __packed;
@@ -608,15 +623,29 @@ enum iwl_umac_scan_general_flags2 {
* struct iwl_scan_channel_cfg_umac
* @flags: bitmap - 0-19: directed scan to i'th ssid.
* @channel_num: channel number 1-13 etc.
+ * @band: band of channel: 0 for 2GHz, 1 for 5GHz
* @iter_count: repetition count for the channel.
* @iter_interval: interval between two scan iterations on one channel.
*/
-struct iwl_scan_channel_cfg_umac {
+struct iwl_scan_channel_cfg_umac {
__le32 flags;
- u8 channel_num;
- u8 iter_count;
- __le16 iter_interval;
-} __packed; /* SCAN_CHANNEL_CFG_S_VER2 */
+ /* Both versions are of the same size, so use a union without adjusting
+ * the command size later
+ */
+ union {
+ struct {
+ u8 channel_num;
+ u8 iter_count;
+ __le16 iter_interval;
+ } v1; /* SCAN_CHANNEL_CFG_S_VER1 */
+ struct {
+ u8 channel_num;
+ u8 band;
+ u8 iter_count;
+ u8 iter_interval;
+ } v2; /* SCAN_CHANNEL_CFG_S_VER2 */
+ };
+} __packed;
/**
* struct iwl_scan_umac_schedule
@@ -630,6 +659,16 @@ struct iwl_scan_umac_schedule {
u8 reserved;
} __packed; /* SCAN_SCHED_PARAM_API_S_VER_1 */
+struct iwl_scan_req_umac_tail_v1 {
+ /* SCAN_PERIODIC_PARAMS_API_S_VER_1 */
+ struct iwl_scan_umac_schedule schedule[IWL_MAX_SCHED_SCAN_PLANS];
+ __le16 delay;
+ __le16 reserved;
+ /* SCAN_PROBE_PARAMS_API_S_VER_1 */
+ struct iwl_scan_probe_req_v1 preq;
+ struct iwl_ssid_ie direct_scan[PROBE_OPTION_MAX];
+} __packed;
+
/**
* struct iwl_scan_req_umac_tail - the rest of the UMAC scan request command
* parameters following channels configuration array.
@@ -639,12 +678,12 @@ struct iwl_scan_umac_schedule {
* @preq: probe request with IEs blocks
* @direct_scan: list of SSIDs for directed active scan
*/
-struct iwl_scan_req_umac_tail {
+struct iwl_scan_req_umac_tail_v2 {
/* SCAN_PERIODIC_PARAMS_API_S_VER_1 */
struct iwl_scan_umac_schedule schedule[IWL_MAX_SCHED_SCAN_PLANS];
__le16 delay;
__le16 reserved;
- /* SCAN_PROBE_PARAMS_API_S_VER_1 */
+ /* SCAN_PROBE_PARAMS_API_S_VER_2 */
struct iwl_scan_probe_req preq;
struct iwl_ssid_ie direct_scan[PROBE_OPTION_MAX];
} __packed;
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
index 4d81776..5c8602d 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dbg.c
@@ -243,7 +243,7 @@ static void iwl_fw_dump_rxf(struct iwl_fw_runtime *fwrt,
/* Pull RXF2 */
iwl_fwrt_dump_rxf(fwrt, dump_data, cfg->rxfifo2_size,
RXF_DIFF_FROM_PREV +
- fwrt->trans->cfg->umac_prph_offset, 1);
+ fwrt->trans->trans_cfg->umac_prph_offset, 1);
/* Pull LMAC2 RXF1 */
if (fwrt->smem_cfg.num_lmacs > 1)
iwl_fwrt_dump_rxf(fwrt, dump_data,
@@ -468,6 +468,9 @@ static const struct iwl_prph_range iwl_prph_dump_addr_9000[] = {
{ .start = 0x00a05400, .end = 0x00a056e8 },
{ .start = 0x00a08000, .end = 0x00a098bc },
{ .start = 0x00a02400, .end = 0x00a02758 },
+ { .start = 0x00a04764, .end = 0x00a0476c },
+ { .start = 0x00a04770, .end = 0x00a04774 },
+ { .start = 0x00a04620, .end = 0x00a04624 },
};
static const struct iwl_prph_range iwl_prph_dump_addr_22000[] = {
@@ -681,17 +684,18 @@ static void iwl_fw_prph_handler(struct iwl_fw_runtime *fwrt, void *ptr,
{
u32 range_len;
- if (fwrt->trans->cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
+ if (fwrt->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
range_len = ARRAY_SIZE(iwl_prph_dump_addr_ax210);
handler(fwrt, iwl_prph_dump_addr_ax210, range_len, ptr);
- } else if (fwrt->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000) {
+ } else if (fwrt->trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_22000) {
range_len = ARRAY_SIZE(iwl_prph_dump_addr_22000);
handler(fwrt, iwl_prph_dump_addr_22000, range_len, ptr);
} else {
range_len = ARRAY_SIZE(iwl_prph_dump_addr_comm);
handler(fwrt, iwl_prph_dump_addr_comm, range_len, ptr);
- if (fwrt->trans->cfg->mq_rx_supported) {
+ if (fwrt->trans->trans_cfg->mq_rx_supported) {
range_len = ARRAY_SIZE(iwl_prph_dump_addr_9000);
handler(fwrt, iwl_prph_dump_addr_9000, range_len, ptr);
}
@@ -853,7 +857,8 @@ iwl_fw_error_dump_file(struct iwl_fw_runtime *fwrt,
iwl_fw_prph_handler(fwrt, &prph_len,
iwl_fw_get_prph_len);
- if (fwrt->trans->cfg->device_family == IWL_DEVICE_FAMILY_7000 &&
+ if (fwrt->trans->trans_cfg->device_family ==
+ IWL_DEVICE_FAMILY_7000 &&
iwl_fw_dbg_type_on(fwrt, IWL_FW_ERROR_DUMP_RADIO_REG))
radio_len = sizeof(*dump_data) + RADIO_REG_MAX_READ;
}
@@ -1103,25 +1108,9 @@ static int iwl_dump_ini_dev_mem_iter(struct iwl_fw_runtime *fwrt,
return sizeof(*range) + le32_to_cpu(range->range_data_size);
}
-static int
-iwl_dump_ini_paging_gen2_iter(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_region_cfg *reg,
- void *range_ptr, int idx)
-{
- struct iwl_fw_ini_error_dump_range *range = range_ptr;
- u32 page_size = fwrt->trans->init_dram.paging[idx].size;
-
- range->page_num = cpu_to_le32(idx);
- range->range_data_size = cpu_to_le32(page_size);
- memcpy(range->data, fwrt->trans->init_dram.paging[idx].block,
- page_size);
-
- return sizeof(*range) + le32_to_cpu(range->range_data_size);
-}
-
-static int iwl_dump_ini_paging_iter(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_region_cfg *reg,
- void *range_ptr, int idx)
+static int _iwl_dump_ini_paging_iter(struct iwl_fw_runtime *fwrt,
+ struct iwl_fw_ini_region_cfg *reg,
+ void *range_ptr, int idx)
{
/* increase idx by 1 since the pages are from 1 to
* fwrt->num_of_paging_blk + 1
@@ -1142,6 +1131,27 @@ static int iwl_dump_ini_paging_iter(struct iwl_fw_runtime *fwrt,
return sizeof(*range) + le32_to_cpu(range->range_data_size);
}
+static int iwl_dump_ini_paging_iter(struct iwl_fw_runtime *fwrt,
+ struct iwl_fw_ini_region_cfg *reg,
+ void *range_ptr, int idx)
+{
+ struct iwl_fw_ini_error_dump_range *range;
+ u32 page_size;
+
+ if (!fwrt->trans->trans_cfg->gen2)
+ return _iwl_dump_ini_paging_iter(fwrt, reg, range_ptr, idx);
+
+ range = range_ptr;
+ page_size = fwrt->trans->init_dram.paging[idx].size;
+
+ range->page_num = cpu_to_le32(idx);
+ range->range_data_size = cpu_to_le32(page_size);
+ memcpy(range->data, fwrt->trans->init_dram.paging[idx].block,
+ page_size);
+
+ return sizeof(*range) + le32_to_cpu(range->range_data_size);
+}
+
static int
iwl_dump_ini_mon_dram_iter(struct iwl_fw_runtime *fwrt,
struct iwl_fw_ini_region_cfg *reg, void *range_ptr,
@@ -1163,35 +1173,23 @@ iwl_dump_ini_mon_dram_iter(struct iwl_fw_runtime *fwrt,
return sizeof(*range) + le32_to_cpu(range->range_data_size);
}
-struct iwl_ini_txf_iter_data {
- int fifo;
- int lmac;
- u32 fifo_size;
- bool internal_txf;
- bool init;
-};
-
static bool iwl_ini_txf_iter(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_region_cfg *reg)
+ struct iwl_fw_ini_region_cfg *reg, int idx)
{
- struct iwl_ini_txf_iter_data *iter = fwrt->dump.fifo_iter;
+ struct iwl_txf_iter_data *iter = &fwrt->dump.txf_iter_data;
struct iwl_fwrt_shared_mem_cfg *cfg = &fwrt->smem_cfg;
int txf_num = cfg->num_txfifo_entries;
int int_txf_num = ARRAY_SIZE(cfg->internal_txfifo_size);
u32 lmac_bitmap = le32_to_cpu(reg->fifos.fid1);
- if (!iter)
- return false;
-
- if (iter->init) {
+ if (!idx) {
if (le32_to_cpu(reg->offset) &&
WARN_ONCE(cfg->num_lmacs == 1,
"Invalid lmac offset: 0x%x\n",
le32_to_cpu(reg->offset)))
return false;
- iter->init = false;
- iter->internal_txf = false;
+ iter->internal_txf = 0;
iter->fifo_size = 0;
iter->fifo = -1;
if (le32_to_cpu(reg->offset))
@@ -1208,7 +1206,7 @@ static bool iwl_ini_txf_iter(struct iwl_fw_runtime *fwrt,
return true;
}
- iter->internal_txf = true;
+ iter->internal_txf = 1;
if (!fw_has_capa(&fwrt->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_EXTEND_SHARED_MEM_CFG))
@@ -1229,7 +1227,7 @@ static int iwl_dump_ini_txf_iter(struct iwl_fw_runtime *fwrt,
void *range_ptr, int idx)
{
struct iwl_fw_ini_error_dump_range *range = range_ptr;
- struct iwl_ini_txf_iter_data *iter;
+ struct iwl_txf_iter_data *iter = &fwrt->dump.txf_iter_data;
struct iwl_fw_ini_error_dump_register *reg_dump = (void *)range->data;
u32 offs = le32_to_cpu(reg->offset), addr;
u32 registers_size =
@@ -1238,14 +1236,12 @@ static int iwl_dump_ini_txf_iter(struct iwl_fw_runtime *fwrt,
unsigned long flags;
int i;
- if (!iwl_ini_txf_iter(fwrt, reg))
+ if (!iwl_ini_txf_iter(fwrt, reg, idx))
return -EIO;
if (!iwl_trans_grab_nic_access(fwrt->trans, &flags))
return -EBUSY;
- iter = fwrt->dump.fifo_iter;
-
range->fifo_hdr.fifo_num = cpu_to_le32(iter->fifo);
range->fifo_hdr.num_of_registers = reg->fifos.num_of_registers;
range->range_data_size = cpu_to_le32(iter->fifo_size + registers_size);
@@ -1448,7 +1444,7 @@ static void
struct iwl_fw_ini_monitor_dump *mon_dump = (void *)data;
u32 write_ptr_addr, write_ptr_msk, cycle_cnt_addr, cycle_cnt_msk;
- switch (fwrt->trans->cfg->device_family) {
+ switch (fwrt->trans->trans_cfg->device_family) {
case IWL_DEVICE_FAMILY_9000:
case IWL_DEVICE_FAMILY_22000:
write_ptr_addr = MON_BUFF_WRPTR_VER2;
@@ -1458,7 +1454,7 @@ static void
break;
default:
IWL_ERR(fwrt, "Unsupported device family %d\n",
- fwrt->trans->cfg->device_family);
+ fwrt->trans->trans_cfg->device_family);
return NULL;
}
@@ -1475,10 +1471,10 @@ static void
struct iwl_fw_ini_monitor_dump *mon_dump = (void *)data;
const struct iwl_cfg *cfg = fwrt->trans->cfg;
- if (fwrt->trans->cfg->device_family != IWL_DEVICE_FAMILY_9000 &&
- fwrt->trans->cfg->device_family != IWL_DEVICE_FAMILY_22000) {
+ if (fwrt->trans->trans_cfg->device_family != IWL_DEVICE_FAMILY_9000 &&
+ fwrt->trans->trans_cfg->device_family != IWL_DEVICE_FAMILY_22000) {
IWL_ERR(fwrt, "Unsupported device family %d\n",
- fwrt->trans->cfg->device_family);
+ fwrt->trans->trans_cfg->device_family);
return NULL;
}
@@ -1496,15 +1492,12 @@ static u32 iwl_dump_ini_mem_ranges(struct iwl_fw_runtime *fwrt,
return le32_to_cpu(reg->internal.num_of_ranges);
}
-static u32 iwl_dump_ini_paging_gen2_ranges(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_region_cfg *reg)
-{
- return fwrt->trans->init_dram.paging_cnt;
-}
-
static u32 iwl_dump_ini_paging_ranges(struct iwl_fw_runtime *fwrt,
struct iwl_fw_ini_region_cfg *reg)
{
+ if (fwrt->trans->trans_cfg->gen2)
+ return fwrt->trans->init_dram.paging_cnt;
+
return fwrt->num_of_paging_blk;
}
@@ -1517,16 +1510,11 @@ static u32 iwl_dump_ini_mon_dram_ranges(struct iwl_fw_runtime *fwrt,
static u32 iwl_dump_ini_txf_ranges(struct iwl_fw_runtime *fwrt,
struct iwl_fw_ini_region_cfg *reg)
{
- struct iwl_ini_txf_iter_data iter = { .init = true };
- void *fifo_iter = fwrt->dump.fifo_iter;
u32 num_of_fifos = 0;
- fwrt->dump.fifo_iter = &iter;
- while (iwl_ini_txf_iter(fwrt, reg))
+ while (iwl_ini_txf_iter(fwrt, reg, num_of_fifos))
num_of_fifos++;
- fwrt->dump.fifo_iter = fifo_iter;
-
return num_of_fifos;
}
@@ -1548,20 +1536,6 @@ static u32 iwl_dump_ini_mem_get_size(struct iwl_fw_runtime *fwrt,
le32_to_cpu(reg->internal.range_data_size));
}
-static u32 iwl_dump_ini_paging_gen2_get_size(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_region_cfg *reg)
-{
- int i;
- u32 range_header_len = sizeof(struct iwl_fw_ini_error_dump_range);
- u32 size = sizeof(struct iwl_fw_ini_error_dump);
-
- for (i = 0; i < iwl_dump_ini_paging_gen2_ranges(fwrt, reg); i++)
- size += range_header_len +
- fwrt->trans->init_dram.paging[i].size;
-
- return size;
-}
-
static u32 iwl_dump_ini_paging_get_size(struct iwl_fw_runtime *fwrt,
struct iwl_fw_ini_region_cfg *reg)
{
@@ -1569,8 +1543,15 @@ static u32 iwl_dump_ini_paging_get_size(struct iwl_fw_runtime *fwrt,
u32 range_header_len = sizeof(struct iwl_fw_ini_error_dump_range);
u32 size = sizeof(struct iwl_fw_ini_error_dump);
- for (i = 1; i <= iwl_dump_ini_paging_ranges(fwrt, reg); i++)
- size += range_header_len + fwrt->fw_paging_db[i].fw_paging_size;
+ if (fwrt->trans->trans_cfg->gen2) {
+ for (i = 0; i < iwl_dump_ini_paging_ranges(fwrt, reg); i++)
+ size += range_header_len +
+ fwrt->trans->init_dram.paging[i].size;
+ } else {
+ for (i = 1; i <= iwl_dump_ini_paging_ranges(fwrt, reg); i++)
+ size += range_header_len +
+ fwrt->fw_paging_db[i].fw_paging_size;
+ }
return size;
}
@@ -1599,25 +1580,21 @@ static u32 iwl_dump_ini_mon_smem_get_size(struct iwl_fw_runtime *fwrt,
static u32 iwl_dump_ini_txf_get_size(struct iwl_fw_runtime *fwrt,
struct iwl_fw_ini_region_cfg *reg)
{
- struct iwl_ini_txf_iter_data iter = { .init = true };
- void *fifo_iter = fwrt->dump.fifo_iter;
+ struct iwl_txf_iter_data *iter = &fwrt->dump.txf_iter_data;
u32 size = 0;
u32 fifo_hdr = sizeof(struct iwl_fw_ini_error_dump_range) +
le32_to_cpu(reg->fifos.num_of_registers) *
sizeof(struct iwl_fw_ini_error_dump_register);
- fwrt->dump.fifo_iter = &iter;
- while (iwl_ini_txf_iter(fwrt, reg)) {
+ while (iwl_ini_txf_iter(fwrt, reg, size)) {
size += fifo_hdr;
if (!reg->fifos.header_only)
- size += iter.fifo_size;
+ size += iter->fifo_size;
}
if (size)
size += sizeof(struct iwl_fw_ini_error_dump);
- fwrt->dump.fifo_iter = fifo_iter;
-
return size;
}
@@ -1661,38 +1638,50 @@ struct iwl_dump_ini_mem_ops {
};
/**
- * iwl_dump_ini_mem - copy a memory region into the dump
- * @fwrt: fw runtime struct.
- * @data: dump memory data.
- * @reg: region to copy to the dump.
- * @ops: memory dump operations.
+ * iwl_dump_ini_mem
+ *
+ * Creates a dump tlv and copy a memory region into it.
+ * Returns the size of the current dump tlv or 0 if failed
+ *
+ * @fwrt: fw runtime struct
+ * @list: list to add the dump tlv to
+ * @reg: memory region
+ * @ops: memory dump operations
*/
-static void
-iwl_dump_ini_mem(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_error_dump_data **data,
- struct iwl_fw_ini_region_cfg *reg,
- struct iwl_dump_ini_mem_ops *ops)
+static u32 iwl_dump_ini_mem(struct iwl_fw_runtime *fwrt, struct list_head *list,
+ struct iwl_fw_ini_region_cfg *reg,
+ const struct iwl_dump_ini_mem_ops *ops)
{
- struct iwl_fw_ini_error_dump_header *header = (void *)(*data)->data;
+ struct iwl_fw_ini_dump_entry *entry;
+ struct iwl_fw_error_dump_data *tlv;
+ struct iwl_fw_ini_error_dump_header *header;
u32 num_of_ranges, i, type = le32_to_cpu(reg->region_type), size;
void *range;
- if (WARN_ON(!ops || !ops->get_num_of_ranges || !ops->get_size ||
- !ops->fill_mem_hdr || !ops->fill_range))
- return;
+ if (!ops->get_num_of_ranges || !ops->get_size || !ops->fill_mem_hdr ||
+ !ops->fill_range)
+ return 0;
size = ops->get_size(fwrt, reg);
if (!size)
- return;
+ return 0;
- IWL_DEBUG_FW(fwrt, "WRT: collecting region: id=%d, type=%d\n",
+ entry = kmalloc(sizeof(*entry) + sizeof(*tlv) + size, GFP_KERNEL);
+ if (!entry)
+ return 0;
+
+ entry->size = sizeof(*tlv) + size;
+
+ tlv = (void *)entry->data;
+ tlv->type = cpu_to_le32(type);
+ tlv->len = cpu_to_le32(size);
+
+ IWL_DEBUG_FW(fwrt, "WRT: Collecting region: id=%d, type=%d\n",
le32_to_cpu(reg->region_id), type);
num_of_ranges = ops->get_num_of_ranges(fwrt, reg);
- (*data)->type = cpu_to_le32(type);
- (*data)->len = cpu_to_le32(size);
-
+ header = (void *)tlv->data;
header->region_id = reg->region_id;
header->num_of_ranges = cpu_to_le32(num_of_ranges);
header->name_len = cpu_to_le32(min_t(int, IWL_FW_INI_MAX_NAME,
@@ -1702,10 +1691,9 @@ iwl_dump_ini_mem(struct iwl_fw_runtime *fwrt,
range = ops->fill_mem_hdr(fwrt, reg, header);
if (!range) {
IWL_ERR(fwrt,
- "WRT: failed to fill region header: id=%d, type=%d\n",
+ "WRT: Failed to fill region header: id=%d, type=%d\n",
le32_to_cpu(reg->region_id), type);
- memset(*data, 0, size);
- return;
+ goto out_err;
}
for (i = 0; i < num_of_ranges; i++) {
@@ -1713,30 +1701,49 @@ iwl_dump_ini_mem(struct iwl_fw_runtime *fwrt,
if (range_size < 0) {
IWL_ERR(fwrt,
- "WRT: failed to dump region: id=%d, type=%d\n",
+ "WRT: Failed to dump region: id=%d, type=%d\n",
le32_to_cpu(reg->region_id), type);
- memset(*data, 0, size);
- return;
+ goto out_err;
}
range = range + range_size;
}
- *data = iwl_fw_error_next_data(*data);
+
+ list_add_tail(&entry->list, list);
+
+ return entry->size;
+
+out_err:
+ kfree(entry);
+
+ return 0;
}
-static void iwl_dump_ini_info(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_trigger *trigger,
- struct iwl_fw_error_dump_data **data)
+static u32 iwl_dump_ini_info(struct iwl_fw_runtime *fwrt,
+ struct iwl_fw_ini_trigger *trigger,
+ struct list_head *list)
{
- struct iwl_fw_ini_dump_info *dump = (void *)(*data)->data;
+ struct iwl_fw_ini_dump_entry *entry;
+ struct iwl_fw_error_dump_data *tlv;
+ struct iwl_fw_ini_dump_info *dump;
u32 reg_ids_size = le32_to_cpu(trigger->num_regions) * sizeof(__le32);
+ u32 size = sizeof(*tlv) + sizeof(*dump) + reg_ids_size;
- (*data)->type = cpu_to_le32(IWL_INI_DUMP_INFO_TYPE);
- (*data)->len = cpu_to_le32(sizeof(*dump) + reg_ids_size);
+ entry = kmalloc(sizeof(*entry) + size, GFP_KERNEL);
+ if (!entry)
+ return 0;
+
+ entry->size = size;
+
+ tlv = (void *)entry->data;
+ tlv->type = cpu_to_le32(IWL_INI_DUMP_INFO_TYPE);
+ tlv->len = cpu_to_le32(sizeof(*dump) + reg_ids_size);
+
+ dump = (void *)tlv->data;
dump->version = cpu_to_le32(IWL_INI_DUMP_VER);
dump->trigger_id = trigger->trigger_id;
dump->is_external_cfg =
- cpu_to_le32(fwrt->trans->dbg.external_ini_loaded);
+ cpu_to_le32(fwrt->trans->dbg.external_ini_cfg);
dump->ver_type = cpu_to_le32(fwrt->dump.fw_ver.type);
dump->ver_subtype = cpu_to_le32(fwrt->dump.fw_ver.subtype);
@@ -1770,30 +1777,98 @@ static void iwl_dump_ini_info(struct iwl_fw_runtime *fwrt,
dump->external_dbg_cfg_name_len =
cpu_to_le32(sizeof(dump->external_dbg_cfg_name));
- /* dump info size is allocated in iwl_fw_ini_get_trigger_len.
- * The driver allocates (sizeof(*dump) + reg_ids_size) so it is safe to
- * use reg_ids_size
- */
memcpy(dump->external_dbg_cfg_name, fwrt->dump.external_dbg_cfg_name,
sizeof(dump->external_dbg_cfg_name));
dump->regions_num = trigger->num_regions;
memcpy(dump->region_ids, trigger->data, reg_ids_size);
- *data = iwl_fw_error_next_data(*data);
+ /* add dump info TLV to the beginning of the list since it needs to be
+ * the first TLV in the dump
+ */
+ list_add(&entry->list, list);
+
+ return entry->size;
}
-static int iwl_fw_ini_get_trigger_len(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_trigger *trigger)
-{
- int i, ret_size = 0, hdr_len = sizeof(struct iwl_fw_error_dump_data);
- u32 size;
+static const struct iwl_dump_ini_mem_ops iwl_dump_ini_region_ops[] = {
+ [IWL_FW_INI_REGION_INVALID] = {},
+ [IWL_FW_INI_REGION_DEVICE_MEMORY] = {
+ .get_num_of_ranges = iwl_dump_ini_mem_ranges,
+ .get_size = iwl_dump_ini_mem_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_dev_mem_iter,
+ },
+ [IWL_FW_INI_REGION_PERIPHERY_MAC] = {
+ .get_num_of_ranges = iwl_dump_ini_mem_ranges,
+ .get_size = iwl_dump_ini_mem_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_prph_iter,
+ },
+ [IWL_FW_INI_REGION_PERIPHERY_PHY] = {},
+ [IWL_FW_INI_REGION_PERIPHERY_AUX] = {},
+ [IWL_FW_INI_REGION_DRAM_BUFFER] = {
+ .get_num_of_ranges = iwl_dump_ini_mon_dram_ranges,
+ .get_size = iwl_dump_ini_mon_dram_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mon_dram_fill_header,
+ .fill_range = iwl_dump_ini_mon_dram_iter,
+ },
+ [IWL_FW_INI_REGION_DRAM_IMR] = {},
+ [IWL_FW_INI_REGION_INTERNAL_BUFFER] = {
+ .get_num_of_ranges = iwl_dump_ini_mem_ranges,
+ .get_size = iwl_dump_ini_mon_smem_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mon_smem_fill_header,
+ .fill_range = iwl_dump_ini_dev_mem_iter,
+ },
+ [IWL_FW_INI_REGION_TXF] = {
+ .get_num_of_ranges = iwl_dump_ini_txf_ranges,
+ .get_size = iwl_dump_ini_txf_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_txf_iter,
+ },
+ [IWL_FW_INI_REGION_RXF] = {
+ .get_num_of_ranges = iwl_dump_ini_rxf_ranges,
+ .get_size = iwl_dump_ini_rxf_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_rxf_iter,
+ },
+ [IWL_FW_INI_REGION_PAGING] = {
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .get_num_of_ranges = iwl_dump_ini_paging_ranges,
+ .get_size = iwl_dump_ini_paging_get_size,
+ .fill_range = iwl_dump_ini_paging_iter,
+ },
+ [IWL_FW_INI_REGION_CSR] = {
+ .get_num_of_ranges = iwl_dump_ini_mem_ranges,
+ .get_size = iwl_dump_ini_mem_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_csr_iter,
+ },
+ [IWL_FW_INI_REGION_NOTIFICATION] = {},
+ [IWL_FW_INI_REGION_DHC] = {},
+ [IWL_FW_INI_REGION_LMAC_ERROR_TABLE] = {
+ .get_num_of_ranges = iwl_dump_ini_mem_ranges,
+ .get_size = iwl_dump_ini_mem_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_dev_mem_iter,
+ },
+ [IWL_FW_INI_REGION_UMAC_ERROR_TABLE] = {
+ .get_num_of_ranges = iwl_dump_ini_mem_ranges,
+ .get_size = iwl_dump_ini_mem_get_size,
+ .fill_mem_hdr = iwl_dump_ini_mem_fill_header,
+ .fill_range = iwl_dump_ini_dev_mem_iter,
+ },
+};
- if (!trigger || !trigger->num_regions)
- return 0;
+static u32 iwl_dump_ini_trigger(struct iwl_fw_runtime *fwrt,
+ struct iwl_fw_ini_trigger *trigger,
+ struct list_head *list)
+{
+ int i;
+ u32 size = 0;
for (i = 0; i < le32_to_cpu(trigger->num_regions); i++) {
- u32 reg_id = le32_to_cpu(trigger->data[i]);
+ u32 reg_id = le32_to_cpu(trigger->data[i]), reg_type;
struct iwl_fw_ini_region_cfg *reg;
if (WARN_ON(reg_id >= ARRAY_SIZE(fwrt->dump.active_regs)))
@@ -1802,7 +1877,7 @@ static int iwl_fw_ini_get_trigger_len(struct iwl_fw_runtime *fwrt,
reg = fwrt->dump.active_regs[reg_id];
if (!reg) {
IWL_WARN(fwrt,
- "WRT: unassigned region id %d, skipping\n",
+ "WRT: Unassigned region id %d, skipping\n",
reg_id);
continue;
}
@@ -1811,205 +1886,55 @@ static int iwl_fw_ini_get_trigger_len(struct iwl_fw_runtime *fwrt,
if (le32_to_cpu(reg->domain) != IWL_FW_INI_DBG_DOMAIN_ALWAYS_ON)
continue;
- switch (le32_to_cpu(reg->region_type)) {
- case IWL_FW_INI_REGION_DEVICE_MEMORY:
- case IWL_FW_INI_REGION_PERIPHERY_MAC:
- case IWL_FW_INI_REGION_PERIPHERY_PHY:
- case IWL_FW_INI_REGION_PERIPHERY_AUX:
- case IWL_FW_INI_REGION_CSR:
- case IWL_FW_INI_REGION_LMAC_ERROR_TABLE:
- case IWL_FW_INI_REGION_UMAC_ERROR_TABLE:
- size = iwl_dump_ini_mem_get_size(fwrt, reg);
- if (size)
- ret_size += hdr_len + size;
- break;
- case IWL_FW_INI_REGION_TXF:
- size = iwl_dump_ini_txf_get_size(fwrt, reg);
- if (size)
- ret_size += hdr_len + size;
- break;
- case IWL_FW_INI_REGION_RXF:
- size = iwl_dump_ini_rxf_get_size(fwrt, reg);
- if (size)
- ret_size += hdr_len + size;
- break;
- case IWL_FW_INI_REGION_PAGING:
- if (iwl_fw_dbg_is_paging_enabled(fwrt))
- size = iwl_dump_ini_paging_get_size(fwrt, reg);
- else
- size = iwl_dump_ini_paging_gen2_get_size(fwrt,
- reg);
- if (size)
- ret_size += hdr_len + size;
- break;
- case IWL_FW_INI_REGION_DRAM_BUFFER:
- if (!fwrt->trans->dbg.num_blocks)
- break;
- size = iwl_dump_ini_mon_dram_get_size(fwrt, reg);
- if (size)
- ret_size += hdr_len + size;
- break;
- case IWL_FW_INI_REGION_INTERNAL_BUFFER:
- size = iwl_dump_ini_mon_smem_get_size(fwrt, reg);
- if (size)
- ret_size += hdr_len + size;
- break;
- case IWL_FW_INI_REGION_DRAM_IMR:
- /* Undefined yet */
- default:
- break;
- }
+ reg_type = le32_to_cpu(reg->region_type);
+ if (reg_type >= ARRAY_SIZE(iwl_dump_ini_region_ops))
+ continue;
+
+ size += iwl_dump_ini_mem(fwrt, list, reg,
+ &iwl_dump_ini_region_ops[reg_type]);
}
- /* add dump info size */
- if (ret_size)
- ret_size += hdr_len + sizeof(struct iwl_fw_ini_dump_info) +
- (le32_to_cpu(trigger->num_regions) * sizeof(__le32));
+ if (size)
+ size += iwl_dump_ini_info(fwrt, trigger, list);
- return ret_size;
+ return size;
}
-static void iwl_fw_ini_dump_trigger(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_trigger *trigger,
- struct iwl_fw_error_dump_data **data)
+static u32 iwl_dump_ini_file_gen(struct iwl_fw_runtime *fwrt,
+ enum iwl_fw_ini_trigger_id trig_id,
+ struct list_head *list)
{
- int i, num = le32_to_cpu(trigger->num_regions);
-
- iwl_dump_ini_info(fwrt, trigger, data);
-
- for (i = 0; i < num; i++) {
- u32 reg_id = le32_to_cpu(trigger->data[i]);
- struct iwl_fw_ini_region_cfg *reg;
- struct iwl_dump_ini_mem_ops ops;
-
- if (reg_id >= ARRAY_SIZE(fwrt->dump.active_regs))
- continue;
-
- reg = fwrt->dump.active_regs[reg_id];
- /* Don't warn, get_trigger_len already warned */
- if (!reg)
- continue;
-
- /* currently the driver supports always on domain only */
- if (le32_to_cpu(reg->domain) != IWL_FW_INI_DBG_DOMAIN_ALWAYS_ON)
- continue;
-
- switch (le32_to_cpu(reg->region_type)) {
- case IWL_FW_INI_REGION_DEVICE_MEMORY:
- case IWL_FW_INI_REGION_LMAC_ERROR_TABLE:
- case IWL_FW_INI_REGION_UMAC_ERROR_TABLE:
- ops.get_num_of_ranges = iwl_dump_ini_mem_ranges;
- ops.get_size = iwl_dump_ini_mem_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mem_fill_header;
- ops.fill_range = iwl_dump_ini_dev_mem_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_PERIPHERY_MAC:
- case IWL_FW_INI_REGION_PERIPHERY_PHY:
- case IWL_FW_INI_REGION_PERIPHERY_AUX:
- ops.get_num_of_ranges = iwl_dump_ini_mem_ranges;
- ops.get_size = iwl_dump_ini_mem_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mem_fill_header;
- ops.fill_range = iwl_dump_ini_prph_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_DRAM_BUFFER:
- ops.get_num_of_ranges = iwl_dump_ini_mon_dram_ranges;
- ops.get_size = iwl_dump_ini_mon_dram_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mon_dram_fill_header;
- ops.fill_range = iwl_dump_ini_mon_dram_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_INTERNAL_BUFFER:
- ops.get_num_of_ranges = iwl_dump_ini_mem_ranges;
- ops.get_size = iwl_dump_ini_mon_smem_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mon_smem_fill_header;
- ops.fill_range = iwl_dump_ini_dev_mem_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_PAGING:
- ops.fill_mem_hdr = iwl_dump_ini_mem_fill_header;
- if (iwl_fw_dbg_is_paging_enabled(fwrt)) {
- ops.get_num_of_ranges =
- iwl_dump_ini_paging_ranges;
- ops.get_size = iwl_dump_ini_paging_get_size;
- ops.fill_range = iwl_dump_ini_paging_iter;
- } else {
- ops.get_num_of_ranges =
- iwl_dump_ini_paging_gen2_ranges;
- ops.get_size =
- iwl_dump_ini_paging_gen2_get_size;
- ops.fill_range = iwl_dump_ini_paging_gen2_iter;
- }
-
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_TXF: {
- struct iwl_ini_txf_iter_data iter = { .init = true };
- void *fifo_iter = fwrt->dump.fifo_iter;
-
- fwrt->dump.fifo_iter = &iter;
- ops.get_num_of_ranges = iwl_dump_ini_txf_ranges;
- ops.get_size = iwl_dump_ini_txf_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mem_fill_header;
- ops.fill_range = iwl_dump_ini_txf_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- fwrt->dump.fifo_iter = fifo_iter;
- break;
- }
- case IWL_FW_INI_REGION_RXF:
- ops.get_num_of_ranges = iwl_dump_ini_rxf_ranges;
- ops.get_size = iwl_dump_ini_rxf_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mem_fill_header;
- ops.fill_range = iwl_dump_ini_rxf_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_CSR:
- ops.get_num_of_ranges = iwl_dump_ini_mem_ranges;
- ops.get_size = iwl_dump_ini_mem_get_size;
- ops.fill_mem_hdr = iwl_dump_ini_mem_fill_header;
- ops.fill_range = iwl_dump_ini_csr_iter;
- iwl_dump_ini_mem(fwrt, data, reg, &ops);
- break;
- case IWL_FW_INI_REGION_DRAM_IMR:
- /* This is undefined yet */
- default:
- break;
- }
- }
-}
-
-static struct iwl_fw_error_dump_file *
-iwl_fw_error_ini_dump_file(struct iwl_fw_runtime *fwrt,
- enum iwl_fw_ini_trigger_id trig_id)
-{
- int size;
- struct iwl_fw_error_dump_data *dump_data;
- struct iwl_fw_error_dump_file *dump_file;
+ struct iwl_fw_ini_dump_entry *entry;
+ struct iwl_fw_ini_dump_file_hdr *hdr;
struct iwl_fw_ini_trigger *trigger;
+ u32 size;
if (!iwl_fw_ini_trigger_on(fwrt, trig_id))
- return NULL;
+ return 0;
trigger = fwrt->dump.active_trigs[trig_id].trig;
+ if (!trigger || !le32_to_cpu(trigger->num_regions))
+ return 0;
- size = iwl_fw_ini_get_trigger_len(fwrt, trigger);
- if (!size)
- return NULL;
+ entry = kmalloc(sizeof(*entry) + sizeof(*hdr), GFP_KERNEL);
+ if (!entry)
+ return 0;
- size += sizeof(*dump_file);
+ entry->size = sizeof(*hdr);
- dump_file = vzalloc(size);
- if (!dump_file)
- return NULL;
+ size = iwl_dump_ini_trigger(fwrt, trigger, list);
+ if (!size) {
+ kfree(entry);
+ return 0;
+ }
- dump_file->barker = cpu_to_le32(IWL_FW_INI_ERROR_DUMP_BARKER);
- dump_data = (void *)dump_file->data;
- dump_file->file_len = cpu_to_le32(size);
+ hdr = (void *)entry->data;
+ hdr->barker = cpu_to_le32(IWL_FW_INI_ERROR_DUMP_BARKER);
+ hdr->file_len = cpu_to_le32(size + entry->size);
- iwl_fw_ini_dump_trigger(fwrt, trigger, &dump_data);
+ list_add(&entry->list, list);
- return dump_file;
+ return le32_to_cpu(hdr->file_len);
}
static void iwl_fw_error_dump(struct iwl_fw_runtime *fwrt)
@@ -2058,27 +1983,44 @@ static void iwl_fw_error_dump(struct iwl_fw_runtime *fwrt)
iwl_fw_free_dump_desc(fwrt);
}
+static void iwl_dump_ini_list_free(struct list_head *list)
+{
+ while (!list_empty(list)) {
+ struct iwl_fw_ini_dump_entry *entry =
+ list_entry(list->next, typeof(*entry), list);
+
+ list_del(&entry->list);
+ kfree(entry);
+ }
+}
+
static void iwl_fw_error_ini_dump(struct iwl_fw_runtime *fwrt, u8 wk_idx)
{
enum iwl_fw_ini_trigger_id trig_id = fwrt->dump.wks[wk_idx].ini_trig_id;
- struct iwl_fw_error_dump_file *dump_file;
+ struct list_head dump_list = LIST_HEAD_INIT(dump_list);
struct scatterlist *sg_dump_data;
u32 file_len;
- dump_file = iwl_fw_error_ini_dump_file(fwrt, trig_id);
- if (!dump_file)
+ file_len = iwl_dump_ini_file_gen(fwrt, trig_id, &dump_list);
+ if (!file_len)
goto out;
- file_len = le32_to_cpu(dump_file->file_len);
-
sg_dump_data = alloc_sgtable(file_len);
if (sg_dump_data) {
- sg_pcopy_from_buffer(sg_dump_data, sg_nents(sg_dump_data),
- dump_file, file_len, 0);
+ struct iwl_fw_ini_dump_entry *entry;
+ int sg_entries = sg_nents(sg_dump_data);
+ u32 offs = 0;
+
+ list_for_each_entry(entry, &dump_list, list) {
+ sg_pcopy_from_buffer(sg_dump_data, sg_entries,
+ entry->data, entry->size, offs);
+ offs += entry->size;
+ }
dev_coredumpsg(fwrt->trans->dev, sg_dump_data, file_len,
GFP_KERNEL);
}
- vfree(dump_file);
+ iwl_dump_ini_list_free(&dump_list);
+
out:
fwrt->dump.wks[wk_idx].ini_trig_id = IWL_FW_TRIGGER_ID_INVALID;
}
@@ -2098,7 +2040,7 @@ int iwl_fw_dbg_collect_desc(struct iwl_fw_runtime *fwrt,
u32 trig_type = le32_to_cpu(desc->trig_desc.type);
int ret;
- if (fwrt->trans->dbg.ini_valid) {
+ if (iwl_trans_dbg_ini_valid(fwrt->trans)) {
ret = iwl_fw_dbg_ini_collect(fwrt, trig_type);
if (!ret)
iwl_fw_free_dump_desc(fwrt);
@@ -2220,7 +2162,7 @@ int _iwl_fw_dbg_ini_collect(struct iwl_fw_runtime *fwrt,
active->trig->occurrences = cpu_to_le32(--occur);
if (le32_to_cpu(active->trig->force_restart)) {
- IWL_WARN(fwrt, "WRT: force restart: trigger %d fired.\n", id);
+ IWL_WARN(fwrt, "WRT: Force restart: trigger %d fired.\n", id);
iwl_force_nmi(fwrt->trans);
return 0;
}
@@ -2240,7 +2182,7 @@ int _iwl_fw_dbg_ini_collect(struct iwl_fw_runtime *fwrt,
fwrt->dump.wks[idx].ini_trig_id = id;
- IWL_WARN(fwrt, "WRT: collecting data: ini trigger %d fired.\n", id);
+ IWL_WARN(fwrt, "WRT: Collecting data: ini trigger %d fired.\n", id);
schedule_delayed_work(&fwrt->dump.wks[idx].wk, usecs_to_jiffies(delay));
@@ -2372,16 +2314,19 @@ static void iwl_fw_dbg_collect_sync(struct iwl_fw_runtime *fwrt, u8 wk_idx)
goto out;
}
- iwl_fw_dbg_stop_recording(fwrt->trans, ¶ms);
+ if (iwl_fw_dbg_stop_restart_recording(fwrt, ¶ms, true)) {
+ IWL_ERR(fwrt, "Failed to stop DBGC recording, aborting dump\n");
+ goto out;
+ }
- IWL_DEBUG_FW_INFO(fwrt, "WRT: data collection start\n");
- if (fwrt->trans->dbg.ini_valid)
+ IWL_DEBUG_FW_INFO(fwrt, "WRT: Data collection start\n");
+ if (iwl_trans_dbg_ini_valid(fwrt->trans))
iwl_fw_error_ini_dump(fwrt, wk_idx);
else
iwl_fw_error_dump(fwrt);
- IWL_DEBUG_FW_INFO(fwrt, "WRT: data collection done\n");
+ IWL_DEBUG_FW_INFO(fwrt, "WRT: Data collection done\n");
- iwl_fw_dbg_restart_recording(fwrt, ¶ms);
+ iwl_fw_dbg_stop_restart_recording(fwrt, ¶ms, false);
out:
clear_bit(wk_idx, &fwrt->dump.active_wks);
@@ -2432,472 +2377,17 @@ void iwl_fw_dbg_read_d3_debug_data(struct iwl_fw_runtime *fwrt)
}
IWL_EXPORT_SYMBOL(iwl_fw_dbg_read_d3_debug_data);
-static void iwl_fw_dbg_info_apply(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_debug_info_tlv *dbg_info,
- bool ext, enum iwl_fw_ini_apply_point pnt)
-{
- u32 img_name_len = le32_to_cpu(dbg_info->img_name_len);
- u32 dbg_cfg_name_len = le32_to_cpu(dbg_info->dbg_cfg_name_len);
-
- if (img_name_len != IWL_FW_INI_MAX_IMG_NAME_LEN) {
- IWL_WARN(fwrt,
- "WRT: ext=%d. Invalid image name length %d, expected %d\n",
- ext, img_name_len,
- IWL_FW_INI_MAX_IMG_NAME_LEN);
- return;
- }
-
- if (dbg_cfg_name_len != IWL_FW_INI_MAX_DBG_CFG_NAME_LEN) {
- IWL_WARN(fwrt,
- "WRT: ext=%d. Invalid debug cfg name length %d, expected %d\n",
- ext, dbg_cfg_name_len,
- IWL_FW_INI_MAX_DBG_CFG_NAME_LEN);
- return;
- }
-
- if (ext) {
- memcpy(fwrt->dump.external_dbg_cfg_name, dbg_info->dbg_cfg_name,
- sizeof(fwrt->dump.external_dbg_cfg_name));
- } else {
- memcpy(fwrt->dump.img_name, dbg_info->img_name,
- sizeof(fwrt->dump.img_name));
- memcpy(fwrt->dump.internal_dbg_cfg_name, dbg_info->dbg_cfg_name,
- sizeof(fwrt->dump.internal_dbg_cfg_name));
- }
-}
-
-static void
-iwl_fw_dbg_buffer_allocation(struct iwl_fw_runtime *fwrt, u32 size)
-{
- struct iwl_trans *trans = fwrt->trans;
- void *virtual_addr = NULL;
- dma_addr_t phys_addr;
-
- if (WARN_ON_ONCE(trans->dbg.num_blocks ==
- ARRAY_SIZE(trans->dbg.fw_mon)))
- return;
-
- virtual_addr =
- dma_alloc_coherent(fwrt->trans->dev, size, &phys_addr,
- GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO |
- __GFP_COMP);
-
- /* TODO: alloc fragments if needed */
- if (!virtual_addr)
- IWL_ERR(fwrt, "Failed to allocate debug memory\n");
-
- IWL_DEBUG_FW(trans,
- "Allocated DRAM buffer[%d], size=0x%x\n",
- trans->dbg.num_blocks, size);
-
- trans->dbg.fw_mon[trans->dbg.num_blocks].block = virtual_addr;
- trans->dbg.fw_mon[trans->dbg.num_blocks].physical = phys_addr;
- trans->dbg.fw_mon[trans->dbg.num_blocks].size = size;
- trans->dbg.num_blocks++;
-}
-
-static void iwl_fw_dbg_buffer_apply(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_allocation_data *alloc,
- enum iwl_fw_ini_apply_point pnt)
-{
- struct iwl_trans *trans = fwrt->trans;
- struct iwl_ldbg_config_cmd ldbg_cmd = {
- .type = cpu_to_le32(BUFFER_ALLOCATION),
- };
- struct iwl_buffer_allocation_cmd *cmd = &ldbg_cmd.buffer_allocation;
- struct iwl_host_cmd hcmd = {
- .id = LDBG_CONFIG_CMD,
- .flags = CMD_ASYNC,
- .data[0] = &ldbg_cmd,
- .len[0] = sizeof(ldbg_cmd),
- };
- int block_idx = trans->dbg.num_blocks;
- u32 buf_location = le32_to_cpu(alloc->tlv.buffer_location);
-
- if (fwrt->trans->dbg.ini_dest == IWL_FW_INI_LOCATION_INVALID)
- fwrt->trans->dbg.ini_dest = buf_location;
-
- if (buf_location != fwrt->trans->dbg.ini_dest) {
- WARN(fwrt,
- "WRT: attempt to override buffer location on apply point %d\n",
- pnt);
-
- return;
- }
-
- if (buf_location == IWL_FW_INI_LOCATION_SRAM_PATH) {
- IWL_DEBUG_FW(trans, "WRT: applying SMEM buffer destination\n");
- /* set sram monitor by enabling bit 7 */
- iwl_set_bit(fwrt->trans, CSR_HW_IF_CONFIG_REG,
- CSR_HW_IF_CONFIG_REG_BIT_MONITOR_SRAM);
-
- return;
- }
-
- if (buf_location != IWL_FW_INI_LOCATION_DRAM_PATH)
- return;
-
- if (!alloc->is_alloc) {
- iwl_fw_dbg_buffer_allocation(fwrt,
- le32_to_cpu(alloc->tlv.size));
- if (block_idx == trans->dbg.num_blocks)
- return;
- alloc->is_alloc = 1;
- }
-
- /* First block is assigned via registers / context info */
- if (trans->dbg.num_blocks == 1)
- return;
-
- IWL_DEBUG_FW(trans,
- "WRT: applying DRAM buffer[%d] destination\n", block_idx);
-
- cmd->num_frags = cpu_to_le32(1);
- cmd->fragments[0].address =
- cpu_to_le64(trans->dbg.fw_mon[block_idx].physical);
- cmd->fragments[0].size = alloc->tlv.size;
- cmd->allocation_id = alloc->tlv.allocation_id;
- cmd->buffer_location = alloc->tlv.buffer_location;
-
- iwl_trans_send_cmd(trans, &hcmd);
-}
-
-static void iwl_fw_dbg_send_hcmd(struct iwl_fw_runtime *fwrt,
- struct iwl_ucode_tlv *tlv,
- bool ext)
-{
- struct iwl_fw_ini_hcmd_tlv *hcmd_tlv = (void *)&tlv->data[0];
- struct iwl_fw_ini_hcmd *data = &hcmd_tlv->hcmd;
- u16 len = le32_to_cpu(tlv->length) - sizeof(*hcmd_tlv);
-
- struct iwl_host_cmd hcmd = {
- .id = WIDE_ID(data->group, data->id),
- .len = { len, },
- .data = { data->data, },
- };
-
- /* currently the driver supports always on domain only */
- if (le32_to_cpu(hcmd_tlv->domain) != IWL_FW_INI_DBG_DOMAIN_ALWAYS_ON)
- return;
-
- IWL_DEBUG_FW(fwrt,
- "WRT: ext=%d. Sending host command id=0x%x, group=0x%x\n",
- ext, data->id, data->group);
-
- iwl_trans_send_cmd(fwrt->trans, &hcmd);
-}
-
-static void iwl_fw_dbg_update_regions(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_region_tlv *tlv,
- bool ext, enum iwl_fw_ini_apply_point pnt)
-{
- void *iter = (void *)tlv->region_config;
- int i, size = le32_to_cpu(tlv->num_regions);
- const char *err_st =
- "WRT: ext=%d. Invalid region %s %d for apply point %d\n";
-
- for (i = 0; i < size; i++) {
- struct iwl_fw_ini_region_cfg *reg = iter, **active;
- int id = le32_to_cpu(reg->region_id);
- u32 type = le32_to_cpu(reg->region_type);
-
- if (WARN(id >= ARRAY_SIZE(fwrt->dump.active_regs), err_st, ext,
- "id", id, pnt))
- break;
-
- if (WARN(type == 0 || type >= IWL_FW_INI_REGION_NUM, err_st,
- ext, "type", type, pnt))
- break;
-
- active = &fwrt->dump.active_regs[id];
-
- if (*active)
- IWL_WARN(fwrt->trans,
- "WRT: ext=%d. Region id %d override\n",
- ext, id);
-
- IWL_DEBUG_FW(fwrt,
- "WRT: ext=%d. Activating region id %d\n",
- ext, id);
-
- *active = reg;
-
- if (type == IWL_FW_INI_REGION_TXF ||
- type == IWL_FW_INI_REGION_RXF)
- iter += le32_to_cpu(reg->fifos.num_of_registers) *
- sizeof(__le32);
- else if (type == IWL_FW_INI_REGION_DEVICE_MEMORY ||
- type == IWL_FW_INI_REGION_PERIPHERY_MAC ||
- type == IWL_FW_INI_REGION_PERIPHERY_PHY ||
- type == IWL_FW_INI_REGION_PERIPHERY_AUX ||
- type == IWL_FW_INI_REGION_INTERNAL_BUFFER ||
- type == IWL_FW_INI_REGION_PAGING ||
- type == IWL_FW_INI_REGION_CSR ||
- type == IWL_FW_INI_REGION_LMAC_ERROR_TABLE ||
- type == IWL_FW_INI_REGION_UMAC_ERROR_TABLE)
- iter += le32_to_cpu(reg->internal.num_of_ranges) *
- sizeof(__le32);
-
- iter += sizeof(*reg);
- }
-}
-
-static int iwl_fw_dbg_trig_realloc(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_active_triggers *active,
- u32 id, int size)
-{
- void *ptr;
-
- if (size <= active->size)
- return 0;
-
- ptr = krealloc(active->trig, size, GFP_KERNEL);
- if (!ptr) {
- IWL_ERR(fwrt, "WRT: Failed to allocate memory for trigger %d\n",
- id);
- return -ENOMEM;
- }
- active->trig = ptr;
- active->size = size;
-
- return 0;
-}
-
-static void iwl_fw_dbg_update_triggers(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_ini_trigger_tlv *tlv,
- bool ext,
- enum iwl_fw_ini_apply_point apply_point)
-{
- int i, size = le32_to_cpu(tlv->num_triggers);
- void *iter = (void *)tlv->trigger_config;
-
- for (i = 0; i < size; i++) {
- struct iwl_fw_ini_trigger *trig = iter;
- struct iwl_fw_ini_active_triggers *active;
- int id = le32_to_cpu(trig->trigger_id);
- u32 trig_regs_size = le32_to_cpu(trig->num_regions) *
- sizeof(__le32);
-
- if (WARN(id >= ARRAY_SIZE(fwrt->dump.active_trigs),
- "WRT: ext=%d. Invalid trigger id %d for apply point %d\n",
- ext, id, apply_point))
- break;
-
- active = &fwrt->dump.active_trigs[id];
-
- if (!active->active) {
- size_t trig_size = sizeof(*trig) + trig_regs_size;
-
- IWL_DEBUG_FW(fwrt,
- "WRT: ext=%d. Activating trigger %d\n",
- ext, id);
-
- if (iwl_fw_dbg_trig_realloc(fwrt, active, id,
- trig_size))
- goto next;
-
- memcpy(active->trig, trig, trig_size);
-
- } else {
- u32 conf_override =
- !(le32_to_cpu(trig->override_trig) & 0xff);
- u32 region_override =
- !(le32_to_cpu(trig->override_trig) & 0xff00);
- u32 offset = 0;
- u32 active_regs =
- le32_to_cpu(active->trig->num_regions);
- u32 new_regs = le32_to_cpu(trig->num_regions);
- int mem_to_add = trig_regs_size;
-
- if (region_override) {
- IWL_DEBUG_FW(fwrt,
- "WRT: ext=%d. Trigger %d regions override\n",
- ext, id);
-
- mem_to_add -= active_regs * sizeof(__le32);
- } else {
- IWL_DEBUG_FW(fwrt,
- "WRT: ext=%d. Trigger %d regions appending\n",
- ext, id);
-
- offset += active_regs;
- new_regs += active_regs;
- }
-
- if (iwl_fw_dbg_trig_realloc(fwrt, active, id,
- active->size + mem_to_add))
- goto next;
-
- if (conf_override) {
- IWL_DEBUG_FW(fwrt,
- "WRT: ext=%d. Trigger %d configuration override\n",
- ext, id);
-
- memcpy(active->trig, trig, sizeof(*trig));
- }
-
- memcpy(active->trig->data + offset, trig->data,
- trig_regs_size);
- active->trig->num_regions = cpu_to_le32(new_regs);
- }
-
- /* Since zero means infinity - just set to -1 */
- if (!le32_to_cpu(active->trig->occurrences))
- active->trig->occurrences = cpu_to_le32(-1);
-
- active->active = true;
-
- if (id == IWL_FW_TRIGGER_ID_PERIODIC_TRIGGER) {
- u32 collect_interval = le32_to_cpu(trig->trigger_data);
-
- /* the minimum allowed interval is 50ms */
- if (collect_interval < 50) {
- collect_interval = 50;
- trig->trigger_data =
- cpu_to_le32(collect_interval);
- }
-
- mod_timer(&fwrt->dump.periodic_trig,
- jiffies + msecs_to_jiffies(collect_interval));
- }
-next:
- iter += sizeof(*trig) + trig_regs_size;
-
- }
-}
-
-static void _iwl_fw_dbg_apply_point(struct iwl_fw_runtime *fwrt,
- struct iwl_apply_point_data *data,
- enum iwl_fw_ini_apply_point pnt,
- bool ext)
-{
- void *iter = data->data;
-
- while (iter && iter < data->data + data->size) {
- struct iwl_ucode_tlv *tlv = iter;
- void *ini_tlv = (void *)tlv->data;
- u32 type = le32_to_cpu(tlv->type);
-
- switch (type) {
- case IWL_UCODE_TLV_TYPE_DEBUG_INFO:
- iwl_fw_dbg_info_apply(fwrt, ini_tlv, ext, pnt);
- break;
- case IWL_UCODE_TLV_TYPE_BUFFER_ALLOCATION: {
- struct iwl_fw_ini_allocation_data *buf_alloc = ini_tlv;
-
- if (pnt != IWL_FW_INI_APPLY_EARLY) {
- IWL_ERR(fwrt,
- "WRT: ext=%d. Invalid apply point %d for buffer allocation\n",
- ext, pnt);
- goto next;
- }
-
- iwl_fw_dbg_buffer_apply(fwrt, ini_tlv, pnt);
- iter += sizeof(buf_alloc->is_alloc);
- break;
- }
- case IWL_UCODE_TLV_TYPE_HCMD:
- if (pnt < IWL_FW_INI_APPLY_AFTER_ALIVE) {
- IWL_ERR(fwrt,
- "WRT: ext=%d. Invalid apply point %d for host command\n",
- ext, pnt);
- goto next;
- }
- iwl_fw_dbg_send_hcmd(fwrt, tlv, ext);
- break;
- case IWL_UCODE_TLV_TYPE_REGIONS:
- iwl_fw_dbg_update_regions(fwrt, ini_tlv, ext, pnt);
- break;
- case IWL_UCODE_TLV_TYPE_TRIGGERS:
- iwl_fw_dbg_update_triggers(fwrt, ini_tlv, ext, pnt);
- break;
- case IWL_UCODE_TLV_TYPE_DEBUG_FLOW:
- break;
- default:
- WARN_ONCE(1,
- "WRT: ext=%d. Invalid TLV 0x%x for apply point\n",
- ext, type);
- break;
- }
-next:
- iter += sizeof(*tlv) + le32_to_cpu(tlv->length);
- }
-}
-
-static void iwl_fw_dbg_ini_reset_cfg(struct iwl_fw_runtime *fwrt)
+void iwl_fw_dbg_stop_sync(struct iwl_fw_runtime *fwrt)
{
int i;
- for (i = 0; i < IWL_FW_INI_MAX_REGION_ID; i++)
- fwrt->dump.active_regs[i] = NULL;
-
- /* disable the triggers, used in recovery flow */
- for (i = 0; i < IWL_FW_TRIGGER_ID_NUM; i++)
- fwrt->dump.active_trigs[i].active = false;
-
- memset(fwrt->dump.img_name, 0,
- sizeof(fwrt->dump.img_name));
- memset(fwrt->dump.internal_dbg_cfg_name, 0,
- sizeof(fwrt->dump.internal_dbg_cfg_name));
- memset(fwrt->dump.external_dbg_cfg_name, 0,
- sizeof(fwrt->dump.external_dbg_cfg_name));
-
- fwrt->trans->dbg.ini_dest = IWL_FW_INI_LOCATION_INVALID;
-}
-
-void iwl_fw_dbg_apply_point(struct iwl_fw_runtime *fwrt,
- enum iwl_fw_ini_apply_point apply_point)
-{
- void *data = &fwrt->trans->dbg.apply_points[apply_point];
-
- IWL_DEBUG_FW(fwrt, "WRT: enabling apply point %d\n", apply_point);
-
- if (apply_point == IWL_FW_INI_APPLY_EARLY)
- iwl_fw_dbg_ini_reset_cfg(fwrt);
-
- _iwl_fw_dbg_apply_point(fwrt, data, apply_point, false);
-
- data = &fwrt->trans->dbg.apply_points_ext[apply_point];
- _iwl_fw_dbg_apply_point(fwrt, data, apply_point, true);
-}
-IWL_EXPORT_SYMBOL(iwl_fw_dbg_apply_point);
-
-void iwl_fwrt_stop_device(struct iwl_fw_runtime *fwrt)
-{
- int i;
-
- del_timer(&fwrt->dump.periodic_trig);
+ iwl_dbg_tlv_del_timers(fwrt->trans);
for (i = 0; i < IWL_FW_RUNTIME_DUMP_WK_NUM; i++)
iwl_fw_dbg_collect_sync(fwrt, i);
- iwl_trans_stop_device(fwrt->trans);
+ iwl_fw_dbg_stop_restart_recording(fwrt, NULL, true);
}
-IWL_EXPORT_SYMBOL(iwl_fwrt_stop_device);
-
-void iwl_fw_dbg_periodic_trig_handler(struct timer_list *t)
-{
- struct iwl_fw_runtime *fwrt;
- enum iwl_fw_ini_trigger_id id = IWL_FW_TRIGGER_ID_PERIODIC_TRIGGER;
- int ret;
- typeof(fwrt->dump) *dump_ptr = container_of(t, typeof(fwrt->dump),
- periodic_trig);
-
- fwrt = container_of(dump_ptr, typeof(*fwrt), dump);
-
- ret = _iwl_fw_dbg_ini_collect(fwrt, id);
- if (!ret || ret == -EBUSY) {
- struct iwl_fw_ini_trigger *trig =
- fwrt->dump.active_trigs[id].trig;
- u32 occur = le32_to_cpu(trig->occurrences);
- u32 collect_interval = le32_to_cpu(trig->trigger_data);
-
- if (!occur)
- return;
-
- mod_timer(&fwrt->dump.periodic_trig,
- jiffies + msecs_to_jiffies(collect_interval));
- }
-}
+IWL_EXPORT_SYMBOL(iwl_fw_dbg_stop_sync);
#define FSEQ_REG(x) { .addr = (x), .str = #x, }
@@ -2937,3 +2427,92 @@ void iwl_fw_error_print_fseq_regs(struct iwl_fw_runtime *fwrt)
iwl_trans_release_nic_access(trans, &flags);
}
IWL_EXPORT_SYMBOL(iwl_fw_error_print_fseq_regs);
+
+static int iwl_fw_dbg_suspend_resume_hcmd(struct iwl_trans *trans, bool suspend)
+{
+ struct iwl_dbg_suspend_resume_cmd cmd = {
+ .operation = suspend ?
+ cpu_to_le32(DBGC_SUSPEND_CMD) :
+ cpu_to_le32(DBGC_RESUME_CMD),
+ };
+ struct iwl_host_cmd hcmd = {
+ .id = WIDE_ID(DEBUG_GROUP, DBGC_SUSPEND_RESUME),
+ .data[0] = &cmd,
+ .len[0] = sizeof(cmd),
+ };
+
+ return iwl_trans_send_cmd(trans, &hcmd);
+}
+
+static void iwl_fw_dbg_stop_recording(struct iwl_trans *trans,
+ struct iwl_fw_dbg_params *params)
+{
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_7000) {
+ iwl_set_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x100);
+ return;
+ }
+
+ if (params) {
+ params->in_sample = iwl_read_umac_prph(trans, DBGC_IN_SAMPLE);
+ params->out_ctrl = iwl_read_umac_prph(trans, DBGC_OUT_CTRL);
+ }
+
+ iwl_write_umac_prph(trans, DBGC_IN_SAMPLE, 0);
+ /* wait for the DBGC to finish writing the internal buffer to DRAM to
+ * avoid halting the HW while writing
+ */
+ usleep_range(700, 1000);
+ iwl_write_umac_prph(trans, DBGC_OUT_CTRL, 0);
+}
+
+static int iwl_fw_dbg_restart_recording(struct iwl_trans *trans,
+ struct iwl_fw_dbg_params *params)
+{
+ if (!params)
+ return -EIO;
+
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_7000) {
+ iwl_clear_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x100);
+ iwl_clear_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x1);
+ iwl_set_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x1);
+ } else {
+ iwl_write_umac_prph(trans, DBGC_IN_SAMPLE, params->in_sample);
+ iwl_write_umac_prph(trans, DBGC_OUT_CTRL, params->out_ctrl);
+ }
+
+ return 0;
+}
+
+int iwl_fw_dbg_stop_restart_recording(struct iwl_fw_runtime *fwrt,
+ struct iwl_fw_dbg_params *params,
+ bool stop)
+{
+ int ret = 0;
+
+ /* if the FW crashed or not debug monitor cfg was given, there is
+ * no point in changing the recording state
+ */
+ if (test_bit(STATUS_FW_ERROR, &fwrt->trans->status) ||
+ (!fwrt->trans->dbg.dest_tlv &&
+ fwrt->trans->dbg.ini_dest == IWL_FW_INI_LOCATION_INVALID))
+ return 0;
+
+ if (fw_has_capa(&fwrt->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_DBG_SUSPEND_RESUME_CMD_SUPP))
+ ret = iwl_fw_dbg_suspend_resume_hcmd(fwrt->trans, stop);
+ else if (stop)
+ iwl_fw_dbg_stop_recording(fwrt->trans, params);
+ else
+ ret = iwl_fw_dbg_restart_recording(fwrt->trans, params);
+#ifdef CONFIG_IWLWIFI_DEBUGFS
+ if (!ret) {
+ if (stop)
+ fwrt->trans->dbg.rec_on = false;
+ else
+ iwl_fw_set_dbg_rec_on(fwrt);
+ }
+#endif
+
+ return ret;
+}
+IWL_EXPORT_SYMBOL(iwl_fw_dbg_stop_restart_recording);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dbg.h b/drivers/net/wireless/intel/iwlwifi/fw/dbg.h
index a8459ac..e3b5dd3 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dbg.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dbg.h
@@ -202,7 +202,7 @@ _iwl_fw_dbg_trigger_on(struct iwl_fw_runtime *fwrt,
{
struct iwl_fw_dbg_trigger_tlv *trig;
- if (fwrt->trans->dbg.ini_valid)
+ if (iwl_trans_dbg_ini_valid(fwrt->trans))
return NULL;
if (!iwl_fw_dbg_trigger_enabled(fwrt->fw, id))
@@ -229,8 +229,9 @@ iwl_fw_ini_trigger_on(struct iwl_fw_runtime *fwrt,
struct iwl_fw_ini_trigger *trig;
u32 usec;
- if (!fwrt->trans->dbg.ini_valid || id == IWL_FW_TRIGGER_ID_INVALID ||
- id >= IWL_FW_TRIGGER_ID_NUM || !fwrt->dump.active_trigs[id].active)
+ if (!iwl_trans_dbg_ini_valid(fwrt->trans) ||
+ id == IWL_FW_TRIGGER_ID_INVALID || id >= IWL_FW_TRIGGER_ID_NUM ||
+ !fwrt->dump.active_trigs[id].active)
return false;
trig = fwrt->dump.active_trigs[id].trig;
@@ -262,69 +263,9 @@ _iwl_fw_dbg_trigger_simple_stop(struct iwl_fw_runtime *fwrt,
_iwl_fw_dbg_trigger_simple_stop((fwrt), (wdev), \
iwl_fw_dbg_get_trigger((fwrt)->fw,\
(trig)))
-
-static inline void
-_iwl_fw_dbg_stop_recording(struct iwl_trans *trans,
- struct iwl_fw_dbg_params *params)
-{
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_7000) {
- iwl_set_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x100);
- return;
- }
-
- if (params) {
- params->in_sample = iwl_read_umac_prph(trans, DBGC_IN_SAMPLE);
- params->out_ctrl = iwl_read_umac_prph(trans, DBGC_OUT_CTRL);
- }
-
- iwl_write_umac_prph(trans, DBGC_IN_SAMPLE, 0);
- /* wait for the DBGC to finish writing the internal buffer to DRAM to
- * avoid halting the HW while writing
- */
- usleep_range(700, 1000);
- iwl_write_umac_prph(trans, DBGC_OUT_CTRL, 0);
-#ifdef CONFIG_IWLWIFI_DEBUGFS
- trans->dbg.rec_on = false;
-#endif
-}
-
-static inline void
-iwl_fw_dbg_stop_recording(struct iwl_trans *trans,
- struct iwl_fw_dbg_params *params)
-{
- /* if the FW crashed or not debug monitor cfg was given, there is
- * no point in stopping
- */
- if (test_bit(STATUS_FW_ERROR, &trans->status) ||
- (!trans->dbg.dest_tlv &&
- trans->dbg.ini_dest == IWL_FW_INI_LOCATION_INVALID))
- return;
-
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
- IWL_ERR(trans,
- "WRT: unsupported device family %d for debug stop recording\n",
- trans->cfg->device_family);
- return;
- }
- _iwl_fw_dbg_stop_recording(trans, params);
-}
-
-static inline void
-_iwl_fw_dbg_restart_recording(struct iwl_trans *trans,
- struct iwl_fw_dbg_params *params)
-{
- if (WARN_ON(!params))
- return;
-
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_7000) {
- iwl_clear_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x100);
- iwl_clear_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x1);
- iwl_set_bits_prph(trans, MON_BUFF_SAMPLE_CTL, 0x1);
- } else {
- iwl_write_umac_prph(trans, DBGC_IN_SAMPLE, params->in_sample);
- iwl_write_umac_prph(trans, DBGC_OUT_CTRL, params->out_ctrl);
- }
-}
+int iwl_fw_dbg_stop_restart_recording(struct iwl_fw_runtime *fwrt,
+ struct iwl_fw_dbg_params *params,
+ bool stop);
#ifdef CONFIG_IWLWIFI_DEBUGFS
static inline void iwl_fw_set_dbg_rec_on(struct iwl_fw_runtime *fwrt)
@@ -336,30 +277,6 @@ static inline void iwl_fw_set_dbg_rec_on(struct iwl_fw_runtime *fwrt)
}
#endif
-static inline void
-iwl_fw_dbg_restart_recording(struct iwl_fw_runtime *fwrt,
- struct iwl_fw_dbg_params *params)
-{
- /* if the FW crashed or not debug monitor cfg was given, there is
- * no point in restarting
- */
- if (test_bit(STATUS_FW_ERROR, &fwrt->trans->status) ||
- (!fwrt->trans->dbg.dest_tlv &&
- fwrt->trans->dbg.ini_dest == IWL_FW_INI_LOCATION_INVALID))
- return;
-
- if (fwrt->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
- IWL_ERR(fwrt,
- "WRT: unsupported device family %d for debug restart recording\n",
- fwrt->trans->cfg->device_family);
- return;
- }
- _iwl_fw_dbg_restart_recording(fwrt->trans, params);
-#ifdef CONFIG_IWLWIFI_DEBUGFS
- iwl_fw_set_dbg_rec_on(fwrt);
-#endif
-}
-
static inline void iwl_fw_dump_conf_clear(struct iwl_fw_runtime *fwrt)
{
fwrt->dump.conf = FW_DBG_INVALID;
@@ -385,7 +302,7 @@ static inline bool iwl_fw_dbg_is_d3_debug_enabled(struct iwl_fw_runtime *fwrt)
static inline bool iwl_fw_dbg_is_paging_enabled(struct iwl_fw_runtime *fwrt)
{
return iwl_fw_dbg_type_on(fwrt, IWL_FW_ERROR_DUMP_PAGING) &&
- !fwrt->trans->cfg->gen2 &&
+ !fwrt->trans->trans_cfg->gen2 &&
fwrt->cur_fw_img < IWL_UCODE_TYPE_MAX &&
fwrt->fw->img[fwrt->cur_fw_img].paging_mem_size &&
fwrt->fw_paging_db[0].fw_paging_block;
@@ -397,24 +314,13 @@ static inline void iwl_fw_flush_dumps(struct iwl_fw_runtime *fwrt)
{
int i;
- del_timer(&fwrt->dump.periodic_trig);
+ iwl_dbg_tlv_del_timers(fwrt->trans);
for (i = 0; i < IWL_FW_RUNTIME_DUMP_WK_NUM; i++) {
flush_delayed_work(&fwrt->dump.wks[i].wk);
fwrt->dump.wks[i].ini_trig_id = IWL_FW_TRIGGER_ID_INVALID;
}
}
-static inline void iwl_fw_cancel_dumps(struct iwl_fw_runtime *fwrt)
-{
- int i;
-
- del_timer(&fwrt->dump.periodic_trig);
- for (i = 0; i < IWL_FW_RUNTIME_DUMP_WK_NUM; i++) {
- cancel_delayed_work_sync(&fwrt->dump.wks[i].wk);
- fwrt->dump.wks[i].ini_trig_id = IWL_FW_TRIGGER_ID_INVALID;
- }
-}
-
#ifdef CONFIG_IWLWIFI_DEBUGFS
static inline void iwl_fw_cancel_timestamp(struct iwl_fw_runtime *fwrt)
{
@@ -451,10 +357,7 @@ static inline void iwl_fw_resume_timestamp(struct iwl_fw_runtime *fwrt) {}
#endif /* CONFIG_IWLWIFI_DEBUGFS */
-void iwl_fw_dbg_apply_point(struct iwl_fw_runtime *fwrt,
- enum iwl_fw_ini_apply_point apply_point);
-
-void iwl_fwrt_stop_device(struct iwl_fw_runtime *fwrt);
+void iwl_fw_dbg_stop_sync(struct iwl_fw_runtime *fwrt);
static inline void iwl_fw_lmac1_set_alive_err_table(struct iwl_trans *trans,
u32 lmac_error_event_table)
@@ -478,7 +381,7 @@ static inline void iwl_fw_umac_set_alive_err_table(struct iwl_trans *trans,
static inline void iwl_fw_error_collect(struct iwl_fw_runtime *fwrt)
{
- if (fwrt->trans->dbg.ini_valid && fwrt->trans->dbg.hw_error) {
+ if (iwl_trans_dbg_ini_valid(fwrt->trans) && fwrt->trans->dbg.hw_error) {
_iwl_fw_dbg_ini_collect(fwrt, IWL_FW_TRIGGER_ID_FW_HW_ERROR);
fwrt->trans->dbg.hw_error = false;
} else {
@@ -486,8 +389,6 @@ static inline void iwl_fw_error_collect(struct iwl_fw_runtime *fwrt)
}
}
-void iwl_fw_dbg_periodic_trig_handler(struct timer_list *t);
-
void iwl_fw_error_print_fseq_regs(struct iwl_fw_runtime *fwrt);
static inline void iwl_fwrt_update_fw_versions(struct iwl_fw_runtime *fwrt,
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h b/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h
index 00a45ea..2e76367 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/error-dump.h
@@ -288,6 +288,28 @@ struct iwl_fw_error_dump_mem {
#define IWL_INI_DUMP_INFO_TYPE BIT(31)
/**
+ * struct iwl_fw_ini_dump_entry
+ * @list: list of dump entries
+ * @size: size of the data
+ * @data: entry data
+ */
+struct iwl_fw_ini_dump_entry {
+ struct list_head list;
+ u32 size;
+ u8 data[];
+} __packed;
+
+/**
+ * struct iwl_fw_error_dump_file - header of dump file
+ * @barker: must be %IWL_FW_INI_ERROR_DUMP_BARKER
+ * @file_len: the length of all the file including the header
+ */
+struct iwl_fw_ini_dump_file_hdr {
+ __le32 barker;
+ __le32 file_len;
+} __packed;
+
+/**
* struct iwl_fw_ini_fifo_hdr - fifo range header
* @fifo_num: the fifo number. In case of umac rx fifo, set BIT(31) to
* distinguish between lmac and umac rx fifos
@@ -301,10 +323,10 @@ struct iwl_fw_ini_fifo_hdr {
/**
* struct iwl_fw_ini_error_dump_range - range of memory
* @range_data_size: the size of this range, in bytes
- * @internal_base_addr - base address of internal memory range
- * @dram_base_addr - base address of dram monitor range
- * @page_num - page number of memory range
- * @fifo_hdr - fifo header of memory range
+ * @internal_base_addr: base address of internal memory range
+ * @dram_base_addr: base address of dram monitor range
+ * @page_num: page number of memory range
+ * @fifo_hdr: fifo header of memory range
* @data: the actual memory
*/
struct iwl_fw_ini_error_dump_range {
@@ -432,10 +454,10 @@ struct iwl_fw_error_dump_rb {
/**
* struct iwl_fw_ini_monitor_dump - ini monitor dump
- * @header - header of the region
- * @write_ptr - write pointer position in the buffer
- * @cycle_cnt - cycles count
- * @ranges - the memory ranges of this this region
+ * @header: header of the region
+ * @write_ptr: write pointer position in the buffer
+ * @cycle_cnt: cycles count
+ * @ranges: the memory ranges of this this region
*/
struct iwl_fw_ini_monitor_dump {
struct iwl_fw_ini_error_dump_header header;
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/file.h b/drivers/net/wireless/intel/iwlwifi/fw/file.h
index 0c38e73..423cc0c 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/file.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/file.h
@@ -157,8 +157,7 @@ enum iwl_ucode_tlv_type {
IWL_UCODE_TLV_TYPE_HCMD = IWL_UCODE_TLV_DEBUG_BASE + 2,
IWL_UCODE_TLV_TYPE_REGIONS = IWL_UCODE_TLV_DEBUG_BASE + 3,
IWL_UCODE_TLV_TYPE_TRIGGERS = IWL_UCODE_TLV_DEBUG_BASE + 4,
- IWL_UCODE_TLV_TYPE_DEBUG_FLOW = IWL_UCODE_TLV_DEBUG_BASE + 5,
- IWL_UCODE_TLV_DEBUG_MAX = IWL_UCODE_TLV_TYPE_DEBUG_FLOW,
+ IWL_UCODE_TLV_DEBUG_MAX = IWL_UCODE_TLV_TYPE_TRIGGERS,
/* TLVs 0x1000-0x2000 are for internal driver usage */
IWL_UCODE_TLV_FW_DBG_DUMP_LST = 0x1000,
@@ -323,6 +322,7 @@ enum iwl_ucode_tlv_api {
IWL_UCODE_TLV_API_FTM_RTT_ACCURACY = (__force iwl_ucode_tlv_api_t)54,
IWL_UCODE_TLV_API_SAR_TABLE_VER = (__force iwl_ucode_tlv_api_t)55,
IWL_UCODE_TLV_API_ADWELL_HB_DEF_N_AP = (__force iwl_ucode_tlv_api_t)57,
+ IWL_UCODE_TLV_API_SCAN_EXT_CHAN_VER = (__force iwl_ucode_tlv_api_t)58,
NUM_IWL_UCODE_TLV_API
#ifdef __CHECKER__
@@ -441,9 +441,11 @@ enum iwl_ucode_tlv_capa {
IWL_UCODE_TLV_CAPA_DYNAMIC_QUOTA = (__force iwl_ucode_tlv_capa_t)44,
IWL_UCODE_TLV_CAPA_COEX_SCHEMA_2 = (__force iwl_ucode_tlv_capa_t)45,
IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD = (__force iwl_ucode_tlv_capa_t)46,
- IWL_UCODE_TLV_CAPA_ULTRA_HB_CHANNELS = (__force iwl_ucode_tlv_capa_t)48,
IWL_UCODE_TLV_CAPA_FTM_CALIBRATED = (__force iwl_ucode_tlv_capa_t)47,
+ IWL_UCODE_TLV_CAPA_ULTRA_HB_CHANNELS = (__force iwl_ucode_tlv_capa_t)48,
IWL_UCODE_TLV_CAPA_CS_MODIFY = (__force iwl_ucode_tlv_capa_t)49,
+ IWL_UCODE_TLV_CAPA_SET_LTR_GEN2 = (__force iwl_ucode_tlv_capa_t)50,
+ IWL_UCODE_TLV_CAPA_SET_PPAG = (__force iwl_ucode_tlv_capa_t)52,
/* set 2 */
IWL_UCODE_TLV_CAPA_EXTENDED_DTS_MEASURE = (__force iwl_ucode_tlv_capa_t)64,
@@ -465,6 +467,8 @@ enum iwl_ucode_tlv_capa {
IWL_UCODE_TLV_CAPA_LED_CMD_SUPPORT = (__force iwl_ucode_tlv_capa_t)88,
IWL_UCODE_TLV_CAPA_MCC_UPDATE_11AX_SUPPORT = (__force iwl_ucode_tlv_capa_t)89,
IWL_UCODE_TLV_CAPA_CSI_REPORTING = (__force iwl_ucode_tlv_capa_t)90,
+ IWL_UCODE_TLV_CAPA_DBG_SUSPEND_RESUME_CMD_SUPP = (__force iwl_ucode_tlv_capa_t)92,
+ IWL_UCODE_TLV_CAPA_DBG_BUF_ALLOC_CMD_SUPP = (__force iwl_ucode_tlv_capa_t)93,
/* set 3 */
IWL_UCODE_TLV_CAPA_MLME_OFFLOAD = (__force iwl_ucode_tlv_capa_t)96,
@@ -521,6 +525,10 @@ enum iwl_fw_phy_cfg {
FW_PHY_CFG_TX_CHAIN = 0xf << FW_PHY_CFG_TX_CHAIN_POS,
FW_PHY_CFG_RX_CHAIN_POS = 20,
FW_PHY_CFG_RX_CHAIN = 0xf << FW_PHY_CFG_RX_CHAIN_POS,
+ FW_PHY_CFG_CHAIN_SAD_POS = 23,
+ FW_PHY_CFG_CHAIN_SAD_ENABLED = 0x1 << FW_PHY_CFG_CHAIN_SAD_POS,
+ FW_PHY_CFG_CHAIN_SAD_ANT_A = 0x2 << FW_PHY_CFG_CHAIN_SAD_POS,
+ FW_PHY_CFG_CHAIN_SAD_ANT_B = 0x4 << FW_PHY_CFG_CHAIN_SAD_POS,
FW_PHY_CFG_SHARED_CLK = BIT(31),
};
@@ -965,4 +973,19 @@ struct iwl_fw_cmd_version {
u8 notif_ver;
} __packed;
+static inline size_t _iwl_tlv_array_len(const struct iwl_ucode_tlv *tlv,
+ size_t fixed_size, size_t var_size)
+{
+ size_t var_len = le32_to_cpu(tlv->length) - fixed_size;
+
+ if (WARN_ON(var_len % var_size))
+ return 0;
+
+ return var_len / var_size;
+}
+
+#define iwl_tlv_array_len(_tlv_ptr, _struct_ptr, _memb) \
+ _iwl_tlv_array_len((_tlv_ptr), sizeof(*(_struct_ptr)), \
+ sizeof(_struct_ptr->_memb[0]))
+
#endif /* __iwl_fw_file_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/img.h b/drivers/net/wireless/intel/iwlwifi/fw/img.h
index 18ca5f1..039576d 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/img.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/img.h
@@ -228,15 +228,6 @@ struct iwl_fw_dbg {
};
/**
- * @tlv: the buffer allocation tlv
- * @is_alloc: indicates if the buffer was already allocated
- */
-struct iwl_fw_ini_allocation_data {
- struct iwl_fw_ini_allocation_tlv tlv;
- u32 is_alloc;
-} __packed;
-
-/**
* struct iwl_fw_ini_active_triggers
* @active: is this trigger active
* @size: allocated memory size of the trigger
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/init.c b/drivers/net/wireless/intel/iwlwifi/fw/init.c
index c16d6e1..ba00d16 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/init.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/init.c
@@ -81,8 +81,6 @@ void iwl_fw_runtime_init(struct iwl_fw_runtime *fwrt, struct iwl_trans *trans,
INIT_DELAYED_WORK(&fwrt->dump.wks[i].wk, iwl_fw_error_dump_wk);
}
iwl_fwrt_dbgfs_register(fwrt, dbgfs_dir);
- timer_setup(&fwrt->dump.periodic_trig,
- iwl_fw_dbg_periodic_trig_handler, 0);
}
IWL_EXPORT_SYMBOL(iwl_fw_runtime_init);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/paging.c b/drivers/net/wireless/intel/iwlwifi/fw/paging.c
index 9b8dd7f..2bd76bd 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/paging.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/paging.c
@@ -8,7 +8,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -31,7 +31,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -322,7 +322,7 @@ int iwl_init_paging(struct iwl_fw_runtime *fwrt, enum iwl_ucode_type type)
const struct fw_img *fw = &fwrt->fw->img[type];
int ret;
- if (fwrt->trans->cfg->gen2)
+ if (fwrt->trans->trans_cfg->gen2)
return 0;
/*
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
index 406ef73..be436c1 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
+++ b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h
@@ -64,7 +64,6 @@
#include "iwl-trans.h"
#include "img.h"
#include "fw/api/debug.h"
-#include "fw/api/dbg-tlv.h"
#include "fw/api/paging.h"
#include "iwl-eeprom-parse.h"
@@ -92,6 +91,20 @@ struct iwl_fwrt_shared_mem_cfg {
#define IWL_FW_RUNTIME_DUMP_WK_NUM 5
/**
+ * struct iwl_txf_iter_data - Tx fifo iterator data struct
+ * @fifo: fifo number
+ * @lmac: lmac number
+ * @fifo_size: fifo size
+ * @internal_txf: non zero if fifo is internal Tx fifo
+ */
+struct iwl_txf_iter_data {
+ int fifo;
+ int lmac;
+ u32 fifo_size;
+ u8 internal_txf;
+};
+
+/**
* struct iwl_fw_runtime - runtime data for firmware
* @fw: firmware image
* @cfg: NIC configuration
@@ -144,8 +157,8 @@ struct iwl_fw_runtime {
struct iwl_fw_ini_active_triggers active_trigs[IWL_FW_TRIGGER_ID_NUM];
u32 lmac_err_id[MAX_NUM_LMAC];
u32 umac_err_id;
- void *fifo_iter;
- struct timer_list periodic_trig;
+
+ struct iwl_txf_iter_data txf_iter_data;
u8 img_name[IWL_FW_INI_MAX_IMG_NAME_LEN];
u8 internal_dbg_cfg_name[IWL_FW_INI_MAX_DBG_CFG_NAME_LEN];
@@ -190,6 +203,10 @@ static inline void iwl_fw_runtime_free(struct iwl_fw_runtime *fwrt)
kfree(active->trig);
active->trig = NULL;
}
+
+ iwl_dbg_tlv_del_timers(fwrt->trans);
+ for (i = 0; i < IWL_FW_RUNTIME_DUMP_WK_NUM; i++)
+ cancel_delayed_work_sync(&fwrt->dump.wks[i].wk);
}
void iwl_fw_runtime_suspend(struct iwl_fw_runtime *fwrt);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/smem.c b/drivers/net/wireless/intel/iwlwifi/fw/smem.c
index 557ee47..409b2dd 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/smem.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/smem.c
@@ -151,7 +151,7 @@ void iwl_get_shared_mem_conf(struct iwl_fw_runtime *fwrt)
}
pkt = cmd.resp_pkt;
- if (fwrt->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000)
+ if (fwrt->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000)
iwl_parse_shared_mem_22000(fwrt, pkt);
else
iwl_parse_shared_mem(fwrt, pkt);
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-config.h b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
index 1c1bf1b..214495a 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-config.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
@@ -161,7 +161,8 @@ static inline u8 num_of_ant(u8 mask)
!!((mask) & ANT_C);
}
-/*
+/**
+ * struct iwl_base_params - params not likely to change within a device family
* @max_ll_items: max number of OTP blocks
* @shadow_ram_support: shadow support for OTP memory
* @led_compensation: compensate on the led on/off time per HW according
@@ -331,7 +332,36 @@ struct iwl_csr_params {
};
/**
+ * struct iwl_cfg_trans - information needed to start the trans
+ *
+ * These values cannot be changed when multiple configs are used for a
+ * single PCI ID, because they are needed before the HW REV or RFID
+ * can be read.
+ *
+ * @base_params: pointer to basic parameters
+ * @csr: csr flags and addresses that are different across devices
+ * @device_family: the device family
+ * @umac_prph_offset: offset to add to UMAC periphery address
+ * @rf_id: need to read rf_id to determine the firmware image
+ * @use_tfh: use TFH
+ * @gen2: 22000 and on transport operation
+ * @mq_rx_supported: multi-queue rx support
+ */
+struct iwl_cfg_trans_params {
+ const struct iwl_base_params *base_params;
+ const struct iwl_csr_params *csr;
+ enum iwl_device_family device_family;
+ u32 umac_prph_offset;
+ u32 rf_id:1,
+ use_tfh:1,
+ gen2:1,
+ mq_rx_supported:1,
+ bisr_workaround:1;
+};
+
+/**
* struct iwl_cfg
+ * @trans: the trans-specific configuration part
* @name: Official name of the device
* @fw_name_pre: Firmware filename prefix. The api version and extension
* (.ucode) will be added to filename before loading from disk. The
@@ -346,10 +376,10 @@ struct iwl_csr_params {
* @nvm_ver: NVM version
* @nvm_calib_ver: NVM calibration version
* @lib: pointer to the lib ops
- * @base_params: pointer to basic parameters
* @ht_params: point to ht parameters
* @led_mode: 0=blinking, 1=On(RF On)/Off(RF Off)
* @rx_with_siso_diversity: 1x1 device with rx antenna diversity
+ * @tx_with_siso_diversity: 1x1 device with tx antenna diversity
* @internal_wimax_coex: internal wifi/wimax combo device
* @high_temp: Is this NIC is designated to be in high temperature.
* @host_interrupt_operation_mode: device needs host interrupt operation
@@ -358,7 +388,6 @@ struct iwl_csr_params {
* @mac_addr_from_csr: read HW address from CSR registers
* @features: hw features, any combination of feature_whitelist
* @pwr_tx_backoffs: translation table between power limits and backoffs
- * @csr: csr flags and addresses that are different across devices
* @max_rx_agg_size: max RX aggregation size of the ADDBA request/response
* @max_tx_agg_size: max TX aggregation size of the ADDBA request/response
* @max_ht_ampdu_factor: the exponent of the max length of A-MPDU that the
@@ -371,18 +400,14 @@ struct iwl_csr_params {
* @dccm2_len: length of the second DCCM
* @smem_offset: offset from which the SMEM begins
* @smem_len: the length of SMEM
- * @mq_rx_supported: multi-queue rx support
* @vht_mu_mimo_supported: VHT MU-MIMO support
- * @rf_id: need to read rf_id to determine the firmware image
* @integrated: discrete or integrated
- * @gen2: 22000 and on transport operation
* @cdb: CDB support
* @nvm_type: see &enum iwl_nvm_type
* @d3_debug_data_base_addr: base address where D3 debug data is stored
* @d3_debug_data_length: length of the D3 debug data
* @bisr_workaround: BISR hardware workaround (for 22260 series devices)
* @min_txq_size: minimum number of slots required in a TX queue
- * @umac_prph_offset: offset to add to UMAC periphery address
* @uhb_supported: ultra high band channels supported
* @min_256_ba_txq_size: minimum number of slots required in a TX queue which
* supports 256 BA aggregation
@@ -392,19 +417,16 @@ struct iwl_csr_params {
* and/or the uCode API version instead.
*/
struct iwl_cfg {
+ struct iwl_cfg_trans_params trans;
/* params specific to an individual device within a device family */
const char *name;
const char *fw_name_pre;
- /* params not likely to change within a device family */
- const struct iwl_base_params *base_params;
/* params likely to change within a device family */
const struct iwl_ht_params *ht_params;
const struct iwl_eeprom_params *eeprom_params;
const struct iwl_pwr_tx_backoff *pwr_tx_backoffs;
const char *default_nvm_file_C_step;
const struct iwl_tt_params *thermal_params;
- const struct iwl_csr_params *csr;
- enum iwl_device_family device_family;
enum iwl_led_mode led_mode;
enum iwl_nvm_type nvm_type;
u32 max_data_size;
@@ -420,6 +442,7 @@ struct iwl_cfg {
u16 nvm_ver;
u16 nvm_calib_ver;
u32 rx_with_siso_diversity:1,
+ tx_with_siso_diversity:1,
bt_shared_single_ant:1,
internal_wimax_coex:1,
host_interrupt_operation_mode:1,
@@ -428,15 +451,10 @@ struct iwl_cfg {
lp_xtal_workaround:1,
disable_dummy_notification:1,
apmg_not_supported:1,
- mq_rx_supported:1,
vht_mu_mimo_supported:1,
- rf_id:1,
integrated:1,
- use_tfh:1,
- gen2:1,
cdb:1,
dbgc_supported:1,
- bisr_workaround:1,
uhb_supported:1;
u8 valid_tx_ant;
u8 valid_rx_ant;
@@ -453,7 +471,6 @@ struct iwl_cfg {
u32 d3_debug_data_base_addr;
u32 d3_debug_data_length;
u32 min_txq_size;
- u32 umac_prph_offset;
u32 fw_mon_smem_write_ptr_addr;
u32 fw_mon_smem_write_ptr_msk;
u32 fw_mon_smem_cycle_cnt_ptr_addr;
@@ -577,6 +594,8 @@ extern const struct iwl_cfg iwl_ax1650i_cfg_quz_hr;
extern const struct iwl_cfg iwl_ax1650s_cfg_quz_hr;
extern const struct iwl_cfg killer1650s_2ax_cfg_qu_b0_hr_b0;
extern const struct iwl_cfg killer1650i_2ax_cfg_qu_b0_hr_b0;
+extern const struct iwl_cfg killer1650s_2ax_cfg_qu_c0_hr_b0;
+extern const struct iwl_cfg killer1650i_2ax_cfg_qu_c0_hr_b0;
extern const struct iwl_cfg killer1650x_2ax_cfg;
extern const struct iwl_cfg killer1650w_2ax_cfg;
extern const struct iwl_cfg iwl9461_2ac_cfg_qu_b0_jf_b0;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
index fcaec41..3d7f8ff 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
@@ -60,149 +60,112 @@
*****************************************************************************/
#include <linux/firmware.h>
+#include "iwl-drv.h"
#include "iwl-trans.h"
#include "iwl-dbg-tlv.h"
+#include "fw/dbg.h"
+#include "fw/runtime.h"
-void iwl_fw_dbg_copy_tlv(struct iwl_trans *trans, struct iwl_ucode_tlv *tlv,
- bool ext)
+/**
+ * enum iwl_dbg_tlv_type - debug TLV types
+ * @IWL_DBG_TLV_TYPE_DEBUG_INFO: debug info TLV
+ * @IWL_DBG_TLV_TYPE_BUF_ALLOC: buffer allocation TLV
+ * @IWL_DBG_TLV_TYPE_HCMD: host command TLV
+ * @IWL_DBG_TLV_TYPE_REGION: region TLV
+ * @IWL_DBG_TLV_TYPE_TRIGGER: trigger TLV
+ * @IWL_DBG_TLV_TYPE_NUM: number of debug TLVs
+ */
+enum iwl_dbg_tlv_type {
+ IWL_DBG_TLV_TYPE_DEBUG_INFO =
+ IWL_UCODE_TLV_TYPE_DEBUG_INFO - IWL_UCODE_TLV_DEBUG_BASE,
+ IWL_DBG_TLV_TYPE_BUF_ALLOC,
+ IWL_DBG_TLV_TYPE_HCMD,
+ IWL_DBG_TLV_TYPE_REGION,
+ IWL_DBG_TLV_TYPE_TRIGGER,
+ IWL_DBG_TLV_TYPE_NUM,
+};
+
+/**
+ * struct iwl_dbg_tlv_ver_data - debug TLV version struct
+ * @min_ver: min version supported
+ * @max_ver: max version supported
+ */
+struct iwl_dbg_tlv_ver_data {
+ int min_ver;
+ int max_ver;
+};
+
+static const struct iwl_dbg_tlv_ver_data
+dbg_ver_table[IWL_DBG_TLV_TYPE_NUM] = {
+ [IWL_DBG_TLV_TYPE_DEBUG_INFO] = {.min_ver = 1, .max_ver = 1,},
+ [IWL_DBG_TLV_TYPE_BUF_ALLOC] = {.min_ver = 1, .max_ver = 1,},
+ [IWL_DBG_TLV_TYPE_HCMD] = {.min_ver = 1, .max_ver = 1,},
+ [IWL_DBG_TLV_TYPE_REGION] = {.min_ver = 1, .max_ver = 1,},
+ [IWL_DBG_TLV_TYPE_TRIGGER] = {.min_ver = 1, .max_ver = 1,},
+};
+
+static bool iwl_dbg_tlv_ver_support(struct iwl_ucode_tlv *tlv)
{
- struct iwl_apply_point_data *data;
- struct iwl_fw_ini_header *header = (void *)&tlv->data[0];
- u32 apply_point = le32_to_cpu(header->apply_point);
+ struct iwl_fw_ini_header *hdr = (void *)&tlv->data[0];
+ u32 type = le32_to_cpu(tlv->type);
+ u32 tlv_idx = type - IWL_UCODE_TLV_DEBUG_BASE;
+ u32 ver = le32_to_cpu(hdr->tlv_version);
- int copy_size = le32_to_cpu(tlv->length) + sizeof(*tlv);
- int offset_size = copy_size;
+ if (ver < dbg_ver_table[tlv_idx].min_ver ||
+ ver > dbg_ver_table[tlv_idx].max_ver)
+ return false;
- if (le32_to_cpu(header->tlv_version) != 1)
- return;
-
- if (WARN_ONCE(apply_point >= IWL_FW_INI_APPLY_NUM,
- "Invalid apply point id %d\n", apply_point))
- return;
-
- if (ext)
- data = &trans->dbg.apply_points_ext[apply_point];
- else
- data = &trans->dbg.apply_points[apply_point];
-
- /* add room for is_alloc field in &iwl_fw_ini_allocation_data struct */
- if (le32_to_cpu(tlv->type) == IWL_UCODE_TLV_TYPE_BUFFER_ALLOCATION) {
- struct iwl_fw_ini_allocation_data *buf_alloc =
- (void *)tlv->data;
-
- offset_size += sizeof(buf_alloc->is_alloc);
- }
-
- /*
- * Make sure we still have room to copy this TLV. Offset points to the
- * location the last copy ended.
- */
- if (WARN_ONCE(data->offset + offset_size > data->size,
- "Not enough memory for apply point %d\n",
- apply_point))
- return;
-
- memcpy(data->data + data->offset, (void *)tlv, copy_size);
- data->offset += offset_size;
+ return true;
}
-void iwl_alloc_dbg_tlv(struct iwl_trans *trans, size_t len, const u8 *data,
+void iwl_dbg_tlv_alloc(struct iwl_trans *trans, struct iwl_ucode_tlv *tlv,
bool ext)
{
- struct iwl_ucode_tlv *tlv;
- u32 size[IWL_FW_INI_APPLY_NUM] = {0};
- int i;
+ struct iwl_fw_ini_header *hdr = (void *)&tlv->data[0];
+ u32 type = le32_to_cpu(tlv->type);
+ u32 pnt = le32_to_cpu(hdr->apply_point);
+ u32 tlv_idx = type - IWL_UCODE_TLV_DEBUG_BASE;
+ enum iwl_ini_cfg_state *cfg_state = ext ?
+ &trans->dbg.external_ini_cfg : &trans->dbg.internal_ini_cfg;
- while (len >= sizeof(*tlv)) {
- u32 tlv_len, tlv_type, apply;
- struct iwl_fw_ini_header *hdr;
+ IWL_DEBUG_FW(trans, "WRT: read TLV 0x%x, apply point %d\n",
+ type, pnt);
- len -= sizeof(*tlv);
- tlv = (void *)data;
-
- tlv_len = le32_to_cpu(tlv->length);
- tlv_type = le32_to_cpu(tlv->type);
-
- if (len < tlv_len)
- return;
-
- len -= ALIGN(tlv_len, 4);
- data += sizeof(*tlv) + ALIGN(tlv_len, 4);
-
- if (tlv_type < IWL_UCODE_TLV_DEBUG_BASE ||
- tlv_type > IWL_UCODE_TLV_DEBUG_MAX)
- continue;
-
- hdr = (void *)&tlv->data[0];
- apply = le32_to_cpu(hdr->apply_point);
-
- if (le32_to_cpu(hdr->tlv_version) != 1)
- continue;
-
- IWL_DEBUG_FW(trans, "WRT: read TLV 0x%x, apply point %d\n",
- le32_to_cpu(tlv->type), apply);
-
- if (WARN_ON(apply >= IWL_FW_INI_APPLY_NUM))
- continue;
-
- /* add room for is_alloc field in &iwl_fw_ini_allocation_data
- * struct
- */
- if (tlv_type == IWL_UCODE_TLV_TYPE_BUFFER_ALLOCATION) {
- struct iwl_fw_ini_allocation_data *buf_alloc =
- (void *)tlv->data;
-
- size[apply] += sizeof(buf_alloc->is_alloc);
- }
-
- size[apply] += sizeof(*tlv) + tlv_len;
+ if (tlv_idx >= IWL_DBG_TLV_TYPE_NUM) {
+ IWL_ERR(trans, "WRT: Unsupported TLV 0x%x\n", type);
+ goto out_err;
}
- for (i = 0; i < ARRAY_SIZE(size); i++) {
- void *mem;
-
- if (!size[i])
- continue;
-
- mem = kzalloc(size[i], GFP_KERNEL);
-
- if (!mem) {
- IWL_ERR(trans, "No memory for apply point %d\n", i);
- return;
- }
-
- if (ext) {
- trans->dbg.apply_points_ext[i].data = mem;
- trans->dbg.apply_points_ext[i].size = size[i];
- } else {
- trans->dbg.apply_points[i].data = mem;
- trans->dbg.apply_points[i].size = size[i];
- }
-
- trans->dbg.ini_valid = true;
+ if (!iwl_dbg_tlv_ver_support(tlv)) {
+ IWL_ERR(trans, "WRT: Unsupported TLV 0x%x version %u\n", type,
+ le32_to_cpu(hdr->tlv_version));
+ goto out_err;
}
+
+ if (*cfg_state == IWL_INI_CFG_STATE_NOT_LOADED)
+ *cfg_state = IWL_INI_CFG_STATE_LOADED;
+
+ return;
+
+out_err:
+ *cfg_state = IWL_INI_CFG_STATE_CORRUPTED;
}
-void iwl_fw_dbg_free(struct iwl_trans *trans)
+void iwl_dbg_tlv_del_timers(struct iwl_trans *trans)
{
- int i;
+ /* will be used later */
+}
+IWL_EXPORT_SYMBOL(iwl_dbg_tlv_del_timers);
- for (i = 0; i < ARRAY_SIZE(trans->dbg.apply_points); i++) {
- kfree(trans->dbg.apply_points[i].data);
- trans->dbg.apply_points[i].size = 0;
- trans->dbg.apply_points[i].offset = 0;
-
- kfree(trans->dbg.apply_points_ext[i].data);
- trans->dbg.apply_points_ext[i].size = 0;
- trans->dbg.apply_points_ext[i].offset = 0;
- }
+void iwl_dbg_tlv_free(struct iwl_trans *trans)
+{
+ /* will be used again later */
}
-static int iwl_parse_fw_dbg_tlv(struct iwl_trans *trans, const u8 *data,
- size_t len)
+static int iwl_dbg_tlv_parse_bin(struct iwl_trans *trans, const u8 *data,
+ size_t len)
{
struct iwl_ucode_tlv *tlv;
- enum iwl_ucode_tlv_type tlv_type;
u32 tlv_len;
while (len >= sizeof(*tlv)) {
@@ -210,7 +173,6 @@ static int iwl_parse_fw_dbg_tlv(struct iwl_trans *trans, const u8 *data,
tlv = (void *)data;
tlv_len = le32_to_cpu(tlv->length);
- tlv_type = le32_to_cpu(tlv->type);
if (len < tlv_len) {
IWL_ERR(trans, "invalid TLV len: %zd/%u\n",
@@ -220,39 +182,33 @@ static int iwl_parse_fw_dbg_tlv(struct iwl_trans *trans, const u8 *data,
len -= ALIGN(tlv_len, 4);
data += sizeof(*tlv) + ALIGN(tlv_len, 4);
- switch (tlv_type) {
- case IWL_UCODE_TLV_TYPE_DEBUG_INFO:
- case IWL_UCODE_TLV_TYPE_BUFFER_ALLOCATION:
- case IWL_UCODE_TLV_TYPE_HCMD:
- case IWL_UCODE_TLV_TYPE_REGIONS:
- case IWL_UCODE_TLV_TYPE_TRIGGERS:
- case IWL_UCODE_TLV_TYPE_DEBUG_FLOW:
- iwl_fw_dbg_copy_tlv(trans, tlv, true);
- break;
- default:
- WARN_ONCE(1, "Invalid TLV %x\n", tlv_type);
- break;
- }
+ iwl_dbg_tlv_alloc(trans, tlv, true);
}
return 0;
}
-void iwl_load_fw_dbg_tlv(struct device *dev, struct iwl_trans *trans)
+void iwl_dbg_tlv_load_bin(struct device *dev, struct iwl_trans *trans)
{
const struct firmware *fw;
int res;
- if (trans->dbg.external_ini_loaded || !iwlwifi_mod_params.enable_ini)
+ if (!iwlwifi_mod_params.enable_ini)
return;
res = request_firmware(&fw, "iwl-dbg-tlv.ini", dev);
if (res)
return;
- iwl_alloc_dbg_tlv(trans, fw->size, fw->data, true);
- iwl_parse_fw_dbg_tlv(trans, fw->data, fw->size);
+ iwl_dbg_tlv_parse_bin(trans, fw->data, fw->size);
- trans->dbg.external_ini_loaded = true;
release_firmware(fw);
}
+
+void iwl_dbg_tlv_time_point(struct iwl_fw_runtime *fwrt,
+ enum iwl_fw_ini_time_point tp_id,
+ union iwl_dbg_tlv_tp_data *tp_data)
+{
+ /* will be used later */
+}
+IWL_EXPORT_SYMBOL(iwl_dbg_tlv_time_point);
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.h b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.h
index 222cd78..e257ad3 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.h
@@ -5,7 +5,7 @@
*
* GPL LICENSE SUMMARY
*
- * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -28,7 +28,7 @@
*
* BSD LICENSE
*
- * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -66,22 +66,32 @@
/**
* struct iwl_apply_point_data
- * @data: start address of this apply point data
- * @size total size of the data
- * @offset: current offset of the copied data
+ * @list: list to go through the TLVs of the apply point
+ * @tlv: a debug TLV
*/
struct iwl_apply_point_data {
- void *data;
- int size;
- int offset;
+ struct list_head list;
+ struct iwl_ucode_tlv tlv;
+};
+
+/**
+ * union iwl_dbg_tlv_tp_data - data that is given in a time point
+ * @fw_pkt: a packet received from the FW
+ */
+union iwl_dbg_tlv_tp_data {
+ struct iwl_rx_packet *fw_pkt;
};
struct iwl_trans;
-void iwl_load_fw_dbg_tlv(struct device *dev, struct iwl_trans *trans);
-void iwl_fw_dbg_free(struct iwl_trans *trans);
-void iwl_fw_dbg_copy_tlv(struct iwl_trans *trans, struct iwl_ucode_tlv *tlv,
- bool ext);
-void iwl_alloc_dbg_tlv(struct iwl_trans *trans, size_t len, const u8 *data,
+struct iwl_fw_runtime;
+
+void iwl_dbg_tlv_load_bin(struct device *dev, struct iwl_trans *trans);
+void iwl_dbg_tlv_free(struct iwl_trans *trans);
+void iwl_dbg_tlv_alloc(struct iwl_trans *trans, struct iwl_ucode_tlv *tlv,
bool ext);
+void iwl_dbg_tlv_time_point(struct iwl_fw_runtime *fwrt,
+ enum iwl_fw_ini_time_point tp_id,
+ union iwl_dbg_tlv_tp_data *tp_data);
+void iwl_dbg_tlv_del_timers(struct iwl_trans *trans);
#endif /* __iwl_dbg_tlv_h__*/
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-devtrace.h b/drivers/net/wireless/intel/iwlwifi/iwl-devtrace.h
index cba958e..fc8bc21 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-devtrace.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-devtrace.h
@@ -75,7 +75,6 @@ static inline size_t iwl_rx_trace_len(const struct iwl_trans *trans,
#include <linux/tracepoint.h>
#include <linux/device.h>
-#include "iwl-trans.h"
#if !defined(CONFIG_IWLWIFI_DEVICE_TRACING) || defined(__CHECKER__)
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
index 38672dd..ff0519e 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
@@ -215,7 +215,7 @@ static int iwl_request_firmware(struct iwl_drv *drv, bool first)
const struct iwl_cfg *cfg = drv->trans->cfg;
char tag[8];
- if (drv->trans->cfg->device_family == IWL_DEVICE_FAMILY_9000 &&
+ if (drv->trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_9000 &&
(CSR_HW_REV_STEP(drv->trans->hw_rev) != SILICON_B_STEP &&
CSR_HW_REV_STEP(drv->trans->hw_rev) != SILICON_C_STEP)) {
IWL_ERR(drv,
@@ -647,9 +647,6 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
len -= sizeof(*ucode);
- if (iwlwifi_mod_params.enable_ini)
- iwl_alloc_dbg_tlv(drv->trans, len, data, false);
-
while (len >= sizeof(*tlv)) {
len -= sizeof(*tlv);
tlv = (void *)data;
@@ -1123,7 +1120,7 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
if (tlv_len != sizeof(*dbg_ptrs))
goto invalid_tlv_len;
- if (drv->trans->cfg->device_family <
+ if (drv->trans->trans_cfg->device_family <
IWL_DEVICE_FAMILY_22000)
break;
drv->trans->dbg.umac_error_event_table =
@@ -1139,7 +1136,7 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
if (tlv_len != sizeof(*dbg_ptrs))
goto invalid_tlv_len;
- if (drv->trans->cfg->device_family <
+ if (drv->trans->trans_cfg->device_family <
IWL_DEVICE_FAMILY_22000)
break;
drv->trans->dbg.lmac_error_event_table[0] =
@@ -1154,9 +1151,8 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv,
case IWL_UCODE_TLV_TYPE_HCMD:
case IWL_UCODE_TLV_TYPE_REGIONS:
case IWL_UCODE_TLV_TYPE_TRIGGERS:
- case IWL_UCODE_TLV_TYPE_DEBUG_FLOW:
if (iwlwifi_mod_params.enable_ini)
- iwl_fw_dbg_copy_tlv(drv->trans, tlv, false);
+ iwl_dbg_tlv_alloc(drv->trans, tlv, false);
break;
case IWL_UCODE_TLV_CMD_VERSIONS:
if (tlv_len % sizeof(struct iwl_fw_cmd_version)) {
@@ -1526,14 +1522,14 @@ static void iwl_req_fw_callback(const struct firmware *ucode_raw, void *context)
fw->init_evtlog_size = (pieces->init_evtlog_size - 16)/12;
else
fw->init_evtlog_size =
- drv->trans->cfg->base_params->max_event_log_size;
+ drv->trans->trans_cfg->base_params->max_event_log_size;
fw->init_errlog_ptr = pieces->init_errlog_ptr;
fw->inst_evtlog_ptr = pieces->inst_evtlog_ptr;
if (pieces->inst_evtlog_size)
fw->inst_evtlog_size = (pieces->inst_evtlog_size - 16)/12;
else
fw->inst_evtlog_size =
- drv->trans->cfg->base_params->max_event_log_size;
+ drv->trans->trans_cfg->base_params->max_event_log_size;
fw->inst_errlog_ptr = pieces->inst_errlog_ptr;
/*
@@ -1640,7 +1636,7 @@ struct iwl_drv *iwl_drv_start(struct iwl_trans *trans)
init_completion(&drv->request_firmware_complete);
INIT_LIST_HEAD(&drv->list);
- iwl_load_fw_dbg_tlv(drv->trans->dev, drv->trans);
+ iwl_dbg_tlv_load_bin(drv->trans->dev, drv->trans);
#ifdef CONFIG_IWLWIFI_DEBUGFS
/* Create the device debugfs entries. */
@@ -1662,8 +1658,8 @@ struct iwl_drv *iwl_drv_start(struct iwl_trans *trans)
err_fw:
#ifdef CONFIG_IWLWIFI_DEBUGFS
debugfs_remove_recursive(drv->dbgfs_drv);
+ iwl_dbg_tlv_free(drv->trans);
#endif
- iwl_fw_dbg_free(drv->trans);
kfree(drv);
err:
return ERR_PTR(ret);
@@ -1693,7 +1689,7 @@ void iwl_drv_stop(struct iwl_drv *drv)
debugfs_remove_recursive(drv->dbgfs_drv);
#endif
- iwl_fw_dbg_free(drv->trans);
+ iwl_dbg_tlv_free(drv->trans);
kfree(drv);
}
@@ -1704,8 +1700,6 @@ struct iwl_mod_params iwlwifi_mod_params = {
.fw_restart = true,
.bt_coex_active = true,
.power_level = IWL_POWER_INDEX_1,
- .d0i3_disable = true,
- .d0i3_timeout = 1000,
.uapsd_disable = IWL_DISABLE_UAPSD_BSS | IWL_DISABLE_UAPSD_P2P_CLIENT,
/* the rest are 0 by default */
};
@@ -1823,9 +1817,6 @@ MODULE_PARM_DESC(antenna_coupling,
module_param_named(nvm_file, iwlwifi_mod_params.nvm_file, charp, 0444);
MODULE_PARM_DESC(nvm_file, "NVM file name");
-module_param_named(d0i3_disable, iwlwifi_mod_params.d0i3_disable, bool, 0444);
-MODULE_PARM_DESC(d0i3_disable, "disable d0i3 functionality (default: Y)");
-
module_param_named(lar_disable, iwlwifi_mod_params.lar_disable, bool, 0444);
MODULE_PARM_DESC(lar_disable, "disable LAR functionality (default: N)");
@@ -1873,9 +1864,6 @@ module_param_named(fw_monitor, iwlwifi_mod_params.fw_monitor, bool, 0444);
MODULE_PARM_DESC(fw_monitor,
"firmware monitor - to debug FW (default: false - needs lots of memory)");
-module_param_named(d0i3_timeout, iwlwifi_mod_params.d0i3_timeout, uint, 0444);
-MODULE_PARM_DESC(d0i3_timeout, "Timeout to D0i3 entry when idle (ms)");
-
module_param_named(disable_11ac, iwlwifi_mod_params.disable_11ac, bool, 0444);
MODULE_PARM_DESC(disable_11ac, "Disable VHT capabilities (default: false)");
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c
index 04338c3..cf7e2a9 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c
@@ -7,7 +7,7 @@
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2015 Intel Mobile Communications GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -29,7 +29,7 @@
*
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2015 Intel Mobile Communications GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -728,12 +728,13 @@ int iwl_init_sband_channels(struct iwl_nvm_data *data,
#define MAX_BIT_RATE_40_MHZ 150 /* Mbps */
#define MAX_BIT_RATE_20_MHZ 72 /* Mbps */
-void iwl_init_ht_hw_capab(const struct iwl_cfg *cfg,
+void iwl_init_ht_hw_capab(struct iwl_trans *trans,
struct iwl_nvm_data *data,
struct ieee80211_sta_ht_cap *ht_info,
enum nl80211_band band,
u8 tx_chains, u8 rx_chains)
{
+ const struct iwl_cfg *cfg = trans->cfg;
int max_bit_rate = 0;
tx_chains = hweight8(tx_chains);
@@ -765,7 +766,7 @@ void iwl_init_ht_hw_capab(const struct iwl_cfg *cfg,
if (cfg->ht_params->ldpc)
ht_info->cap |= IEEE80211_HT_CAP_LDPC_CODING;
- if ((cfg->mq_rx_supported &&
+ if ((trans->trans_cfg->mq_rx_supported &&
iwlwifi_mod_params.amsdu_size == IWL_AMSDU_DEF) ||
iwlwifi_mod_params.amsdu_size >= IWL_AMSDU_8K)
ht_info->cap |= IEEE80211_HT_CAP_MAX_AMSDU;
@@ -805,10 +806,11 @@ void iwl_init_ht_hw_capab(const struct iwl_cfg *cfg,
}
}
-static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
+static void iwl_init_sbands(struct iwl_trans *trans, const struct iwl_cfg *cfg,
struct iwl_nvm_data *data,
const u8 *eeprom, size_t eeprom_size)
{
+ struct device *dev = trans->dev;
int n_channels = iwl_init_channel_map(dev, cfg, data,
eeprom, eeprom_size);
int n_used = 0;
@@ -820,7 +822,7 @@ static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
sband->n_bitrates = N_RATES_24;
n_used += iwl_init_sband_channels(data, sband, n_channels,
NL80211_BAND_2GHZ);
- iwl_init_ht_hw_capab(cfg, data, &sband->ht_cap, NL80211_BAND_2GHZ,
+ iwl_init_ht_hw_capab(trans, data, &sband->ht_cap, NL80211_BAND_2GHZ,
data->valid_tx_ant, data->valid_rx_ant);
sband = &data->bands[NL80211_BAND_5GHZ];
@@ -829,7 +831,7 @@ static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
sband->n_bitrates = N_RATES_52;
n_used += iwl_init_sband_channels(data, sband, n_channels,
NL80211_BAND_5GHZ);
- iwl_init_ht_hw_capab(cfg, data, &sband->ht_cap, NL80211_BAND_5GHZ,
+ iwl_init_ht_hw_capab(trans, data, &sband->ht_cap, NL80211_BAND_5GHZ,
data->valid_tx_ant, data->valid_rx_ant);
if (n_channels != n_used)
@@ -840,10 +842,11 @@ static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
/* EEPROM data functions */
struct iwl_nvm_data *
-iwl_parse_eeprom_data(struct device *dev, const struct iwl_cfg *cfg,
+iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
const u8 *eeprom, size_t eeprom_size)
{
struct iwl_nvm_data *data;
+ struct device *dev = trans->dev;
const void *tmp;
u16 radio_cfg, sku;
@@ -918,7 +921,7 @@ iwl_parse_eeprom_data(struct device *dev, const struct iwl_cfg *cfg,
goto err_free;
}
- iwl_init_sbands(dev, cfg, data, eeprom, eeprom_size);
+ iwl_init_sbands(trans, cfg, data, eeprom, eeprom_size);
return data;
err_free:
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
index 2375d30..03a748c 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
@@ -116,14 +116,14 @@ struct iwl_nvm_data {
* later with iwl_free_nvm_data().
*/
struct iwl_nvm_data *
-iwl_parse_eeprom_data(struct device *dev, const struct iwl_cfg *cfg,
+iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
const u8 *eeprom, size_t eeprom_size);
int iwl_init_sband_channels(struct iwl_nvm_data *data,
struct ieee80211_supported_band *sband,
int n_channels, enum nl80211_band band);
-void iwl_init_ht_hw_capab(const struct iwl_cfg *cfg,
+void iwl_init_ht_hw_capab(struct iwl_trans *trans,
struct iwl_nvm_data *data,
struct ieee80211_sta_ht_cap *ht_info,
enum nl80211_band band,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c
index 82e8719..ad6dc44 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c
@@ -6,7 +6,7 @@
* GPL LICENSE SUMMARY
*
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -27,7 +27,7 @@
* BSD LICENSE
*
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -193,7 +193,7 @@ static int iwl_init_otp_access(struct iwl_trans *trans)
{
int ret;
- ret = iwl_finish_nic_init(trans);
+ ret = iwl_finish_nic_init(trans, trans->trans_cfg);
if (ret)
return ret;
@@ -207,7 +207,7 @@ static int iwl_init_otp_access(struct iwl_trans *trans)
* CSR auto clock gate disable bit -
* this is only applicable for HW with OTP shadow RAM
*/
- if (trans->cfg->base_params->shadow_ram_support)
+ if (trans->trans_cfg->base_params->shadow_ram_support)
iwl_set_bit(trans, CSR_DBG_LINK_PWR_MGMT_REG,
CSR_RESET_LINK_PWR_MGMT_DISABLED);
@@ -328,7 +328,7 @@ static int iwl_find_otp_image(struct iwl_trans *trans,
}
/* more in the link list, continue */
usedblocks++;
- } while (usedblocks <= trans->cfg->base_params->max_ll_items);
+ } while (usedblocks <= trans->trans_cfg->base_params->max_ll_items);
/* OTP has no valid blocks */
IWL_DEBUG_EEPROM(trans->dev, "OTP has no valid blocks\n");
@@ -361,7 +361,7 @@ int iwl_read_eeprom(struct iwl_trans *trans, u8 **eeprom, size_t *eeprom_size)
if (nvm_is_otp < 0)
return nvm_is_otp;
- sz = trans->cfg->base_params->eeprom_size;
+ sz = trans->trans_cfg->base_params->eeprom_size;
IWL_DEBUG_EEPROM(trans->dev, "NVM size = %d\n", sz);
e = kmalloc(sz, GFP_KERNEL);
@@ -396,7 +396,7 @@ int iwl_read_eeprom(struct iwl_trans *trans, u8 **eeprom, size_t *eeprom_size)
CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK |
CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK);
/* traversing the linked list if no shadow ram supported */
- if (!trans->cfg->base_params->shadow_ram_support) {
+ if (!trans->trans_cfg->base_params->shadow_ram_support) {
ret = iwl_find_otp_image(trans, &validblockaddr);
if (ret)
goto err_unlock;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-fh.h b/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
index c6a5343..0c12df5 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
@@ -7,7 +7,7 @@
*
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2015 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -29,7 +29,7 @@
*
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2015 - 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -127,7 +127,7 @@
static inline unsigned int FH_MEM_CBBC_QUEUE(struct iwl_trans *trans,
unsigned int chnl)
{
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
WARN_ON_ONCE(chnl >= 64);
return TFH_TFDQ_CBB_TABLE + 8 * chnl;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-io.c b/drivers/net/wireless/intel/iwlwifi/iwl-io.c
index a704e25..1b7414b 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-io.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-io.c
@@ -304,10 +304,10 @@ IWL_EXPORT_SYMBOL(iwl_clear_bits_prph);
void iwl_force_nmi(struct iwl_trans *trans)
{
- if (trans->cfg->device_family < IWL_DEVICE_FAMILY_9000)
+ if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_9000)
iwl_write_prph(trans, DEVICE_SET_NMI_REG,
DEVICE_SET_NMI_VAL_DRV);
- else if (trans->cfg->device_family < IWL_DEVICE_FAMILY_AX210)
+ else if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_AX210)
iwl_write_umac_prph(trans, UREG_NIC_SET_NMI_DRIVER,
UREG_NIC_SET_NMI_DRIVER_NMI_FROM_DRIVER_MSK);
else
@@ -458,7 +458,7 @@ int iwl_dump_fh(struct iwl_trans *trans, char **buf)
FH_TSSR_TX_ERROR_REG
};
- if (trans->cfg->mq_rx_supported)
+ if (trans->trans_cfg->mq_rx_supported)
return iwl_dump_rfh(trans, buf);
#ifdef CONFIG_IWLWIFI_DEBUGFS
@@ -492,11 +492,12 @@ int iwl_dump_fh(struct iwl_trans *trans, char **buf)
return 0;
}
-int iwl_finish_nic_init(struct iwl_trans *trans)
+int iwl_finish_nic_init(struct iwl_trans *trans,
+ const struct iwl_cfg_trans_params *cfg_trans)
{
int err;
- if (trans->cfg->bisr_workaround) {
+ if (cfg_trans->bisr_workaround) {
/* ensure the TOP FSM isn't still in previous reset */
mdelay(2);
}
@@ -506,9 +507,9 @@ int iwl_finish_nic_init(struct iwl_trans *trans)
* D0U* --> D0A* (powered-up active) state.
*/
iwl_set_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_init_done));
+ BIT(cfg_trans->csr->flag_init_done));
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_8000)
+ if (cfg_trans->device_family == IWL_DEVICE_FAMILY_8000)
udelay(2);
/*
@@ -517,13 +518,13 @@ int iwl_finish_nic_init(struct iwl_trans *trans)
* and accesses to uCode SRAM.
*/
err = iwl_poll_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_clock_ready),
- BIT(trans->cfg->csr->flag_mac_clock_ready),
+ BIT(cfg_trans->csr->flag_mac_clock_ready),
+ BIT(cfg_trans->csr->flag_mac_clock_ready),
25000);
if (err < 0)
IWL_DEBUG_INFO(trans, "Failed to wake NIC\n");
- if (trans->cfg->bisr_workaround) {
+ if (cfg_trans->bisr_workaround) {
/* ensure BISR shift has finished */
udelay(200);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-io.h b/drivers/net/wireless/intel/iwlwifi/iwl-io.h
index 920e214..f8e4f0f 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-io.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-io.h
@@ -99,7 +99,8 @@ void iwl_set_bits_mask_prph(struct iwl_trans *trans, u32 ofs,
void iwl_clear_bits_prph(struct iwl_trans *trans, u32 ofs, u32 mask);
void iwl_force_nmi(struct iwl_trans *trans);
-int iwl_finish_nic_init(struct iwl_trans *trans);
+int iwl_finish_nic_init(struct iwl_trans *trans,
+ const struct iwl_cfg_trans_params *cfg_trans);
/* Error handling */
int iwl_dump_fh(struct iwl_trans *trans, char **buf);
@@ -111,35 +112,38 @@ int iwl_dump_fh(struct iwl_trans *trans, char **buf);
*/
static inline u32 iwl_umac_prph(struct iwl_trans *trans, u32 ofs)
{
- return ofs + trans->cfg->umac_prph_offset;
+ return ofs + trans->cfg->trans.umac_prph_offset;
}
static inline u32 iwl_read_umac_prph_no_grab(struct iwl_trans *trans, u32 ofs)
{
- return iwl_read_prph_no_grab(trans, ofs + trans->cfg->umac_prph_offset);
+ return iwl_read_prph_no_grab(trans, ofs +
+ trans->cfg->trans.umac_prph_offset);
}
static inline u32 iwl_read_umac_prph(struct iwl_trans *trans, u32 ofs)
{
- return iwl_read_prph(trans, ofs + trans->cfg->umac_prph_offset);
+ return iwl_read_prph(trans, ofs + trans->cfg->trans.umac_prph_offset);
}
static inline void iwl_write_umac_prph_no_grab(struct iwl_trans *trans, u32 ofs,
u32 val)
{
- iwl_write_prph_no_grab(trans, ofs + trans->cfg->umac_prph_offset, val);
+ iwl_write_prph_no_grab(trans, ofs + trans->cfg->trans.umac_prph_offset,
+ val);
}
static inline void iwl_write_umac_prph(struct iwl_trans *trans, u32 ofs,
u32 val)
{
- iwl_write_prph(trans, ofs + trans->cfg->umac_prph_offset, val);
+ iwl_write_prph(trans, ofs + trans->cfg->trans.umac_prph_offset, val);
}
static inline int iwl_poll_umac_prph_bit(struct iwl_trans *trans, u32 addr,
u32 bits, u32 mask, int timeout)
{
- return iwl_poll_prph_bit(trans, addr + trans->cfg->umac_prph_offset,
+ return iwl_poll_prph_bit(trans, addr +
+ trans->cfg->trans.umac_prph_offset,
bits, mask, timeout);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h b/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h
index 0cae2ef..ebea3f3 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h
@@ -6,7 +6,7 @@
* GPL LICENSE SUMMARY
*
* Copyright(c) 2007 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -27,7 +27,7 @@
* BSD LICENSE
*
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -115,9 +115,6 @@ enum iwl_uapsd_disable {
* @nvm_file: specifies a external NVM file
* @uapsd_disable: disable U-APSD, see &enum iwl_uapsd_disable, default =
* IWL_DISABLE_UAPSD_BSS | IWL_DISABLE_UAPSD_P2P_CLIENT
- * @d0i3_disable: disable d0i3, default = 1,
- * @d0i3_timeout: time to wait after no refs are taken before
- * entering D0i3 (in msecs)
* @lar_disable: disable LAR (regulatory), default = 0
* @fw_monitor: allow to use firmware monitor
* @disable_11ac: disable VHT capabilities, default = false.
@@ -139,8 +136,6 @@ struct iwl_mod_params {
int antenna_coupling;
char *nvm_file;
u32 uapsd_disable;
- bool d0i3_disable;
- unsigned int d0i3_timeout;
bool lar_disable;
bool fw_monitor;
bool disable_11ac;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
index d87a6bb..c8972f6e 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
@@ -393,11 +393,12 @@ static int iwl_init_channel_map(struct device *dev, const struct iwl_cfg *cfg,
return n_channels;
}
-static void iwl_init_vht_hw_capab(const struct iwl_cfg *cfg,
+static void iwl_init_vht_hw_capab(struct iwl_trans *trans,
struct iwl_nvm_data *data,
struct ieee80211_sta_vht_cap *vht_cap,
u8 tx_chains, u8 rx_chains)
{
+ const struct iwl_cfg *cfg = trans->cfg;
int num_rx_ants = num_of_ant(rx_chains);
int num_tx_ants = num_of_ant(tx_chains);
unsigned int max_ampdu_exponent = (cfg->max_vht_ampdu_exponent ?:
@@ -434,14 +435,14 @@ static void iwl_init_vht_hw_capab(const struct iwl_cfg *cfg,
switch (iwlwifi_mod_params.amsdu_size) {
case IWL_AMSDU_DEF:
- if (cfg->mq_rx_supported)
+ if (trans->trans_cfg->mq_rx_supported)
vht_cap->cap |=
IEEE80211_VHT_CAP_MAX_MPDU_LENGTH_11454;
else
vht_cap->cap |= IEEE80211_VHT_CAP_MAX_MPDU_LENGTH_3895;
break;
case IWL_AMSDU_2K:
- if (cfg->mq_rx_supported)
+ if (trans->trans_cfg->mq_rx_supported)
vht_cap->cap |=
IEEE80211_VHT_CAP_MAX_MPDU_LENGTH_11454;
else
@@ -669,11 +670,13 @@ static void iwl_init_he_hw_capab(struct ieee80211_supported_band *sband,
}
}
-static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
+static void iwl_init_sbands(struct iwl_trans *trans,
struct iwl_nvm_data *data,
const void *nvm_ch_flags, u8 tx_chains,
u8 rx_chains, u32 sbands_flags, bool v4)
{
+ struct device *dev = trans->dev;
+ const struct iwl_cfg *cfg = trans->cfg;
int n_channels;
int n_used = 0;
struct ieee80211_supported_band *sband;
@@ -686,7 +689,7 @@ static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
sband->n_bitrates = N_RATES_24;
n_used += iwl_init_sband_channels(data, sband, n_channels,
NL80211_BAND_2GHZ);
- iwl_init_ht_hw_capab(cfg, data, &sband->ht_cap, NL80211_BAND_2GHZ,
+ iwl_init_ht_hw_capab(trans, data, &sband->ht_cap, NL80211_BAND_2GHZ,
tx_chains, rx_chains);
if (data->sku_cap_11ax_enable && !iwlwifi_mod_params.disable_11ax)
@@ -698,10 +701,10 @@ static void iwl_init_sbands(struct device *dev, const struct iwl_cfg *cfg,
sband->n_bitrates = N_RATES_52;
n_used += iwl_init_sband_channels(data, sband, n_channels,
NL80211_BAND_5GHZ);
- iwl_init_ht_hw_capab(cfg, data, &sband->ht_cap, NL80211_BAND_5GHZ,
+ iwl_init_ht_hw_capab(trans, data, &sband->ht_cap, NL80211_BAND_5GHZ,
tx_chains, rx_chains);
if (data->sku_cap_11ac_enable && !iwlwifi_mod_params.disable_11ac)
- iwl_init_vht_hw_capab(cfg, data, &sband->vht_cap,
+ iwl_init_vht_hw_capab(trans, data, &sband->vht_cap,
tx_chains, rx_chains);
if (data->sku_cap_11ax_enable && !iwlwifi_mod_params.disable_11ax)
@@ -793,10 +796,10 @@ static void iwl_set_hw_address_from_csr(struct iwl_trans *trans,
{
__le32 mac_addr0 =
cpu_to_le32(iwl_read32(trans,
- trans->cfg->csr->mac_addr0_strap));
+ trans->trans_cfg->csr->mac_addr0_strap));
__le32 mac_addr1 =
cpu_to_le32(iwl_read32(trans,
- trans->cfg->csr->mac_addr1_strap));
+ trans->trans_cfg->csr->mac_addr1_strap));
iwl_flip_hw_address(mac_addr0, mac_addr1, data->hw_addr);
/*
@@ -807,9 +810,9 @@ static void iwl_set_hw_address_from_csr(struct iwl_trans *trans,
return;
mac_addr0 = cpu_to_le32(iwl_read32(trans,
- trans->cfg->csr->mac_addr0_otp));
+ trans->trans_cfg->csr->mac_addr0_otp));
mac_addr1 = cpu_to_le32(iwl_read32(trans,
- trans->cfg->csr->mac_addr1_otp));
+ trans->trans_cfg->csr->mac_addr1_otp));
iwl_flip_hw_address(mac_addr0, mac_addr1, data->hw_addr);
}
@@ -896,7 +899,7 @@ static int iwl_set_hw_address(struct iwl_trans *trans,
}
static bool
-iwl_nvm_no_wide_in_5ghz(struct device *dev, const struct iwl_cfg *cfg,
+iwl_nvm_no_wide_in_5ghz(struct iwl_trans *trans, const struct iwl_cfg *cfg,
const __be16 *nvm_hw)
{
/*
@@ -908,7 +911,7 @@ iwl_nvm_no_wide_in_5ghz(struct device *dev, const struct iwl_cfg *cfg,
* in 5GHz otherwise the FW will throw a sysassert when we try
* to use them.
*/
- if (cfg->device_family == IWL_DEVICE_FAMILY_7000) {
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_7000) {
/*
* Unlike the other sections in the NVM, the hw
* section uses big-endian.
@@ -917,7 +920,7 @@ iwl_nvm_no_wide_in_5ghz(struct device *dev, const struct iwl_cfg *cfg,
u8 sku = (subsystem_id & 0x1e) >> 1;
if (sku == 5 || sku == 9) {
- IWL_DEBUG_EEPROM(dev,
+ IWL_DEBUG_EEPROM(trans->dev,
"disabling wide channels in 5GHz (0x%0x %d)\n",
subsystem_id, sku);
return true;
@@ -934,7 +937,6 @@ iwl_parse_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
const __le16 *mac_override, const __le16 *phy_sku,
u8 tx_chains, u8 rx_chains, bool lar_fw_supported)
{
- struct device *dev = trans->dev;
struct iwl_nvm_data *data;
bool lar_enabled;
u32 sku, radio_cfg;
@@ -942,7 +944,11 @@ iwl_parse_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
u16 lar_config;
const __le16 *ch_section;
- if (cfg->nvm_type != IWL_NVM_EXT)
+ if (cfg->uhb_supported)
+ data = kzalloc(struct_size(data, channels,
+ IWL_NVM_NUM_CHANNELS_UHB),
+ GFP_KERNEL);
+ else if (cfg->nvm_type != IWL_NVM_EXT)
data = kzalloc(struct_size(data, channels,
IWL_NVM_NUM_CHANNELS),
GFP_KERNEL);
@@ -1012,10 +1018,10 @@ iwl_parse_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg,
if (lar_fw_supported && lar_enabled)
sbands_flags |= IWL_NVM_SBANDS_FLAGS_LAR;
- if (iwl_nvm_no_wide_in_5ghz(dev, cfg, nvm_hw))
+ if (iwl_nvm_no_wide_in_5ghz(trans, cfg, nvm_hw))
sbands_flags |= IWL_NVM_SBANDS_FLAGS_NO_WIDE_IN_5GHZ;
- iwl_init_sbands(dev, cfg, data, ch_section, tx_chains, rx_chains,
+ iwl_init_sbands(trans, data, ch_section, tx_chains, rx_chains,
sbands_flags, false);
data->calib_version = 255;
@@ -1066,11 +1072,6 @@ static u32 iwl_nvm_get_regdom_bw_flags(const u16 *nvm_chan,
return flags;
}
-struct regdb_ptrs {
- struct ieee80211_wmm_rule *rule;
- u32 token;
-};
-
struct ieee80211_regdomain *
iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg,
int num_of_ch, __le32 *channels, u16 fw_mcc,
@@ -1082,7 +1083,6 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg,
const u16 *nvm_chan;
struct ieee80211_regdomain *regd, *copy_rd;
struct ieee80211_reg_rule *rule;
- struct regdb_ptrs *regdb_ptrs;
enum nl80211_band band;
int center_freq, prev_center_freq = 0;
int valid_rules = 0;
@@ -1114,12 +1114,6 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg,
if (!regd)
return ERR_PTR(-ENOMEM);
- regdb_ptrs = kcalloc(num_of_ch, sizeof(*regdb_ptrs), GFP_KERNEL);
- if (!regdb_ptrs) {
- copy_rd = ERR_PTR(-ENOMEM);
- goto out;
- }
-
/* set alpha2 from FW. */
regd->alpha2[0] = fw_mcc >> 8;
regd->alpha2[1] = fw_mcc & 0xff;
@@ -1191,8 +1185,6 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg,
if (!copy_rd)
copy_rd = ERR_PTR(-ENOMEM);
-out:
- kfree(regdb_ptrs);
kfree(regd);
return copy_rd;
}
@@ -1311,7 +1303,7 @@ int iwl_read_external_nvm(struct iwl_trans *trans,
le32_to_cpu(dword_buff[3]));
/* nvm file validation, dword_buff[2] holds the file version */
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_8000 &&
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_8000 &&
CSR_HW_REV_STEP(trans->hw_rev) == SILICON_C_STEP &&
le32_to_cpu(dword_buff[2]) < 0xE4A) {
ret = -EFAULT;
@@ -1495,7 +1487,7 @@ struct iwl_nvm_data *iwl_get_nvm(struct iwl_trans *trans,
channel_profile = v4 ? (void *)rsp->regulatory.channel_profile :
(void *)rsp_v3->regulatory.channel_profile;
- iwl_init_sbands(trans->dev, trans->cfg, nvm,
+ iwl_init_sbands(trans, nvm,
channel_profile,
nvm->valid_tx_ant & fw->valid_tx_ant,
nvm->valid_rx_ant & fw->valid_rx_ant,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-op-mode.h b/drivers/net/wireless/intel/iwlwifi/iwl-op-mode.h
index cbd1a8e..3008a52 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-op-mode.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-op-mode.h
@@ -8,7 +8,7 @@
* Copyright(c) 2007 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2015 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -31,7 +31,7 @@
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2015 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -140,9 +140,6 @@ struct iwl_cfg;
* @nic_config: configure NIC, called before firmware is started.
* May sleep
* @wimax_active: invoked when WiMax becomes active. May sleep
- * @enter_d0i3: configure the fw to enter d0i3. return 1 to indicate d0i3
- * entrance is aborted (e.g. due to held reference). May sleep.
- * @exit_d0i3: configure the fw to exit d0i3. May sleep.
*/
struct iwl_op_mode_ops {
struct iwl_op_mode *(*start)(struct iwl_trans *trans,
@@ -164,8 +161,6 @@ struct iwl_op_mode_ops {
void (*cmd_queue_full)(struct iwl_op_mode *op_mode);
void (*nic_config)(struct iwl_op_mode *op_mode);
void (*wimax_active)(struct iwl_op_mode *op_mode);
- int (*enter_d0i3)(struct iwl_op_mode *op_mode);
- int (*exit_d0i3)(struct iwl_op_mode *op_mode);
};
int iwl_opmode_register(const char *name, const struct iwl_op_mode_ops *ops);
@@ -258,22 +253,4 @@ static inline void iwl_op_mode_wimax_active(struct iwl_op_mode *op_mode)
op_mode->ops->wimax_active(op_mode);
}
-static inline int iwl_op_mode_enter_d0i3(struct iwl_op_mode *op_mode)
-{
- might_sleep();
-
- if (!op_mode->ops->enter_d0i3)
- return 0;
- return op_mode->ops->enter_d0i3(op_mode);
-}
-
-static inline int iwl_op_mode_exit_d0i3(struct iwl_op_mode *op_mode)
-{
- might_sleep();
-
- if (!op_mode->ops->exit_d0i3)
- return 0;
- return op_mode->ops->exit_d0i3(op_mode);
-}
-
#endif /* __iwl_op_mode_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-prph.h b/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
index 8d930bf..f47e0f9 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-prph.h
@@ -451,6 +451,8 @@ enum {
#define UREG_DOORBELL_TO_ISR6 0xA05C04
#define UREG_DOORBELL_TO_ISR6_NMI_BIT BIT(0)
+#define UREG_DOORBELL_TO_ISR6_SUSPEND BIT(18)
+#define UREG_DOORBELL_TO_ISR6_RESUME BIT(19)
#define FSEQ_ERROR_CODE 0xA340C8
#define FSEQ_TOP_INIT_VERSION 0xA34038
@@ -460,4 +462,7 @@ enum {
#define FSEQ_ALIVE_TOKEN 0xA340F0
#define FSEQ_CNVI_ID 0xA3408C
#define FSEQ_CNVR_ID 0xA34090
+
+#define IWL_D3_SLEEP_STATUS_SUSPEND 0xD3
+#define IWL_D3_SLEEP_STATUS_RESUME 0xD0
#endif /* __iwl_prph_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-trans.c b/drivers/net/wireless/intel/iwlwifi/iwl-trans.c
index 727f73e..28bdc9a 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-trans.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-trans.c
@@ -66,7 +66,6 @@
struct iwl_trans *iwl_trans_alloc(unsigned int priv_size,
struct device *dev,
- const struct iwl_cfg *cfg,
const struct iwl_trans_ops *ops)
{
struct iwl_trans *trans;
@@ -84,7 +83,6 @@ struct iwl_trans *iwl_trans_alloc(unsigned int priv_size,
#endif
trans->dev = dev;
- trans->cfg = cfg;
trans->ops = ops;
trans->num_rx_queues = 1;
@@ -202,17 +200,3 @@ int iwl_cmd_groups_verify_sorted(const struct iwl_trans_config *trans)
return 0;
}
IWL_EXPORT_SYMBOL(iwl_cmd_groups_verify_sorted);
-
-void iwl_trans_ref(struct iwl_trans *trans)
-{
- if (trans->ops->ref)
- trans->ops->ref(trans);
-}
-IWL_EXPORT_SYMBOL(iwl_trans_ref);
-
-void iwl_trans_unref(struct iwl_trans *trans)
-{
- if (trans->ops->unref)
- trans->ops->unref(trans);
-}
-IWL_EXPORT_SYMBOL(iwl_trans_unref);
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
index 0f8aeb1..a314081 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
@@ -159,13 +159,6 @@ static inline u32 iwl_rx_packet_payload_len(const struct iwl_rx_packet *pkt)
* @CMD_ASYNC: Return right away and don't wait for the response
* @CMD_WANT_SKB: Not valid with CMD_ASYNC. The caller needs the buffer of
* the response. The caller needs to call iwl_free_resp when done.
- * @CMD_HIGH_PRIO: The command is high priority - it goes to the front of the
- * command queue, but after other high priority commands. Valid only
- * with CMD_ASYNC.
- * @CMD_SEND_IN_IDLE: The command should be sent even when the trans is idle.
- * @CMD_MAKE_TRANS_IDLE: The command response should mark the trans as idle.
- * @CMD_WAKE_UP_TRANS: The command response should wake up the trans
- * (i.e. mark it as non-idle).
* @CMD_WANT_ASYNC_CALLBACK: the op_mode's async callback function must be
* called after this command completes. Valid only with CMD_ASYNC.
*/
@@ -173,11 +166,7 @@ enum CMD_MODE {
CMD_ASYNC = BIT(0),
CMD_WANT_SKB = BIT(1),
CMD_SEND_IN_RFKILL = BIT(2),
- CMD_HIGH_PRIO = BIT(3),
- CMD_SEND_IN_IDLE = BIT(4),
- CMD_MAKE_TRANS_IDLE = BIT(5),
- CMD_WAKE_UP_TRANS = BIT(6),
- CMD_WANT_ASYNC_CALLBACK = BIT(7),
+ CMD_WANT_ASYNC_CALLBACK = BIT(3),
};
#define DEF_CMD_PAYLOAD_SIZE 320
@@ -463,9 +452,8 @@ struct iwl_trans_rxq_dma_data {
*
* All the handlers MUST be implemented
*
- * @start_hw: starts the HW. If low_power is true, the NIC needs to be taken
- * out of a low power state. From that point on, the HW can send
- * interrupts. May sleep.
+ * @start_hw: starts the HW. From that point on, the HW can send interrupts.
+ * May sleep.
* @op_mode_leave: Turn off the HW RF kill indication if on
* May sleep
* @start_fw: allocates and inits all the resources for the transport
@@ -475,9 +463,8 @@ struct iwl_trans_rxq_dma_data {
* the SCD base address in SRAM, then provide it here, or 0 otherwise.
* May sleep
* @stop_device: stops the whole device (embedded CPU put to reset) and stops
- * the HW. If low_power is true, the NIC will be put in low power state.
- * From that point on, the HW will be stopped but will still issue an
- * interrupt if the HW RF kill switch is triggered.
+ * the HW. From that point on, the HW will be stopped but will still issue
+ * an interrupt if the HW RF kill switch is triggered.
* This callback must do the right thing and not crash even if %start_hw()
* was called but not &start_fw(). May sleep.
* @d3_suspend: put the device into the correct mode for WoWLAN during
@@ -535,11 +522,6 @@ struct iwl_trans_rxq_dma_data {
* @release_nic_access: let the NIC go to sleep. The "flags" parameter
* must be the same one that was sent before to the grab_nic_access.
* @set_bits_mask - set SRAM register according to value and mask.
- * @ref: grab a reference to the transport/FW layers, disallowing
- * certain low power states
- * @unref: release a reference previously taken with @ref. Note that
- * initially the reference count is 1, making an initial @unref
- * necessary to allow low power states.
* @dump_data: return a vmalloc'ed buffer with debug data, maybe containing last
* TX'ed commands and similar. The buffer will be vfree'd by the caller.
* Note that the transport must fill in the proper file headers.
@@ -548,14 +530,14 @@ struct iwl_trans_rxq_dma_data {
*/
struct iwl_trans_ops {
- int (*start_hw)(struct iwl_trans *iwl_trans, bool low_power);
+ int (*start_hw)(struct iwl_trans *iwl_trans);
void (*op_mode_leave)(struct iwl_trans *iwl_trans);
int (*start_fw)(struct iwl_trans *trans, const struct fw_img *fw,
bool run_in_rfkill);
void (*fw_alive)(struct iwl_trans *trans, u32 scd_addr);
- void (*stop_device)(struct iwl_trans *trans, bool low_power);
+ void (*stop_device)(struct iwl_trans *trans);
- void (*d3_suspend)(struct iwl_trans *trans, bool test, bool reset);
+ int (*d3_suspend)(struct iwl_trans *trans, bool test, bool reset);
int (*d3_resume)(struct iwl_trans *trans, enum iwl_d3_status *status,
bool test, bool reset);
@@ -566,6 +548,8 @@ struct iwl_trans_ops {
void (*reclaim)(struct iwl_trans *trans, int queue, int ssn,
struct sk_buff_head *skbs);
+ void (*set_q_ptrs)(struct iwl_trans *trans, int queue, int ptr);
+
bool (*txq_enable)(struct iwl_trans *trans, int queue, u16 ssn,
const struct iwl_trans_txq_scd_cfg *cfg,
unsigned int queue_wdg_timeout);
@@ -607,8 +591,6 @@ struct iwl_trans_ops {
unsigned long *flags);
void (*set_bits_mask)(struct iwl_trans *trans, u32 reg, u32 mask,
u32 value);
- void (*ref)(struct iwl_trans *trans);
- void (*unref)(struct iwl_trans *trans);
int (*suspend)(struct iwl_trans *trans);
void (*resume)(struct iwl_trans *trans);
@@ -632,9 +614,6 @@ enum iwl_trans_state {
/**
* DOC: Platform power management
*
- * There are two types of platform power management: system-wide
- * (WoWLAN) and runtime.
- *
* In system-wide power management the entire platform goes into a low
* power state (e.g. idle or suspend to RAM) at the same time and the
* device is configured as a wakeup source for the entire platform.
@@ -643,54 +622,46 @@ enum iwl_trans_state {
* put the platform in low power mode). The device's behavior in this
* mode is dictated by the wake-on-WLAN configuration.
*
- * In runtime power management, only the devices which are themselves
- * idle enter a low power state. This is done at runtime, which means
- * that the entire system is still running normally. This mode is
- * usually triggered automatically by the device driver and requires
- * the ability to enter and exit the low power modes in a very short
- * time, so there is not much impact in usability.
- *
* The terms used for the device's behavior are as follows:
*
* - D0: the device is fully powered and the host is awake;
* - D3: the device is in low power mode and only reacts to
* specific events (e.g. magic-packet received or scan
* results found);
- * - D0I3: the device is in low power mode and reacts to any
- * activity (e.g. RX);
*
* These terms reflect the power modes in the firmware and are not to
- * be confused with the physical device power state. The NIC can be
- * in D0I3 mode even if, for instance, the PCI device is in D3 state.
+ * be confused with the physical device power state.
*/
/**
* enum iwl_plat_pm_mode - platform power management mode
*
* This enumeration describes the device's platform power management
- * behavior when in idle mode (i.e. runtime power management) or when
- * in system-wide suspend (i.e WoWLAN).
+ * behavior when in system-wide suspend (i.e WoWLAN).
*
* @IWL_PLAT_PM_MODE_DISABLED: power management is disabled for this
- * device. At runtime, this means that nothing happens and the
- * device always remains in active. In system-wide suspend mode,
- * it means that the all connections will be closed automatically
- * by mac80211 before the platform is suspended.
+ * device. In system-wide suspend mode, it means that the all
+ * connections will be closed automatically by mac80211 before
+ * the platform is suspended.
* @IWL_PLAT_PM_MODE_D3: the device goes into D3 mode (i.e. WoWLAN).
- * For runtime power management, this mode is not officially
- * supported.
- * @IWL_PLAT_PM_MODE_D0I3: the device goes into D0I3 mode.
*/
enum iwl_plat_pm_mode {
IWL_PLAT_PM_MODE_DISABLED,
IWL_PLAT_PM_MODE_D3,
- IWL_PLAT_PM_MODE_D0I3,
};
-/* Max time to wait for trans to become idle/non-idle on d0i3
- * enter/exit (in msecs).
+/**
+ * enum iwl_ini_cfg_state
+ * @IWL_INI_CFG_STATE_NOT_LOADED: no debug cfg was given
+ * @IWL_INI_CFG_STATE_LOADED: debug cfg was found and loaded
+ * @IWL_INI_CFG_STATE_CORRUPTED: debug cfg was found and some of the TLVs
+ * are corrupted. The rest of the debug TLVs will still be used
*/
-#define IWL_TRANS_IDLE_TIMEOUT 2000
+enum iwl_ini_cfg_state {
+ IWL_INI_CFG_STATE_NOT_LOADED,
+ IWL_INI_CFG_STATE_LOADED,
+ IWL_INI_CFG_STATE_CORRUPTED,
+};
/* Max time to wait for nmi interrupt */
#define IWL_TRANS_NMI_TIMEOUT (HZ / 4)
@@ -733,8 +704,8 @@ struct iwl_self_init_dram {
* @umac_error_event_table: addr of umac error table
* @error_event_table_tlv_status: bitmap that indicates what error table
* pointers was recevied via TLV. uses enum &iwl_error_event_table_status
- * @external_ini_loaded: indicates if an external ini cfg was given
- * @ini_valid: indicates if debug ini mode is on
+ * @internal_ini_cfg: internal debug cfg state. Uses &enum iwl_ini_cfg_state
+ * @external_ini_cfg: external debug cfg state. Uses &enum iwl_ini_cfg_state
* @num_blocks: number of blocks in fw_mon
* @fw_mon: address of the buffers for firmware monitor
* @hw_error: equals true if hw error interrupt was received from the FW
@@ -752,14 +723,11 @@ struct iwl_trans_debug {
u32 umac_error_event_table;
unsigned int error_event_table_tlv_status;
- bool external_ini_loaded;
- bool ini_valid;
-
- struct iwl_apply_point_data apply_points[IWL_FW_INI_APPLY_NUM];
- struct iwl_apply_point_data apply_points_ext[IWL_FW_INI_APPLY_NUM];
+ enum iwl_ini_cfg_state internal_ini_cfg;
+ enum iwl_ini_cfg_state external_ini_cfg;
int num_blocks;
- struct iwl_dram_data fw_mon[IWL_FW_INI_APPLY_NUM];
+ struct iwl_dram_data fw_mon[IWL_FW_INI_ALLOCATION_NUM];
bool hw_error;
enum iwl_fw_ini_buffer_location ini_dest;
@@ -770,6 +738,7 @@ struct iwl_trans_debug {
*
* @ops - pointer to iwl_trans_ops
* @op_mode - pointer to the op_mode
+ * @trans_cfg: the trans-specific configuration part
* @cfg - pointer to the configuration
* @drv - pointer to iwl_drv
* @status: a bit-mask of transport status flags
@@ -797,13 +766,11 @@ struct iwl_trans_debug {
* @system_pm_mode: the system-wide power management mode in use.
* This mode is set dynamically, depending on the WoWLAN values
* configured from the userspace at runtime.
- * @runtime_pm_mode: the runtime power management mode in use. This
- * mode is set during the initialization phase and is not
- * supposed to change during runtime.
*/
struct iwl_trans {
const struct iwl_trans_ops *ops;
struct iwl_op_mode *op_mode;
+ const struct iwl_cfg_trans_params *trans_cfg;
const struct iwl_cfg *cfg;
struct iwl_drv *drv;
enum iwl_trans_state state;
@@ -844,8 +811,6 @@ struct iwl_trans {
struct iwl_self_init_dram init_dram;
enum iwl_plat_pm_mode system_pm_mode;
- enum iwl_plat_pm_mode runtime_pm_mode;
- bool suspending;
/* pointer to trans specific struct */
/*Ensure that this pointer will always be aligned to sizeof pointer */
@@ -864,16 +829,11 @@ static inline void iwl_trans_configure(struct iwl_trans *trans,
WARN_ON(iwl_cmd_groups_verify_sorted(trans_cfg));
}
-static inline int _iwl_trans_start_hw(struct iwl_trans *trans, bool low_power)
+static inline int iwl_trans_start_hw(struct iwl_trans *trans)
{
might_sleep();
- return trans->ops->start_hw(trans, low_power);
-}
-
-static inline int iwl_trans_start_hw(struct iwl_trans *trans)
-{
- return trans->ops->start_hw(trans, true);
+ return trans->ops->start_hw(trans);
}
static inline void iwl_trans_op_mode_leave(struct iwl_trans *trans)
@@ -909,27 +869,23 @@ static inline int iwl_trans_start_fw(struct iwl_trans *trans,
return trans->ops->start_fw(trans, fw, run_in_rfkill);
}
-static inline void _iwl_trans_stop_device(struct iwl_trans *trans,
- bool low_power)
+static inline void iwl_trans_stop_device(struct iwl_trans *trans)
{
might_sleep();
- trans->ops->stop_device(trans, low_power);
+ trans->ops->stop_device(trans);
trans->state = IWL_TRANS_NO_FW;
}
-static inline void iwl_trans_stop_device(struct iwl_trans *trans)
-{
- _iwl_trans_stop_device(trans, true);
-}
-
-static inline void iwl_trans_d3_suspend(struct iwl_trans *trans, bool test,
- bool reset)
+static inline int iwl_trans_d3_suspend(struct iwl_trans *trans, bool test,
+ bool reset)
{
might_sleep();
- if (trans->ops->d3_suspend)
- trans->ops->d3_suspend(trans, test, reset);
+ if (!trans->ops->d3_suspend)
+ return 0;
+
+ return trans->ops->d3_suspend(trans, test, reset);
}
static inline int iwl_trans_d3_resume(struct iwl_trans *trans,
@@ -1004,6 +960,17 @@ static inline void iwl_trans_reclaim(struct iwl_trans *trans, int queue,
trans->ops->reclaim(trans, queue, ssn, skbs);
}
+static inline void iwl_trans_set_q_ptrs(struct iwl_trans *trans, int queue,
+ int ptr)
+{
+ if (WARN_ON_ONCE(trans->state != IWL_TRANS_FW_ALIVE)) {
+ IWL_ERR(trans, "%s bad state = %d\n", __func__, trans->state);
+ return;
+ }
+
+ trans->ops->set_q_ptrs(trans, queue, ptr);
+}
+
static inline void iwl_trans_txq_disable(struct iwl_trans *trans, int queue,
bool configure_scd)
{
@@ -1261,16 +1228,19 @@ static inline void iwl_trans_sync_nmi(struct iwl_trans *trans)
trans->ops->sync_nmi(trans);
}
+static inline bool iwl_trans_dbg_ini_valid(struct iwl_trans *trans)
+{
+ return trans->dbg.internal_ini_cfg != IWL_INI_CFG_STATE_NOT_LOADED ||
+ trans->dbg.external_ini_cfg != IWL_INI_CFG_STATE_NOT_LOADED;
+}
+
/*****************************************************
* transport helper functions
*****************************************************/
struct iwl_trans *iwl_trans_alloc(unsigned int priv_size,
struct device *dev,
- const struct iwl_cfg *cfg,
const struct iwl_trans_ops *ops);
void iwl_trans_free(struct iwl_trans *trans);
-void iwl_trans_ref(struct iwl_trans *trans);
-void iwl_trans_unref(struct iwl_trans *trans);
/*****************************************************
* driver (transport) register/unregister functions
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
index 915b172..60aff2e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
@@ -153,5 +153,6 @@
#define IWL_MVM_FTM_INITIATOR_DYNACK true
#define IWL_MVM_D3_DEBUG false
#define IWL_MVM_USE_TWT false
+#define IWL_MVM_AMPDU_CONSEC_DROPS_DELBA 10
#endif /* __MVM_CONSTANTS_H */
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
index cec4085..86c2c58 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
@@ -735,40 +735,16 @@ iwl_mvm_get_wowlan_config(struct iwl_mvm *mvm,
return 0;
}
-static void
-iwl_mvm_iter_d0i3_ap_keys(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- void (*iter)(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif,
- struct ieee80211_sta *sta,
- struct ieee80211_key_conf *key,
- void *data),
- void *data)
-{
- struct ieee80211_sta *ap_sta;
-
- rcu_read_lock();
-
- ap_sta = rcu_dereference(mvm->fw_id_to_mac_id[mvm->d0i3_ap_sta_id]);
- if (IS_ERR_OR_NULL(ap_sta))
- goto out;
-
- ieee80211_iter_keys_rcu(mvm->hw, vif, iter, data);
-out:
- rcu_read_unlock();
-}
-
-int iwl_mvm_wowlan_config_key_params(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- bool d0i3,
- u32 cmd_flags)
+static int iwl_mvm_wowlan_config_key_params(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ u32 cmd_flags)
{
struct iwl_wowlan_kek_kck_material_cmd kek_kck_cmd = {};
struct iwl_wowlan_tkip_params_cmd tkip_cmd = {};
bool unified = fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_CNSLDTD_D3_D0_IMG);
struct wowlan_key_data key_data = {
- .configure_keys = !d0i3 && !unified,
+ .configure_keys = !unified,
.use_rsc_tsc = false,
.tkip = &tkip_cmd,
.use_tkip = false,
@@ -784,25 +760,16 @@ int iwl_mvm_wowlan_config_key_params(struct iwl_mvm *mvm,
* if we have to configure keys, call ieee80211_iter_keys(),
* as we need non-atomic context in order to take the
* required locks.
- * for the d0i3 we can't use ieee80211_iter_keys(), as
- * taking (almost) any mutex might result in deadlock.
*/
- if (!d0i3) {
- /*
- * Note that currently we don't propagate cmd_flags
- * to the iterator. In case of key_data.configure_keys,
- * all the configured commands are SYNC, and
- * iwl_mvm_wowlan_program_keys() will take care of
- * locking/unlocking mvm->mutex.
- */
- ieee80211_iter_keys(mvm->hw, vif,
- iwl_mvm_wowlan_program_keys,
- &key_data);
- } else {
- iwl_mvm_iter_d0i3_ap_keys(mvm, vif,
- iwl_mvm_wowlan_program_keys,
- &key_data);
- }
+ /*
+ * Note that currently we don't propagate cmd_flags
+ * to the iterator. In case of key_data.configure_keys,
+ * all the configured commands are SYNC, and
+ * iwl_mvm_wowlan_program_keys() will take care of
+ * locking/unlocking mvm->mutex.
+ */
+ ieee80211_iter_keys(mvm->hw, vif, iwl_mvm_wowlan_program_keys,
+ &key_data);
if (key_data.error) {
ret = -EIO;
@@ -830,7 +797,7 @@ int iwl_mvm_wowlan_config_key_params(struct iwl_mvm *mvm,
}
/* configure rekey data only if offloaded rekey is supported (d3) */
- if (mvmvif->rekey_data.valid && !d0i3) {
+ if (mvmvif->rekey_data.valid) {
memset(&kek_kck_cmd, 0, sizeof(kek_kck_cmd));
memcpy(kek_kck_cmd.kck, mvmvif->rekey_data.kck,
NL80211_KCK_LEN);
@@ -864,6 +831,8 @@ iwl_mvm_wowlan_config(struct iwl_mvm *mvm,
bool unified_image = fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_CNSLDTD_D3_D0_IMG);
+ mvm->offload_tid = wowlan_config_cmd->offloading_tid;
+
if (!unified_image) {
ret = iwl_mvm_switch_to_d3(mvm);
if (ret)
@@ -881,8 +850,7 @@ iwl_mvm_wowlan_config(struct iwl_mvm *mvm,
* that isn't really a problem though.
*/
mutex_unlock(&mvm->mutex);
- ret = iwl_mvm_wowlan_config_key_params(mvm, vif, false,
- CMD_ASYNC);
+ ret = iwl_mvm_wowlan_config_key_params(mvm, vif, CMD_ASYNC);
mutex_lock(&mvm->mutex);
if (ret)
return ret;
@@ -936,6 +904,8 @@ iwl_mvm_netdetect_config(struct iwl_mvm *mvm,
wowlan_config_cmd.wakeup_filter |=
cpu_to_le32(IWL_WOWLAN_WAKEUP_RF_KILL_DEASSERT);
+ wowlan_config_cmd.sta_id = mvm->aux_sta.sta_id;
+
ret = iwl_mvm_send_cmd_pdu(mvm, WOWLAN_CONFIGURATION, 0,
sizeof(wowlan_config_cmd),
&wowlan_config_cmd);
@@ -1043,6 +1013,8 @@ static int __iwl_mvm_suspend(struct ieee80211_hw *hw,
} else {
struct iwl_wowlan_config_cmd wowlan_config_cmd = {};
+ wowlan_config_cmd.sta_id = mvmvif->ap_sta_id;
+
ap_sta = rcu_dereference_protected(
mvm->fw_id_to_mac_id[mvmvif->ap_sta_id],
lockdep_is_held(&mvm->mutex));
@@ -1082,8 +1054,8 @@ static int __iwl_mvm_suspend(struct ieee80211_hw *hw,
* recording before entering D3. In later devices the FW stops the
* recording automatically.
*/
- if (mvm->trans->cfg->device_family < IWL_DEVICE_FAMILY_9000)
- iwl_fw_dbg_stop_recording(mvm->trans, NULL);
+ if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_9000)
+ iwl_fw_dbg_stop_restart_recording(&mvm->fwrt, NULL, true);
/* must be last -- this switches firmware state */
ret = iwl_mvm_send_cmd(mvm, &d3_cfg_cmd);
@@ -1100,13 +1072,12 @@ static int __iwl_mvm_suspend(struct ieee80211_hw *hw,
clear_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status);
- iwl_trans_d3_suspend(mvm->trans, test, !unified_image);
+ ret = iwl_trans_d3_suspend(mvm->trans, test, !unified_image);
out:
if (ret < 0) {
iwl_mvm_free_nd(mvm);
if (!unified_image) {
- iwl_mvm_ref(mvm, IWL_MVM_REF_UCODE_DOWN);
if (mvm->fw_restart > 0) {
mvm->fw_restart--;
ieee80211_restart_hw(mvm->hw);
@@ -1119,37 +1090,12 @@ static int __iwl_mvm_suspend(struct ieee80211_hw *hw,
return ret;
}
-static int iwl_mvm_enter_d0i3_sync(struct iwl_mvm *mvm)
-{
- struct iwl_notification_wait wait_d3;
- static const u16 d3_notif[] = { D3_CONFIG_CMD };
- int ret;
-
- iwl_init_notification_wait(&mvm->notif_wait, &wait_d3,
- d3_notif, ARRAY_SIZE(d3_notif),
- NULL, NULL);
-
- ret = iwl_mvm_enter_d0i3(mvm->hw->priv);
- if (ret)
- goto remove_notif;
-
- ret = iwl_wait_notification(&mvm->notif_wait, &wait_d3, HZ);
- WARN_ON_ONCE(ret);
- return ret;
-
-remove_notif:
- iwl_remove_notification(&mvm->notif_wait, &wait_d3);
- return ret;
-}
-
int iwl_mvm_suspend(struct ieee80211_hw *hw, struct cfg80211_wowlan *wowlan)
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
struct iwl_trans *trans = mvm->trans;
int ret;
- /* make sure the d0i3 exit work is not pending */
- flush_work(&mvm->d0i3_exit_work);
iwl_mvm_pause_tcm(mvm, true);
iwl_fw_runtime_suspend(&mvm->fwrt);
@@ -1158,25 +1104,6 @@ int iwl_mvm_suspend(struct ieee80211_hw *hw, struct cfg80211_wowlan *wowlan)
if (ret)
return ret;
- if (wowlan->any) {
- trans->system_pm_mode = IWL_PLAT_PM_MODE_D0I3;
-
- if (iwl_mvm_enter_d0i3_on_suspend(mvm)) {
- ret = iwl_mvm_enter_d0i3_sync(mvm);
-
- if (ret)
- return ret;
- }
-
- mutex_lock(&mvm->d0i3_suspend_mutex);
- __set_bit(D0I3_DEFER_WAKEUP, &mvm->d0i3_suspend_flags);
- mutex_unlock(&mvm->d0i3_suspend_mutex);
-
- iwl_trans_d3_suspend(trans, false, false);
-
- return 0;
- }
-
trans->system_pm_mode = IWL_PLAT_PM_MODE_D3;
return __iwl_mvm_suspend(hw, wowlan, false);
@@ -1735,6 +1662,13 @@ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm,
mvm_ap_sta->tid_data[i].seq_number = seq;
}
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000) {
+ i = mvm->offload_tid;
+ iwl_trans_set_q_ptrs(mvm->trans,
+ mvm_ap_sta->tid_data[i].txq_id,
+ mvm_ap_sta->tid_data[i].seq_number >> 4);
+ }
+
/* now we have all the data we need, unlock to avoid mac80211 issues */
mutex_unlock(&mvm->mutex);
@@ -1752,30 +1686,6 @@ static bool iwl_mvm_query_wakeup_reasons(struct iwl_mvm *mvm,
return false;
}
-void iwl_mvm_d0i3_update_keys(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- struct iwl_wowlan_status *status)
-{
- struct iwl_mvm_d3_gtk_iter_data gtkdata = {
- .mvm = mvm,
- .status = status,
- };
-
- /*
- * rekey handling requires taking locks that can't be taken now.
- * however, d0i3 doesn't offload rekey, so we're fine.
- */
- if (WARN_ON_ONCE(status->num_of_gtk_rekeys))
- return;
-
- /* find last GTK that we used initially, if any */
- gtkdata.find_phase = true;
- iwl_mvm_iter_d0i3_ap_keys(mvm, vif, iwl_mvm_d3_update_keys, >kdata);
-
- gtkdata.find_phase = false;
- iwl_mvm_iter_d0i3_ap_keys(mvm, vif, iwl_mvm_d3_update_keys, >kdata);
-}
-
#define ND_QUERY_BUF_LEN (sizeof(struct iwl_scan_offload_profile_match) * \
IWL_SCAN_MAX_PROFILES)
@@ -2024,15 +1934,6 @@ static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test)
if (IS_ERR_OR_NULL(vif))
goto err;
- ret = iwl_trans_d3_resume(mvm->trans, &d3_status, test, !unified_image);
- if (ret)
- goto err;
-
- if (d3_status != IWL_D3_STATUS_ALIVE) {
- IWL_INFO(mvm, "Device was reset during suspend\n");
- goto err;
- }
-
iwl_fw_dbg_read_d3_debug_data(&mvm->fwrt);
if (iwl_mvm_check_rt_status(mvm, vif)) {
@@ -2044,6 +1945,15 @@ static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test)
goto err;
}
+ ret = iwl_trans_d3_resume(mvm->trans, &d3_status, test, !unified_image);
+ if (ret)
+ goto err;
+
+ if (d3_status != IWL_D3_STATUS_ALIVE) {
+ IWL_INFO(mvm, "Device was reset during suspend\n");
+ goto err;
+ }
+
if (d0i3_first) {
ret = iwl_mvm_send_cmd_pdu(mvm, D0I3_END_CMD, 0, 0, NULL);
if (ret < 0) {
@@ -2059,6 +1969,9 @@ static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test)
*/
iwl_mvm_update_changed_regdom(mvm);
+ /* Re-configure PPAG settings */
+ iwl_mvm_ppag_send_cmd(mvm);
+
if (!unified_image)
/* Re-configure default SAR profile */
iwl_mvm_sar_select_profile(mvm, 1, 1);
@@ -2115,14 +2028,6 @@ static int __iwl_mvm_resume(struct iwl_mvm *mvm, bool test)
* 2. We are using a unified image but had an error while exiting D3
*/
set_bit(IWL_MVM_STATUS_HW_RESTART_REQUESTED, &mvm->status);
- /*
- * When switching images we return 1, which causes mac80211
- * to do a reconfig with IEEE80211_RECONFIG_TYPE_RESTART.
- * This type of reconfig calls iwl_mvm_restart_complete(),
- * where we unref the IWL_MVM_REF_UCODE_DOWN, so we need
- * to take the reference here.
- */
- iwl_mvm_ref(mvm, IWL_MVM_REF_UCODE_DOWN);
return 1;
}
@@ -2134,53 +2039,12 @@ static int iwl_mvm_resume_d3(struct iwl_mvm *mvm)
return __iwl_mvm_resume(mvm, false);
}
-static int iwl_mvm_resume_d0i3(struct iwl_mvm *mvm)
-{
- bool exit_now;
- enum iwl_d3_status d3_status;
- struct iwl_trans *trans = mvm->trans;
-
- iwl_trans_d3_resume(trans, &d3_status, false, false);
-
- /*
- * make sure to clear D0I3_DEFER_WAKEUP before
- * calling iwl_trans_resume(), which might wait
- * for d0i3 exit completion.
- */
- mutex_lock(&mvm->d0i3_suspend_mutex);
- __clear_bit(D0I3_DEFER_WAKEUP, &mvm->d0i3_suspend_flags);
- exit_now = __test_and_clear_bit(D0I3_PENDING_WAKEUP,
- &mvm->d0i3_suspend_flags);
- mutex_unlock(&mvm->d0i3_suspend_mutex);
- if (exit_now) {
- IWL_DEBUG_RPM(mvm, "Run deferred d0i3 exit\n");
- _iwl_mvm_exit_d0i3(mvm);
- }
-
- iwl_trans_resume(trans);
-
- if (iwl_mvm_enter_d0i3_on_suspend(mvm)) {
- int ret = iwl_mvm_exit_d0i3(mvm->hw->priv);
-
- if (ret)
- return ret;
- /*
- * d0i3 exit will be deferred until reconfig_complete.
- * make sure there we are out of d0i3.
- */
- }
- return 0;
-}
-
int iwl_mvm_resume(struct ieee80211_hw *hw)
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
int ret;
- if (mvm->trans->system_pm_mode == IWL_PLAT_PM_MODE_D0I3)
- ret = iwl_mvm_resume_d0i3(mvm);
- else
- ret = iwl_mvm_resume_d3(mvm);
+ ret = iwl_mvm_resume_d3(mvm);
mvm->trans->system_pm_mode = IWL_PLAT_PM_MODE_DISABLED;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
index 0c188a8..ad18c2f 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
@@ -1056,19 +1056,11 @@ static ssize_t iwl_dbgfs_fw_restart_write(struct iwl_mvm *mvm, char *buf,
static ssize_t iwl_dbgfs_fw_nmi_write(struct iwl_mvm *mvm, char *buf,
size_t count, loff_t *ppos)
{
- int ret;
-
if (!iwl_mvm_firmware_running(mvm))
return -EIO;
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_NMI);
- if (ret)
- return ret;
-
iwl_force_nmi(mvm->trans);
- iwl_mvm_unref(mvm, IWL_MVM_REF_NMI);
-
return count;
}
@@ -1181,8 +1173,8 @@ static ssize_t iwl_dbgfs_inject_packet_write(struct iwl_mvm *mvm,
struct iwl_rx_mpdu_desc *desc;
int bin_len = count / 2;
int ret = -EINVAL;
- size_t mpdu_cmd_hdr_size =
- (mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) ?
+ size_t mpdu_cmd_hdr_size = (mvm->trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_22560) ?
sizeof(struct iwl_rx_mpdu_desc) :
IWL_RX_DESC_SIZE_V1;
@@ -1190,7 +1182,7 @@ static ssize_t iwl_dbgfs_inject_packet_write(struct iwl_mvm *mvm,
return -EIO;
/* supporting only 9000 descriptor */
- if (!mvm->trans->cfg->mq_rx_supported)
+ if (!mvm->trans->trans_cfg->mq_rx_supported)
return -ENOTSUPP;
rxb._page = alloc_pages(GFP_ATOMIC, 0);
@@ -1380,19 +1372,12 @@ static ssize_t iwl_dbgfs_fw_dbg_collect_write(struct iwl_mvm *mvm,
char *buf, size_t count,
loff_t *ppos)
{
- int ret;
-
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PRPH_WRITE);
- if (ret)
- return ret;
if (count == 0)
return 0;
iwl_fw_dbg_collect(&mvm->fwrt, FW_DBG_TRIGGER_USER, buf,
(count - 1), NULL);
- iwl_mvm_unref(mvm, IWL_MVM_REF_PRPH_WRITE);
-
return count;
}
@@ -1579,87 +1564,6 @@ static ssize_t iwl_dbgfs_bcast_filters_macs_write(struct iwl_mvm *mvm,
}
#endif
-#define PRINT_MVM_REF(ref) do { \
- if (mvm->refs[ref]) \
- pos += scnprintf(buf + pos, bufsz - pos, \
- "\t(0x%lx): %d %s\n", \
- BIT(ref), mvm->refs[ref], #ref); \
-} while (0)
-
-static ssize_t iwl_dbgfs_d0i3_refs_read(struct file *file,
- char __user *user_buf,
- size_t count, loff_t *ppos)
-{
- struct iwl_mvm *mvm = file->private_data;
- int i, pos = 0;
- char buf[256];
- const size_t bufsz = sizeof(buf);
- u32 refs = 0;
-
- for (i = 0; i < IWL_MVM_REF_COUNT; i++)
- if (mvm->refs[i])
- refs |= BIT(i);
-
- pos += scnprintf(buf + pos, bufsz - pos, "taken mvm refs: 0x%x\n",
- refs);
-
- PRINT_MVM_REF(IWL_MVM_REF_UCODE_DOWN);
- PRINT_MVM_REF(IWL_MVM_REF_SCAN);
- PRINT_MVM_REF(IWL_MVM_REF_ROC);
- PRINT_MVM_REF(IWL_MVM_REF_ROC_AUX);
- PRINT_MVM_REF(IWL_MVM_REF_P2P_CLIENT);
- PRINT_MVM_REF(IWL_MVM_REF_AP_IBSS);
- PRINT_MVM_REF(IWL_MVM_REF_USER);
- PRINT_MVM_REF(IWL_MVM_REF_TX);
- PRINT_MVM_REF(IWL_MVM_REF_TX_AGG);
- PRINT_MVM_REF(IWL_MVM_REF_ADD_IF);
- PRINT_MVM_REF(IWL_MVM_REF_START_AP);
- PRINT_MVM_REF(IWL_MVM_REF_BSS_CHANGED);
- PRINT_MVM_REF(IWL_MVM_REF_PREPARE_TX);
- PRINT_MVM_REF(IWL_MVM_REF_PROTECT_TDLS);
- PRINT_MVM_REF(IWL_MVM_REF_CHECK_CTKILL);
- PRINT_MVM_REF(IWL_MVM_REF_PRPH_READ);
- PRINT_MVM_REF(IWL_MVM_REF_PRPH_WRITE);
- PRINT_MVM_REF(IWL_MVM_REF_NMI);
- PRINT_MVM_REF(IWL_MVM_REF_TM_CMD);
- PRINT_MVM_REF(IWL_MVM_REF_EXIT_WORK);
- PRINT_MVM_REF(IWL_MVM_REF_PROTECT_CSA);
- PRINT_MVM_REF(IWL_MVM_REF_FW_DBG_COLLECT);
- PRINT_MVM_REF(IWL_MVM_REF_INIT_UCODE);
- PRINT_MVM_REF(IWL_MVM_REF_SENDING_CMD);
- PRINT_MVM_REF(IWL_MVM_REF_RX);
-
- return simple_read_from_buffer(user_buf, count, ppos, buf, pos);
-}
-
-static ssize_t iwl_dbgfs_d0i3_refs_write(struct iwl_mvm *mvm, char *buf,
- size_t count, loff_t *ppos)
-{
- unsigned long value;
- int ret;
- bool taken;
-
- ret = kstrtoul(buf, 10, &value);
- if (ret < 0)
- return ret;
-
- mutex_lock(&mvm->mutex);
-
- taken = mvm->refs[IWL_MVM_REF_USER];
- if (value == 1 && !taken)
- iwl_mvm_ref(mvm, IWL_MVM_REF_USER);
- else if (value == 0 && taken)
- iwl_mvm_unref(mvm, IWL_MVM_REF_USER);
- else
- ret = -EINVAL;
-
- mutex_unlock(&mvm->mutex);
-
- if (ret < 0)
- return ret;
- return count;
-}
-
#define MVM_DEBUGFS_WRITE_FILE_OPS(name, bufsz) \
_MVM_DEBUGFS_WRITE_FILE_OPS(name, bufsz, struct iwl_mvm)
#define MVM_DEBUGFS_READ_WRITE_FILE_OPS(name, bufsz) \
@@ -1692,21 +1596,14 @@ iwl_dbgfs_prph_reg_read(struct file *file,
int pos = 0;
char buf[32];
const size_t bufsz = sizeof(buf);
- int ret;
if (!mvm->dbgfs_prph_reg_addr)
return -EINVAL;
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PRPH_READ);
- if (ret)
- return ret;
-
pos += scnprintf(buf + pos, bufsz - pos, "Reg 0x%x: (0x%x)\n",
mvm->dbgfs_prph_reg_addr,
iwl_read_prph(mvm->trans, mvm->dbgfs_prph_reg_addr));
- iwl_mvm_unref(mvm, IWL_MVM_REF_PRPH_READ);
-
return simple_read_from_buffer(user_buf, count, ppos, buf, pos);
}
@@ -1716,7 +1613,6 @@ iwl_dbgfs_prph_reg_write(struct iwl_mvm *mvm, char *buf,
{
u8 args;
u32 value;
- int ret;
args = sscanf(buf, "%i %i", &mvm->dbgfs_prph_reg_addr, &value);
/* if we only want to set the reg address - nothing more to do */
@@ -1727,13 +1623,8 @@ iwl_dbgfs_prph_reg_write(struct iwl_mvm *mvm, char *buf,
if (args != 2)
return -EINVAL;
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PRPH_WRITE);
- if (ret)
- return ret;
-
iwl_write_prph(mvm->trans, mvm->dbgfs_prph_reg_addr, value);
- iwl_mvm_unref(mvm, IWL_MVM_REF_PRPH_WRITE);
out:
return count;
}
@@ -1867,6 +1758,38 @@ iwl_dbgfs_uapsd_noagg_bssids_read(struct file *file, char __user *user_buf,
return simple_read_from_buffer(user_buf, count, ppos, buf, pos);
}
+static ssize_t
+iwl_dbgfs_ltr_config_write(struct iwl_mvm *mvm,
+ char *buf, size_t count, loff_t *ppos)
+{
+ int ret;
+ struct iwl_ltr_config_cmd ltr_config = {0};
+
+ if (!iwl_mvm_firmware_running(mvm))
+ return -EIO;
+
+ if (sscanf(buf, "%x,%x,%x,%x,%x,%x,%x",
+ <r_config.flags,
+ <r_config.static_long,
+ <r_config.static_short,
+ <r_config.ltr_cfg_values[0],
+ <r_config.ltr_cfg_values[1],
+ <r_config.ltr_cfg_values[2],
+ <r_config.ltr_cfg_values[3]) != 7) {
+ return -EINVAL;
+ }
+
+ mutex_lock(&mvm->mutex);
+ ret = iwl_mvm_send_cmd_pdu(mvm, LTR_CONFIG, 0, sizeof(ltr_config),
+ <r_config);
+ mutex_unlock(&mvm->mutex);
+
+ if (ret)
+ IWL_ERR(mvm, "failed to send ltr configuration cmd\n");
+
+ return ret ?: count;
+}
+
MVM_DEBUGFS_READ_WRITE_FILE_OPS(prph_reg, 64);
/* Device wide debugfs entries */
@@ -1892,7 +1815,6 @@ MVM_DEBUGFS_WRITE_FILE_OPS(fw_nmi, 10);
MVM_DEBUGFS_WRITE_FILE_OPS(bt_tx_prio, 10);
MVM_DEBUGFS_WRITE_FILE_OPS(bt_force_ant, 10);
MVM_DEBUGFS_READ_WRITE_FILE_OPS(scan_ant_rxchain, 8);
-MVM_DEBUGFS_READ_WRITE_FILE_OPS(d0i3_refs, 8);
MVM_DEBUGFS_READ_WRITE_FILE_OPS(fw_dbg_conf, 8);
MVM_DEBUGFS_WRITE_FILE_OPS(fw_dbg_collect, 64);
MVM_DEBUGFS_WRITE_FILE_OPS(indirection_tbl,
@@ -1916,6 +1838,8 @@ MVM_DEBUGFS_READ_WRITE_STA_FILE_OPS(amsdu_len, 16);
MVM_DEBUGFS_READ_WRITE_FILE_OPS(he_sniffer_params, 32);
+MVM_DEBUGFS_WRITE_FILE_OPS(ltr_config, 512);
+
static ssize_t iwl_dbgfs_mem_read(struct file *file, char __user *user_buf,
size_t count, loff_t *ppos)
{
@@ -2091,7 +2015,6 @@ void iwl_mvm_dbgfs_register(struct iwl_mvm *mvm, struct dentry *dbgfs_dir)
MVM_DEBUGFS_ADD_FILE(bt_force_ant, mvm->debugfs_dir, 0200);
MVM_DEBUGFS_ADD_FILE(scan_ant_rxchain, mvm->debugfs_dir, 0600);
MVM_DEBUGFS_ADD_FILE(prph_reg, mvm->debugfs_dir, 0600);
- MVM_DEBUGFS_ADD_FILE(d0i3_refs, mvm->debugfs_dir, 0600);
MVM_DEBUGFS_ADD_FILE(fw_dbg_conf, mvm->debugfs_dir, 0600);
MVM_DEBUGFS_ADD_FILE(fw_dbg_collect, mvm->debugfs_dir, 0200);
MVM_DEBUGFS_ADD_FILE(send_echo_cmd, mvm->debugfs_dir, 0200);
@@ -2104,6 +2027,9 @@ void iwl_mvm_dbgfs_register(struct iwl_mvm *mvm, struct dentry *dbgfs_dir)
#endif
MVM_DEBUGFS_ADD_FILE(he_sniffer_params, mvm->debugfs_dir, 0600);
+ if (fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_SET_LTR_GEN2))
+ MVM_DEBUGFS_ADD_FILE(ltr_config, mvm->debugfs_dir, 0200);
+
debugfs_create_bool("enable_scan_iteration_notif", 0600,
mvm->debugfs_dir, &mvm->scan_iter_notif_enabled);
debugfs_create_bool("drop_bcn_ap_mode", 0600, mvm->debugfs_dir,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
index 5de54d1..014eca6 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
@@ -357,13 +357,14 @@ static int iwl_mvm_load_ucode_wait_alive(struct iwl_mvm *mvm,
iwl_fw_dbg_error_collect(&mvm->fwrt,
FW_DBG_TRIGGER_ALIVE_TIMEOUT);
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000)
IWL_ERR(mvm,
"SecBoot CPU1 Status: 0x%x, CPU2 Status: 0x%x\n",
iwl_read_umac_prph(trans, UMAG_SB_CPU_1_STATUS),
iwl_read_umac_prph(trans,
UMAG_SB_CPU_2_STATUS));
- else if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_8000)
+ else if (trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_8000)
IWL_ERR(mvm,
"SecBoot CPU1 Status: 0x%x, CPU2 Status: 0x%x\n",
iwl_read_prph(trans, SB_CPU_1_STATUS),
@@ -430,7 +431,7 @@ static int iwl_run_unified_mvm_ucode(struct iwl_mvm *mvm, bool read_nvm)
iwl_wait_init_complete,
NULL);
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_EARLY);
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_EARLY, NULL);
/* Will also start the device */
ret = iwl_mvm_load_ucode_wait_alive(mvm, IWL_UCODE_REGULAR);
@@ -438,7 +439,8 @@ static int iwl_run_unified_mvm_ucode(struct iwl_mvm *mvm, bool read_nvm)
IWL_ERR(mvm, "Failed to start RT ucode: %d\n", ret);
goto error;
}
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_AFTER_ALIVE);
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_AFTER_ALIVE,
+ NULL);
/* Send init config command to mark that we are sending NVM access
* commands
@@ -557,7 +559,7 @@ int iwl_run_init_mvm_ucode(struct iwl_mvm *mvm, bool read_nvm)
goto remove_notif;
}
- if (mvm->cfg->device_family < IWL_DEVICE_FAMILY_8000) {
+ if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_8000) {
ret = iwl_mvm_send_bt_init_conf(mvm);
if (ret)
goto remove_notif;
@@ -1002,6 +1004,113 @@ static int iwl_mvm_sar_geo_init(struct iwl_mvm *mvm)
return iwl_mvm_send_cmd_pdu(mvm, cmd_wide_id, 0, sizeof(cmd), &cmd);
}
+static int iwl_mvm_get_ppag_table(struct iwl_mvm *mvm)
+{
+ union acpi_object *wifi_pkg, *data, *enabled;
+ int i, j, ret, tbl_rev;
+ int idx = 2;
+
+ mvm->ppag_table.enabled = cpu_to_le32(0);
+ data = iwl_acpi_get_object(mvm->dev, ACPI_PPAG_METHOD);
+ if (IS_ERR(data))
+ return PTR_ERR(data);
+
+ wifi_pkg = iwl_acpi_get_wifi_pkg(mvm->dev, data,
+ ACPI_PPAG_WIFI_DATA_SIZE, &tbl_rev);
+
+ if (IS_ERR(wifi_pkg) || tbl_rev != 0) {
+ ret = PTR_ERR(wifi_pkg);
+ goto out_free;
+ }
+
+ enabled = &wifi_pkg->package.elements[1];
+ if (enabled->type != ACPI_TYPE_INTEGER ||
+ (enabled->integer.value != 0 && enabled->integer.value != 1)) {
+ ret = -EINVAL;
+ goto out_free;
+ }
+
+ mvm->ppag_table.enabled = cpu_to_le32(enabled->integer.value);
+ if (!mvm->ppag_table.enabled) {
+ ret = 0;
+ goto out_free;
+ }
+
+ /*
+ * read, verify gain values and save them into the PPAG table.
+ * first sub-band (j=0) corresponds to Low-Band (2.4GHz), and the
+ * following sub-bands to High-Band (5GHz).
+ */
+ for (i = 0; i < ACPI_PPAG_NUM_CHAINS; i++) {
+ for (j = 0; j < ACPI_PPAG_NUM_SUB_BANDS; j++) {
+ union acpi_object *ent;
+
+ ent = &wifi_pkg->package.elements[idx++];
+ if (ent->type != ACPI_TYPE_INTEGER ||
+ (j == 0 && ent->integer.value > ACPI_PPAG_MAX_LB) ||
+ (j == 0 && ent->integer.value < ACPI_PPAG_MIN_LB) ||
+ (j != 0 && ent->integer.value > ACPI_PPAG_MAX_HB) ||
+ (j != 0 && ent->integer.value < ACPI_PPAG_MIN_HB)) {
+ mvm->ppag_table.enabled = cpu_to_le32(0);
+ ret = -EINVAL;
+ goto out_free;
+ }
+ mvm->ppag_table.gain[i][j] = ent->integer.value;
+ }
+ }
+ ret = 0;
+out_free:
+ kfree(data);
+ return ret;
+}
+
+int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm)
+{
+ int i, j, ret;
+
+ if (!fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_SET_PPAG)) {
+ IWL_DEBUG_RADIO(mvm,
+ "PPAG capability not supported by FW, command not sent.\n");
+ return 0;
+ }
+
+ IWL_DEBUG_RADIO(mvm, "Sending PER_PLATFORM_ANT_GAIN_CMD\n");
+ IWL_DEBUG_RADIO(mvm, "PPAG is %s\n",
+ mvm->ppag_table.enabled ? "enabled" : "disabled");
+
+ for (i = 0; i < ACPI_PPAG_NUM_CHAINS; i++) {
+ for (j = 0; j < ACPI_PPAG_NUM_SUB_BANDS; j++) {
+ IWL_DEBUG_RADIO(mvm,
+ "PPAG table: chain[%d] band[%d]: gain = %d\n",
+ i, j, mvm->ppag_table.gain[i][j]);
+ }
+ }
+
+ ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(PHY_OPS_GROUP,
+ PER_PLATFORM_ANT_GAIN_CMD),
+ 0, sizeof(mvm->ppag_table),
+ &mvm->ppag_table);
+ if (ret < 0)
+ IWL_ERR(mvm, "failed to send PER_PLATFORM_ANT_GAIN_CMD (%d)\n",
+ ret);
+
+ return ret;
+}
+
+static int iwl_mvm_ppag_init(struct iwl_mvm *mvm)
+{
+ int ret;
+
+ ret = iwl_mvm_get_ppag_table(mvm);
+ if (ret < 0) {
+ IWL_DEBUG_RADIO(mvm,
+ "PPAG BIOS table invalid or unavailable. (%d)\n",
+ ret);
+ return 0;
+ }
+ return iwl_mvm_ppag_send_cmd(mvm);
+}
+
#else /* CONFIG_ACPI */
static int iwl_mvm_sar_get_wrds_table(struct iwl_mvm *mvm)
{
@@ -1033,6 +1142,16 @@ int iwl_mvm_get_sar_geo_profile(struct iwl_mvm *mvm)
{
return -ENOENT;
}
+
+int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm)
+{
+ return -ENOENT;
+}
+
+static int iwl_mvm_ppag_init(struct iwl_mvm *mvm)
+{
+ return -ENOENT;
+}
#endif /* CONFIG_ACPI */
void iwl_mvm_send_recovery_cmd(struct iwl_mvm *mvm, u32 flags)
@@ -1140,17 +1259,13 @@ static int iwl_mvm_load_rt_fw(struct iwl_mvm *mvm)
return ret;
}
- /*
- * Stop and start the transport without entering low power
- * mode. This will save the state of other components on the
- * device that are triggered by the INIT firwmare (MFUART).
- */
- _iwl_trans_stop_device(mvm->trans, false);
- ret = _iwl_trans_start_hw(mvm->trans, false);
+ iwl_fw_dbg_stop_sync(&mvm->fwrt);
+ iwl_trans_stop_device(mvm->trans);
+ ret = iwl_trans_start_hw(mvm->trans);
if (ret)
return ret;
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_EARLY);
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_EARLY, NULL);
mvm->rfkill_safe_init_done = false;
ret = iwl_mvm_load_ucode_wait_alive(mvm, IWL_UCODE_REGULAR);
@@ -1159,7 +1274,8 @@ static int iwl_mvm_load_rt_fw(struct iwl_mvm *mvm)
mvm->rfkill_safe_init_done = true;
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_AFTER_ALIVE);
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_AFTER_ALIVE,
+ NULL);
return iwl_init_paging(&mvm->fwrt, mvm->fwrt.cur_fw_img);
}
@@ -1169,6 +1285,7 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
int ret, i;
struct ieee80211_channel *chan;
struct cfg80211_chan_def chandef;
+ struct ieee80211_supported_band *sband = NULL;
lockdep_assert_held(&mvm->mutex);
@@ -1191,7 +1308,7 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
if (ret)
IWL_ERR(mvm, "Failed to initialize Smart Fifo\n");
- if (!mvm->trans->dbg.ini_valid) {
+ if (!iwl_trans_dbg_ini_valid(mvm->trans)) {
mvm->fwrt.dump.conf = FW_DBG_INVALID;
/* if we have a destination, assume EARLY START */
if (mvm->fw->dbg.dest_tlv)
@@ -1219,7 +1336,7 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
goto error;
/* Init RSS configuration */
- if (mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000) {
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000) {
ret = iwl_configure_rxq(mvm);
if (ret) {
IWL_ERR(mvm, "Failed to configure RX queues: %d\n",
@@ -1246,9 +1363,11 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
/* reset quota debouncing buffer - 0xff will yield invalid data */
memset(&mvm->last_quota_cmd, 0xff, sizeof(mvm->last_quota_cmd));
- ret = iwl_mvm_send_dqa_cmd(mvm);
- if (ret)
- goto error;
+ if (fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_DQA_SUPPORT)) {
+ ret = iwl_mvm_send_dqa_cmd(mvm);
+ if (ret)
+ goto error;
+ }
/* Add auxiliary station for scanning */
ret = iwl_mvm_add_aux_sta(mvm);
@@ -1256,7 +1375,15 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
goto error;
/* Add all the PHY contexts */
- chan = &mvm->hw->wiphy->bands[NL80211_BAND_2GHZ]->channels[0];
+ i = 0;
+ while (!sband && i < NUM_NL80211_BANDS)
+ sband = mvm->hw->wiphy->bands[i++];
+
+ if (WARN_ON_ONCE(!sband))
+ goto error;
+
+ chan = &sband->channels[0];
+
cfg80211_chandef_create(&chandef, chan, NL80211_CHAN_NO_HT);
for (i = 0; i < NUM_PHY_CTX; i++) {
/*
@@ -1270,7 +1397,6 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
goto error;
}
-#ifdef CONFIG_THERMAL
if (iwl_mvm_is_tt_in_fw(mvm)) {
/* in order to give the responsibility of ct-kill and
* TX backoff to FW we need to send empty temperature reporting
@@ -1282,6 +1408,7 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
iwl_mvm_tt_tx_backoff(mvm, 0);
}
+#ifdef CONFIG_THERMAL
/* TODO: read the budget from BIOS / Platform NVM */
/*
@@ -1294,12 +1421,10 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
if (ret)
goto error;
}
-#else
- /* Initialize tx backoffs to the minimal possible */
- iwl_mvm_tt_tx_backoff(mvm, 0);
#endif
- WARN_ON(iwl_mvm_config_ltr(mvm));
+ if (!fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_SET_LTR_GEN2))
+ WARN_ON(iwl_mvm_config_ltr(mvm));
ret = iwl_mvm_power_update_device(mvm);
if (ret)
@@ -1323,16 +1448,16 @@ int iwl_mvm_up(struct iwl_mvm *mvm)
goto error;
}
- /* allow FW/transport low power modes if not during restart */
- if (!test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status))
- iwl_mvm_unref(mvm, IWL_MVM_REF_UCODE_DOWN);
-
if (test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status))
iwl_mvm_send_recovery_cmd(mvm, ERROR_RECOVERY_UPDATE_DB);
if (iwl_acpi_get_eckv(mvm->dev, &mvm->ext_clock_valid))
IWL_DEBUG_INFO(mvm, "ECKV table doesn't exist in BIOS\n");
+ ret = iwl_mvm_ppag_init(mvm);
+ if (ret)
+ goto error;
+
ret = iwl_mvm_sar_init(mvm);
if (ret == 0) {
ret = iwl_mvm_sar_geo_init(mvm);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/led.c b/drivers/net/wireless/intel/iwlwifi/mvm/led.c
index 4348bb0..d104da9 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/led.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/led.c
@@ -7,7 +7,7 @@
*
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -29,7 +29,7 @@
*
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2017 Intel Deutschland GmbH
- * Copyright(c) 2018 Intel Corporation
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -156,7 +156,7 @@ void iwl_mvm_leds_sync(struct iwl_mvm *mvm)
* if we control through the register, we're doing it
* even when the firmware isn't up, so no need to sync
*/
- if (mvm->cfg->device_family < IWL_DEVICE_FAMILY_8000)
+ if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_8000)
return;
iwl_mvm_led_set(mvm, mvm->led.brightness > 0);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
index cb22d44..9c417dd 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
@@ -554,7 +554,7 @@ static void iwl_mvm_mac_ctxt_cmd_common(struct iwl_mvm *mvm,
cpu_to_le32(vif->bss_conf.use_short_slot ?
MAC_FLG_SHORT_SLOT : 0);
- cmd->filter_flags = cpu_to_le32(MAC_FILTER_ACCEPT_GRP);
+ cmd->filter_flags = 0;
for (i = 0; i < IEEE80211_NUM_ACS; i++) {
u8 txf = iwl_mvm_mac_ac_to_tx_fifo(mvm, i);
@@ -623,6 +623,8 @@ static int iwl_mvm_mac_ctxt_cmd_sta(struct iwl_mvm *mvm,
/* We need the dtim_period to set the MAC as associated */
if (vif->bss_conf.assoc && vif->bss_conf.dtim_period &&
!force_assoc_off) {
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ u8 ap_sta_id = mvmvif->ap_sta_id;
u32 dtim_offs;
/*
@@ -658,6 +660,29 @@ static int iwl_mvm_mac_ctxt_cmd_sta(struct iwl_mvm *mvm,
dtim_offs);
ctxt_sta->is_assoc = cpu_to_le32(1);
+
+ /*
+ * allow multicast data frames only as long as the station is
+ * authorized, i.e., GTK keys are already installed (if needed)
+ */
+ if (ap_sta_id < IWL_MVM_STATION_COUNT) {
+ struct ieee80211_sta *sta;
+
+ rcu_read_lock();
+
+ sta = rcu_dereference(mvm->fw_id_to_mac_id[ap_sta_id]);
+ if (!IS_ERR_OR_NULL(sta)) {
+ struct iwl_mvm_sta *mvmsta =
+ iwl_mvm_sta_from_mac80211(sta);
+
+ if (mvmsta->sta_state ==
+ IEEE80211_STA_AUTHORIZED)
+ cmd.filter_flags |=
+ cpu_to_le32(MAC_FILTER_ACCEPT_GRP);
+ }
+
+ rcu_read_unlock();
+ }
} else {
ctxt_sta->is_assoc = cpu_to_le32(0);
@@ -703,7 +728,8 @@ static int iwl_mvm_mac_ctxt_cmd_listener(struct iwl_mvm *mvm,
MAC_FILTER_IN_CONTROL_AND_MGMT |
MAC_FILTER_IN_BEACON |
MAC_FILTER_IN_PROBE_REQUEST |
- MAC_FILTER_IN_CRC32);
+ MAC_FILTER_IN_CRC32 |
+ MAC_FILTER_ACCEPT_GRP);
ieee80211_hw_set(mvm->hw, RX_INCLUDES_FCS);
/* Allocate sniffer station */
@@ -727,7 +753,8 @@ static int iwl_mvm_mac_ctxt_cmd_ibss(struct iwl_mvm *mvm,
iwl_mvm_mac_ctxt_cmd_common(mvm, vif, &cmd, NULL, action);
cmd.filter_flags = cpu_to_le32(MAC_FILTER_IN_BEACON |
- MAC_FILTER_IN_PROBE_REQUEST);
+ MAC_FILTER_IN_PROBE_REQUEST |
+ MAC_FILTER_ACCEPT_GRP);
/* cmd.ibss.beacon_time/cmd.ibss.beacon_tsf are curently ignored */
cmd.ibss.bi = cpu_to_le32(vif->bss_conf.beacon_int);
@@ -1404,6 +1431,9 @@ void iwl_mvm_rx_missed_beacons_notif(struct iwl_mvm *mvm,
else if (rx_missed_bcon_since_rx > IWL_MVM_MISSED_BEACONS_THRESHOLD)
ieee80211_beacon_loss(vif);
+ iwl_dbg_tlv_time_point(&mvm->fwrt,
+ IWL_FW_INI_TIME_POINT_MISSED_BEACONS, NULL);
+
trigger = iwl_fw_dbg_trigger_on(&mvm->fwrt, ieee80211_vif_to_wdev(vif),
FW_DBG_TRIGGER_MISSED_BEACONS);
if (!trigger)
@@ -1420,8 +1450,6 @@ void iwl_mvm_rx_missed_beacons_notif(struct iwl_mvm *mvm,
rx_missed_bcon >= stop_trig_missed_bcon)
iwl_fw_dbg_collect_trig(&mvm->fwrt, trigger, NULL);
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_MISSED_BEACONS);
-
out:
rcu_read_unlock();
}
@@ -1567,7 +1595,9 @@ void iwl_mvm_channel_switch_noa_notif(struct iwl_mvm *mvm,
RCU_INIT_POINTER(mvm->csa_vif, NULL);
return;
case NL80211_IFTYPE_STATION:
- iwl_mvm_csa_client_absent(mvm, vif);
+ if (!fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD))
+ iwl_mvm_csa_client_absent(mvm, vif);
cancel_delayed_work(&mvmvif->csa_work);
ieee80211_chswitch_done(vif, true);
break;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
index b74bd58..cd1b100 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
@@ -213,91 +213,6 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
struct ieee80211_sta *sta,
struct ieee80211_key_conf *key);
-void iwl_mvm_ref(struct iwl_mvm *mvm, enum iwl_mvm_ref_type ref_type)
-{
- if (!iwl_mvm_is_d0i3_supported(mvm))
- return;
-
- IWL_DEBUG_RPM(mvm, "Take mvm reference - type %d\n", ref_type);
- spin_lock_bh(&mvm->refs_lock);
- mvm->refs[ref_type]++;
- spin_unlock_bh(&mvm->refs_lock);
- iwl_trans_ref(mvm->trans);
-}
-
-void iwl_mvm_unref(struct iwl_mvm *mvm, enum iwl_mvm_ref_type ref_type)
-{
- if (!iwl_mvm_is_d0i3_supported(mvm))
- return;
-
- IWL_DEBUG_RPM(mvm, "Leave mvm reference - type %d\n", ref_type);
- spin_lock_bh(&mvm->refs_lock);
- if (WARN_ON(!mvm->refs[ref_type])) {
- spin_unlock_bh(&mvm->refs_lock);
- return;
- }
- mvm->refs[ref_type]--;
- spin_unlock_bh(&mvm->refs_lock);
- iwl_trans_unref(mvm->trans);
-}
-
-static void iwl_mvm_unref_all_except(struct iwl_mvm *mvm,
- enum iwl_mvm_ref_type except_ref)
-{
- int i, j;
-
- if (!iwl_mvm_is_d0i3_supported(mvm))
- return;
-
- spin_lock_bh(&mvm->refs_lock);
- for (i = 0; i < IWL_MVM_REF_COUNT; i++) {
- if (except_ref == i || !mvm->refs[i])
- continue;
-
- IWL_DEBUG_RPM(mvm, "Cleanup: remove mvm ref type %d (%d)\n",
- i, mvm->refs[i]);
- for (j = 0; j < mvm->refs[i]; j++)
- iwl_trans_unref(mvm->trans);
- mvm->refs[i] = 0;
- }
- spin_unlock_bh(&mvm->refs_lock);
-}
-
-bool iwl_mvm_ref_taken(struct iwl_mvm *mvm)
-{
- int i;
- bool taken = false;
-
- if (!iwl_mvm_is_d0i3_supported(mvm))
- return true;
-
- spin_lock_bh(&mvm->refs_lock);
- for (i = 0; i < IWL_MVM_REF_COUNT; i++) {
- if (mvm->refs[i]) {
- taken = true;
- break;
- }
- }
- spin_unlock_bh(&mvm->refs_lock);
-
- return taken;
-}
-
-int iwl_mvm_ref_sync(struct iwl_mvm *mvm, enum iwl_mvm_ref_type ref_type)
-{
- iwl_mvm_ref(mvm, ref_type);
-
- if (!wait_event_timeout(mvm->d0i3_exit_waitq,
- !test_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status),
- HZ)) {
- WARN_ON_ONCE(1);
- iwl_mvm_unref(mvm, ref_type);
- return -EIO;
- }
-
- return 0;
-}
-
static void iwl_mvm_reset_phy_ctxts(struct iwl_mvm *mvm)
{
int i;
@@ -485,7 +400,7 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
* for older devices. We also don't see this issue on any newer
* devices.
*/
- if (mvm->cfg->device_family >= IWL_DEVICE_FAMILY_9000)
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_9000)
ieee80211_hw_set(hw, TX_AMSDU);
ieee80211_hw_set(hw, TX_FRAG_LIST);
@@ -762,12 +677,6 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
mvm->rts_threshold = IEEE80211_MAX_RTS_THRESHOLD;
#ifdef CONFIG_PM_SLEEP
- if (iwl_mvm_is_d0i3_supported(mvm) &&
- device_can_wakeup(mvm->trans->dev)) {
- mvm->wowlan.flags = WIPHY_WOWLAN_ANY;
- hw->wiphy->wowlan = &mvm->wowlan;
- }
-
if ((unified || mvm->fw->img[IWL_UCODE_WOWLAN].num_sec) &&
mvm->trans->ops->d3_suspend &&
mvm->trans->ops->d3_resume &&
@@ -833,46 +742,6 @@ int iwl_mvm_mac_setup_register(struct iwl_mvm *mvm)
return ret;
}
-static bool iwl_mvm_defer_tx(struct iwl_mvm *mvm,
- struct ieee80211_sta *sta,
- struct sk_buff *skb)
-{
- struct iwl_mvm_sta *mvmsta;
- bool defer = false;
-
- /*
- * double check the IN_D0I3 flag both before and after
- * taking the spinlock, in order to prevent taking
- * the spinlock when not needed.
- */
- if (likely(!test_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status)))
- return false;
-
- spin_lock(&mvm->d0i3_tx_lock);
- /*
- * testing the flag again ensures the skb dequeue
- * loop (on d0i3 exit) hasn't run yet.
- */
- if (!test_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status))
- goto out;
-
- mvmsta = iwl_mvm_sta_from_mac80211(sta);
- if (mvmsta->sta_id == IWL_MVM_INVALID_STA ||
- mvmsta->sta_id != mvm->d0i3_ap_sta_id)
- goto out;
-
- __skb_queue_tail(&mvm->d0i3_tx, skb);
-
- /* trigger wakeup */
- iwl_mvm_ref(mvm, IWL_MVM_REF_TX);
- iwl_mvm_unref(mvm, IWL_MVM_REF_TX);
-
- defer = true;
-out:
- spin_unlock(&mvm->d0i3_tx_lock);
- return defer;
-}
-
static void iwl_mvm_mac_tx(struct ieee80211_hw *hw,
struct ieee80211_tx_control *control,
struct sk_buff *skb)
@@ -917,8 +786,6 @@ static void iwl_mvm_mac_tx(struct ieee80211_hw *hw,
}
if (sta) {
- if (iwl_mvm_defer_tx(mvm, sta, skb))
- return;
if (iwl_mvm_tx_skb(mvm, skb, sta))
goto drop;
return;
@@ -1086,7 +953,6 @@ static int iwl_mvm_mac_ampdu_action(struct ieee80211_hw *hw,
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
int ret;
- bool tx_agg_ref = false;
struct ieee80211_sta *sta = params->sta;
enum ieee80211_ampdu_mlme_action action = params->action;
u16 tid = params->tid;
@@ -1101,31 +967,6 @@ static int iwl_mvm_mac_ampdu_action(struct ieee80211_hw *hw,
if (!(mvm->nvm_data->sku_cap_11n_enable))
return -EACCES;
- /* return from D0i3 before starting a new Tx aggregation */
- switch (action) {
- case IEEE80211_AMPDU_TX_START:
- case IEEE80211_AMPDU_TX_STOP_CONT:
- case IEEE80211_AMPDU_TX_STOP_FLUSH:
- case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
- case IEEE80211_AMPDU_TX_OPERATIONAL:
- /*
- * for tx start, wait synchronously until D0i3 exit to
- * get the correct sequence number for the tid.
- * additionally, some other ampdu actions use direct
- * target access, which is not handled automatically
- * by the trans layer (unlike commands), so wait for
- * d0i3 exit in these cases as well.
- */
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_TX_AGG);
- if (ret)
- return ret;
-
- tx_agg_ref = true;
- break;
- default:
- break;
- }
-
mutex_lock(&mvm->mutex);
switch (action) {
@@ -1186,13 +1027,6 @@ static int iwl_mvm_mac_ampdu_action(struct ieee80211_hw *hw,
}
mutex_unlock(&mvm->mutex);
- /*
- * If the tid is marked as started, we won't use it for offloaded
- * traffic on the next D0i3 entry. It's safe to unref.
- */
- if (tx_agg_ref)
- iwl_mvm_unref(mvm, IWL_MVM_REF_TX_AGG);
-
return ret;
}
@@ -1216,11 +1050,6 @@ static void iwl_mvm_cleanup_iterator(void *data, u8 *mac,
static void iwl_mvm_restart_cleanup(struct iwl_mvm *mvm)
{
- /* cleanup all stale references (scan, roc), but keep the
- * ucode_down ref until reconfig is complete
- */
- iwl_mvm_unref_all_except(mvm, IWL_MVM_REF_UCODE_DOWN);
-
iwl_mvm_stop_device(mvm);
mvm->cur_aid = 0;
@@ -1242,7 +1071,6 @@ static void iwl_mvm_restart_cleanup(struct iwl_mvm *mvm)
ieee80211_iterate_interfaces(mvm->hw, 0, iwl_mvm_cleanup_iterator, mvm);
mvm->p2p_device_vif = NULL;
- mvm->d0i3_ap_sta_id = IWL_MVM_INVALID_STA;
iwl_mvm_reset_phy_ctxts(mvm);
memset(mvm->fw_key_table, 0, sizeof(mvm->fw_key_table));
@@ -1251,9 +1079,6 @@ static void iwl_mvm_restart_cleanup(struct iwl_mvm *mvm)
ieee80211_wake_queues(mvm->hw);
- /* clear any stale d0i3 state */
- clear_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status);
-
mvm->vif_count = 0;
mvm->rx_ba_sessions = 0;
mvm->fwrt.dump.conf = FW_DBG_INVALID;
@@ -1278,18 +1103,13 @@ int __iwl_mvm_mac_start(struct iwl_mvm *mvm)
clear_bit(IWL_MVM_STATUS_HW_RESTART_REQUESTED, &mvm->status);
/* Clean up some internal and mac80211 state on restart */
iwl_mvm_restart_cleanup(mvm);
- } else {
- /* Hold the reference to prevent runtime suspend while
- * the start procedure runs. It's a bit confusing
- * that the UCODE_DOWN reference is taken, but it just
- * means "UCODE is not UP yet". ( TODO: rename this
- * reference).
- */
- iwl_mvm_ref(mvm, IWL_MVM_REF_UCODE_DOWN);
}
ret = iwl_mvm_up(mvm);
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_POST_INIT);
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_POST_INIT,
+ NULL);
+ iwl_dbg_tlv_time_point(&mvm->fwrt, IWL_FW_INI_TIME_POINT_PERIODIC,
+ NULL);
if (ret && test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status)) {
/* Something went wrong - we need to finish some cleanup
@@ -1297,9 +1117,6 @@ int __iwl_mvm_mac_start(struct iwl_mvm *mvm)
* would do.
*/
clear_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status);
-#ifdef CONFIG_PM
- iwl_mvm_d0i3_enable_tx(mvm, NULL);
-#endif
}
return ret;
@@ -1310,19 +1127,6 @@ static int iwl_mvm_mac_start(struct ieee80211_hw *hw)
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
int ret;
- /* Some hw restart cleanups must not hold the mutex */
- if (test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status)) {
- /*
- * Make sure we are out of d0i3. This is needed
- * to make sure the reference accounting is correct
- * (and there is no stale d0i3_exit_work).
- */
- wait_event_timeout(mvm->d0i3_exit_waitq,
- !test_bit(IWL_MVM_STATUS_IN_D0I3,
- &mvm->status),
- HZ);
- }
-
mutex_lock(&mvm->mutex);
ret = __iwl_mvm_mac_start(mvm);
mutex_unlock(&mvm->mutex);
@@ -1337,17 +1141,12 @@ static void iwl_mvm_restart_complete(struct iwl_mvm *mvm)
mutex_lock(&mvm->mutex);
clear_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status);
-#ifdef CONFIG_PM
- iwl_mvm_d0i3_enable_tx(mvm, NULL);
-#endif
+
ret = iwl_mvm_update_quotas(mvm, true, NULL);
if (ret)
IWL_ERR(mvm, "Failed to update quotas after restart (%d)\n",
ret);
- /* allow transport/FW low power modes */
- iwl_mvm_unref(mvm, IWL_MVM_REF_UCODE_DOWN);
-
iwl_mvm_send_recovery_cmd(mvm, ERROR_RECOVERY_END_OF_RECOVERY);
/*
@@ -1359,17 +1158,6 @@ static void iwl_mvm_restart_complete(struct iwl_mvm *mvm)
mutex_unlock(&mvm->mutex);
}
-static void iwl_mvm_resume_complete(struct iwl_mvm *mvm)
-{
- if (iwl_mvm_is_d0i3_supported(mvm) &&
- iwl_mvm_enter_d0i3_on_suspend(mvm))
- WARN_ONCE(!wait_event_timeout(mvm->d0i3_exit_waitq,
- !test_bit(IWL_MVM_STATUS_IN_D0I3,
- &mvm->status),
- HZ),
- "D0i3 exit on resume timed out\n");
-}
-
static void
iwl_mvm_mac_reconfig_complete(struct ieee80211_hw *hw,
enum ieee80211_reconfig_type reconfig_type)
@@ -1381,7 +1169,6 @@ iwl_mvm_mac_reconfig_complete(struct ieee80211_hw *hw,
iwl_mvm_restart_complete(mvm);
break;
case IEEE80211_RECONFIG_TYPE_SUSPEND:
- iwl_mvm_resume_complete(mvm);
break;
}
}
@@ -1443,7 +1230,6 @@ static void iwl_mvm_mac_stop(struct ieee80211_hw *hw)
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
- flush_work(&mvm->d0i3_exit_work);
flush_work(&mvm->async_handlers_wk);
flush_work(&mvm->add_stream_wk);
@@ -1457,7 +1243,6 @@ static void iwl_mvm_mac_stop(struct ieee80211_hw *hw)
*/
clear_bit(IWL_MVM_STATUS_FIRMWARE_RUNNING, &mvm->status);
- iwl_fw_cancel_dumps(&mvm->fwrt);
cancel_delayed_work_sync(&mvm->cs_tx_unblock_dwork);
cancel_delayed_work_sync(&mvm->scan_timeout_dwork);
iwl_fw_free_dump_desc(&mvm->fwrt);
@@ -1543,15 +1328,20 @@ static int iwl_mvm_post_channel_switch(struct ieee80211_hw *hw,
goto out_unlock;
}
- iwl_mvm_sta_modify_disable_tx(mvm, mvmsta, false);
+ if (!fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD))
+ iwl_mvm_sta_modify_disable_tx(mvm, mvmsta, false);
iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
- ret = iwl_mvm_enable_beacon_filter(mvm, vif, 0);
- if (ret)
- goto out_unlock;
+ if (!fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD)) {
+ ret = iwl_mvm_enable_beacon_filter(mvm, vif, 0);
+ if (ret)
+ goto out_unlock;
- iwl_mvm_stop_session_protection(mvm, vif);
+ iwl_mvm_stop_session_protection(mvm, vif);
+ }
}
mvmvif->ps_disabled = false;
@@ -1612,15 +1402,6 @@ static int iwl_mvm_mac_add_interface(struct ieee80211_hw *hw,
RCU_INIT_POINTER(mvmvif->probe_resp_data, NULL);
/*
- * make sure D0i3 exit is completed, otherwise a target access
- * during tx queue configuration could be done when still in
- * D0i3 state.
- */
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_ADD_IF);
- if (ret)
- return ret;
-
- /*
* Not much to do here. The stack will not allow interface
* types or combinations that we didn't advertise, so we
* don't really have to check the types.
@@ -1755,8 +1536,6 @@ static int iwl_mvm_mac_add_interface(struct ieee80211_hw *hw,
out_unlock:
mutex_unlock(&mvm->mutex);
- iwl_mvm_unref(mvm, IWL_MVM_REF_ADD_IF);
-
return ret;
}
@@ -2254,6 +2033,10 @@ static void iwl_mvm_cfg_he_sta(struct iwl_mvm *mvm,
flags = 0;
+ /* Block 26-tone RU OFDMA transmissions */
+ if (mvmvif->he_ru_2mhz_block)
+ flags |= STA_CTXT_HE_RU_2MHZ_BLOCK;
+
/* HTC flags */
if (sta->he_cap.he_cap_elem.mac_cap_info[0] &
IEEE80211_HE_MAC_CAP0_HTC_HE)
@@ -2520,7 +2303,6 @@ static void iwl_mvm_bss_info_changed_station(struct iwl_mvm *mvm,
iwl_mvm_sf_update(mvm, vif, false);
iwl_mvm_power_vif_assoc(mvm, vif);
if (vif->p2p) {
- iwl_mvm_ref(mvm, IWL_MVM_REF_P2P_CLIENT);
iwl_mvm_update_smps(mvm, vif,
IWL_MVM_SMPS_REQ_PROT,
IEEE80211_SMPS_DYNAMIC);
@@ -2556,9 +2338,6 @@ static void iwl_mvm_bss_info_changed_station(struct iwl_mvm *mvm,
IWL_ERR(mvm,
"failed to remove AP station\n");
- if (mvm->d0i3_ap_sta_id == mvmvif->ap_sta_id)
- mvm->d0i3_ap_sta_id =
- IWL_MVM_INVALID_STA;
mvmvif->ap_sta_id = IWL_MVM_INVALID_STA;
}
@@ -2567,9 +2346,6 @@ static void iwl_mvm_bss_info_changed_station(struct iwl_mvm *mvm,
if (ret)
IWL_ERR(mvm, "failed to update quotas\n");
- if (vif->p2p)
- iwl_mvm_unref(mvm, IWL_MVM_REF_P2P_CLIENT);
-
/* this will take the cleared BSSID from bss_conf */
ret = iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
if (ret)
@@ -2657,14 +2433,6 @@ static int iwl_mvm_start_ap_ibss(struct ieee80211_hw *hw,
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
int ret, i;
- /*
- * iwl_mvm_mac_ctxt_add() might read directly from the device
- * (the system time), so make sure it is available.
- */
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_START_AP);
- if (ret)
- return ret;
-
mutex_lock(&mvm->mutex);
/* Send the beacon template */
@@ -2760,8 +2528,6 @@ static int iwl_mvm_start_ap_ibss(struct ieee80211_hw *hw,
if (vif->p2p && mvm->p2p_device_vif)
iwl_mvm_mac_ctxt_changed(mvm, mvm->p2p_device_vif, false, NULL);
- iwl_mvm_ref(mvm, IWL_MVM_REF_AP_IBSS);
-
iwl_mvm_bt_coex_vif_change(mvm);
/* we don't support TDLS during DCM */
@@ -2783,7 +2549,6 @@ static int iwl_mvm_start_ap_ibss(struct ieee80211_hw *hw,
iwl_mvm_mac_ctxt_remove(mvm, vif);
out_unlock:
mutex_unlock(&mvm->mutex);
- iwl_mvm_unref(mvm, IWL_MVM_REF_START_AP);
return ret;
}
@@ -2821,8 +2586,6 @@ static void iwl_mvm_stop_ap_ibss(struct ieee80211_hw *hw,
iwl_mvm_bt_coex_vif_change(mvm);
- iwl_mvm_unref(mvm, IWL_MVM_REF_AP_IBSS);
-
/* Need to update the P2P Device MAC (only GO, IBSS is single vif) */
if (vif->p2p && mvm->p2p_device_vif)
iwl_mvm_mac_ctxt_changed(mvm, mvm->p2p_device_vif, false, NULL);
@@ -2896,14 +2659,6 @@ static void iwl_mvm_bss_info_changed(struct ieee80211_hw *hw,
{
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
- /*
- * iwl_mvm_bss_info_changed_station() might call
- * iwl_mvm_protect_session(), which reads directly from
- * the device (the system time), so make sure it is available.
- */
- if (iwl_mvm_ref_sync(mvm, IWL_MVM_REF_BSS_CHANGED))
- return;
-
mutex_lock(&mvm->mutex);
if (changes & BSS_CHANGED_IDLE && !bss_conf->idle)
@@ -2927,7 +2682,6 @@ static void iwl_mvm_bss_info_changed(struct ieee80211_hw *hw,
}
mutex_unlock(&mvm->mutex);
- iwl_mvm_unref(mvm, IWL_MVM_REF_BSS_CHANGED);
}
static int iwl_mvm_mac_hw_scan(struct ieee80211_hw *hw,
@@ -3205,6 +2959,51 @@ iwl_mvm_tdls_check_trigger(struct iwl_mvm *mvm,
peer_addr, action);
}
+struct iwl_mvm_he_obss_narrow_bw_ru_data {
+ bool tolerated;
+};
+
+static void iwl_mvm_check_he_obss_narrow_bw_ru_iter(struct wiphy *wiphy,
+ struct cfg80211_bss *bss,
+ void *_data)
+{
+ struct iwl_mvm_he_obss_narrow_bw_ru_data *data = _data;
+ const struct element *elem;
+
+ elem = cfg80211_find_elem(WLAN_EID_EXT_CAPABILITY, bss->ies->data,
+ bss->ies->len);
+
+ if (!elem || elem->datalen < 10 ||
+ !(elem->data[10] &
+ WLAN_EXT_CAPA10_OBSS_NARROW_BW_RU_TOLERANCE_SUPPORT)) {
+ data->tolerated = false;
+ }
+}
+
+static void iwl_mvm_check_he_obss_narrow_bw_ru(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif)
+{
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ struct iwl_mvm_he_obss_narrow_bw_ru_data iter_data = {
+ .tolerated = true,
+ };
+
+ if (!(vif->bss_conf.chandef.chan->flags & IEEE80211_CHAN_RADAR)) {
+ mvmvif->he_ru_2mhz_block = false;
+ return;
+ }
+
+ cfg80211_bss_iter(hw->wiphy, &vif->bss_conf.chandef,
+ iwl_mvm_check_he_obss_narrow_bw_ru_iter,
+ &iter_data);
+
+ /*
+ * If there is at least one AP on radar channel that cannot
+ * tolerate 26-tone RU UL OFDMA transmissions using HE TB PPDU.
+ */
+ mvmvif->he_ru_2mhz_block = !iter_data.tolerated;
+}
+
static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct ieee80211_sta *sta,
@@ -3306,6 +3105,11 @@ static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
iwl_mvm_cfg_he_sta(mvm, vif, mvm_sta->sta_id);
} else if (vif->type == NL80211_IFTYPE_STATION) {
vif->bss_conf.he_support = sta->he_cap.has_he;
+
+ mvmvif->he_ru_2mhz_block = false;
+ if (sta->he_cap.has_he)
+ iwl_mvm_check_he_obss_narrow_bw_ru(hw, vif);
+
iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
}
@@ -3327,10 +3131,20 @@ static int iwl_mvm_mac_sta_state(struct ieee80211_hw *hw,
/* enable beacon filtering */
WARN_ON(iwl_mvm_enable_beacon_filter(mvm, vif, 0));
+ /*
+ * Now that the station is authorized, i.e., keys were already
+ * installed, need to indicate to the FW that
+ * multicast data frames can be forwarded to the driver
+ */
+ iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
+
iwl_mvm_rs_rate_init(mvm, sta, mvmvif->phy_ctxt->channel->band,
true);
} else if (old_state == IEEE80211_STA_AUTHORIZED &&
new_state == IEEE80211_STA_ASSOC) {
+ /* Multicast data frames are no longer allowed */
+ iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL);
+
/* disable beacon filtering */
ret = iwl_mvm_disable_beacon_filter(mvm, vif, 0);
WARN_ON(ret &&
@@ -3437,13 +3251,6 @@ static void iwl_mvm_mac_mgd_prepare_tx(struct ieee80211_hw *hw,
u32 duration = IWL_MVM_TE_SESSION_PROTECTION_MAX_TIME_MS;
u32 min_duration = IWL_MVM_TE_SESSION_PROTECTION_MIN_TIME_MS;
- /*
- * iwl_mvm_protect_session() reads directly from the device
- * (the system time), so make sure it is available.
- */
- if (iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PREPARE_TX))
- return;
-
if (req_duration > duration)
duration = req_duration;
@@ -3451,8 +3258,6 @@ static void iwl_mvm_mac_mgd_prepare_tx(struct ieee80211_hw *hw,
/* Try really hard to protect the session and hear a beacon */
iwl_mvm_protect_session(mvm, vif, duration, min_duration, 500, false);
mutex_unlock(&mvm->mutex);
-
- iwl_mvm_unref(mvm, IWL_MVM_REF_PREPARE_TX);
}
static int iwl_mvm_mac_sched_scan_start(struct ieee80211_hw *hw,
@@ -3527,7 +3332,7 @@ static int __iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
switch (key->cipher) {
case WLAN_CIPHER_SUITE_TKIP:
- if (!mvm->trans->cfg->gen2) {
+ if (!mvm->trans->trans_cfg->gen2) {
key->flags |= IEEE80211_KEY_FLAG_GENERATE_MMIC;
key->flags |= IEEE80211_KEY_FLAG_PUT_IV_SPACE;
} else if (vif->type == NL80211_IFTYPE_STATION) {
@@ -4268,23 +4073,12 @@ static int __iwl_mvm_assign_vif_chanctx(struct iwl_mvm *mvm,
IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD)) {
u32 duration = 3 * vif->bss_conf.beacon_int;
-
- /* iwl_mvm_protect_session() reads directly from the
- * device (the system time), so make sure it is
- * available.
- */
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PROTECT_CSA);
- if (ret)
- goto out_remove_binding;
-
/* Protect the session to make sure we hear the first
* beacon on the new channel.
*/
iwl_mvm_protect_session(mvm, vif, duration, duration,
vif->bss_conf.beacon_int / 2,
true);
-
- iwl_mvm_unref(mvm, IWL_MVM_REF_PROTECT_CSA);
}
iwl_mvm_update_quotas(mvm, false, NULL);
@@ -4634,6 +4428,42 @@ static int iwl_mvm_schedule_client_csa(struct iwl_mvm *mvm,
0, sizeof(cmd), &cmd);
}
+static int iwl_mvm_old_pre_chan_sw_sta(struct iwl_mvm *mvm,
+ struct ieee80211_vif *vif,
+ struct ieee80211_channel_switch *chsw)
+{
+ struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
+ u32 apply_time;
+
+ /* Schedule the time event to a bit before beacon 1,
+ * to make sure we're in the new channel when the
+ * GO/AP arrives. In case count <= 1 immediately schedule the
+ * TE (this might result with some packet loss or connection
+ * loss).
+ */
+ if (chsw->count <= 1)
+ apply_time = 0;
+ else
+ apply_time = chsw->device_timestamp +
+ ((vif->bss_conf.beacon_int * (chsw->count - 1) -
+ IWL_MVM_CHANNEL_SWITCH_TIME_CLIENT) * 1024);
+
+ if (chsw->block_tx)
+ iwl_mvm_csa_client_absent(mvm, vif);
+
+ if (mvmvif->bf_data.bf_enabled) {
+ int ret = iwl_mvm_disable_beacon_filter(mvm, vif, 0);
+
+ if (ret)
+ return ret;
+ }
+
+ iwl_mvm_schedule_csa_period(mvm, vif, vif->bss_conf.beacon_int,
+ apply_time);
+
+ return 0;
+}
+
#define IWL_MAX_CSA_BLOCK_TX 1500
static int iwl_mvm_pre_channel_switch(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
@@ -4642,7 +4472,6 @@ static int iwl_mvm_pre_channel_switch(struct ieee80211_hw *hw,
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
struct ieee80211_vif *csa_vif;
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- u32 apply_time;
int ret;
mutex_lock(&mvm->mutex);
@@ -4686,21 +4515,7 @@ static int iwl_mvm_pre_channel_switch(struct ieee80211_hw *hw,
break;
case NL80211_IFTYPE_STATION:
- /* Schedule the time event to a bit before beacon 1,
- * to make sure we're in the new channel when the
- * GO/AP arrives. In case count <= 1 immediately schedule the
- * TE (this might result with some packet loss or connection
- * loss).
- */
- if (chsw->count <= 1)
- apply_time = 0;
- else
- apply_time = chsw->device_timestamp +
- ((vif->bss_conf.beacon_int * (chsw->count - 1) -
- IWL_MVM_CHANNEL_SWITCH_TIME_CLIENT) * 1024);
-
if (chsw->block_tx) {
- iwl_mvm_csa_client_absent(mvm, vif);
/*
* In case of undetermined / long time with immediate
* quiet monitor status to gracefully disconnect
@@ -4712,19 +4527,14 @@ static int iwl_mvm_pre_channel_switch(struct ieee80211_hw *hw,
msecs_to_jiffies(IWL_MAX_CSA_BLOCK_TX));
}
- if (mvmvif->bf_data.bf_enabled) {
- ret = iwl_mvm_disable_beacon_filter(mvm, vif, 0);
+ if (!fw_has_capa(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD)) {
+ ret = iwl_mvm_old_pre_chan_sw_sta(mvm, vif, chsw);
if (ret)
goto out_unlock;
- }
-
- if (fw_has_capa(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_CAPA_CHANNEL_SWITCH_CMD))
+ } else {
iwl_mvm_schedule_client_csa(mvm, vif, chsw);
- else
- iwl_mvm_schedule_csa_period(mvm, vif,
- vif->bss_conf.beacon_int,
- apply_time);
+ }
mvmvif->csa_count = chsw->count;
mvmvif->csa_misbehave = false;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
index a263cc6..843d00b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
@@ -258,38 +258,6 @@ enum iwl_mvm_smps_type_request {
NUM_IWL_MVM_SMPS_REQ,
};
-enum iwl_mvm_ref_type {
- IWL_MVM_REF_UCODE_DOWN,
- IWL_MVM_REF_SCAN,
- IWL_MVM_REF_ROC,
- IWL_MVM_REF_ROC_AUX,
- IWL_MVM_REF_P2P_CLIENT,
- IWL_MVM_REF_AP_IBSS,
- IWL_MVM_REF_USER,
- IWL_MVM_REF_TX,
- IWL_MVM_REF_TX_AGG,
- IWL_MVM_REF_ADD_IF,
- IWL_MVM_REF_START_AP,
- IWL_MVM_REF_BSS_CHANGED,
- IWL_MVM_REF_PREPARE_TX,
- IWL_MVM_REF_PROTECT_TDLS,
- IWL_MVM_REF_CHECK_CTKILL,
- IWL_MVM_REF_PRPH_READ,
- IWL_MVM_REF_PRPH_WRITE,
- IWL_MVM_REF_NMI,
- IWL_MVM_REF_TM_CMD,
- IWL_MVM_REF_EXIT_WORK,
- IWL_MVM_REF_PROTECT_CSA,
- IWL_MVM_REF_FW_DBG_COLLECT,
- IWL_MVM_REF_INIT_UCODE,
- IWL_MVM_REF_SENDING_CMD,
- IWL_MVM_REF_RX,
-
- /* update debugfs.c when changing this */
-
- IWL_MVM_REF_COUNT,
-};
-
enum iwl_bt_force_ant_mode {
BT_FORCE_ANT_DIS = 0,
BT_FORCE_ANT_AUTO,
@@ -504,6 +472,9 @@ struct iwl_mvm_vif {
/* we can only have 2 GTK + 2 IGTK active at a time */
struct ieee80211_key_conf *ap_early_keys[4];
+
+ /* 26-tone RU OFDMA transmissions should be blocked */
+ bool he_ru_2mhz_block;
};
static inline struct iwl_mvm_vif *
@@ -617,11 +588,6 @@ struct iwl_mvm_frame_stats {
int last_frame_idx;
};
-enum {
- D0I3_DEFER_WAKEUP,
- D0I3_PENDING_WAKEUP,
-};
-
#define IWL_MVM_DEBUG_SET_TEMPERATURE_DISABLE 0xff
#define IWL_MVM_DEBUG_SET_TEMPERATURE_MIN -100
#define IWL_MVM_DEBUG_SET_TEMPERATURE_MAX 200
@@ -695,6 +661,12 @@ struct iwl_mvm_tcm {
* @valid: reordering is valid for this queue
* @lock: protect reorder buffer internal state
* @mvm: mvm pointer, needed for frame timer context
+ * @consec_oldsn_drops: consecutive drops due to old SN
+ * @consec_oldsn_ampdu_gp2: A-MPDU GP2 timestamp to track
+ * when to apply old SN consecutive drop workaround
+ * @consec_oldsn_prev_drop: track whether or not an MPDU
+ * that was single/part of the previous A-MPDU was
+ * dropped due to old SN
*/
struct iwl_mvm_reorder_buffer {
u16 head_sn;
@@ -708,6 +680,9 @@ struct iwl_mvm_reorder_buffer {
bool valid;
spinlock_t lock;
struct iwl_mvm *mvm;
+ unsigned int consec_oldsn_drops;
+ u32 consec_oldsn_ampdu_gp2;
+ unsigned int consec_oldsn_prev_drop:1;
} ____cacheline_aligned_in_smp;
/**
@@ -1011,10 +986,6 @@ struct iwl_mvm {
unsigned long fw_key_table[BITS_TO_LONGS(STA_KEY_MAX_NUM)];
u8 fw_key_deleted[STA_KEY_MAX_NUM];
- /* references taken by the driver and spinlock protecting them */
- spinlock_t refs_lock;
- u8 refs[IWL_MVM_REF_COUNT];
-
u8 vif_count;
struct ieee80211_vif __rcu *vif_id_to_mac[NUM_MAC_INDEX_DRIVER];
@@ -1039,6 +1010,7 @@ struct iwl_mvm {
struct ieee80211_channel **nd_channels;
int n_nd_channels;
bool net_detect;
+ u8 offload_tid;
#ifdef CONFIG_IWLWIFI_DEBUGFS
bool d3_wake_sysassert;
bool d3_test_active;
@@ -1048,17 +1020,6 @@ struct iwl_mvm {
#endif
#endif
- /* d0i3 */
- u8 d0i3_ap_sta_id;
- bool d0i3_offloading;
- struct work_struct d0i3_exit_work;
- struct sk_buff_head d0i3_tx;
- /* protect d0i3_suspend_flags */
- struct mutex d0i3_suspend_mutex;
- unsigned long d0i3_suspend_flags;
- /* sync d0i3_tx queue and IWL_MVM_STATUS_IN_D0I3 status flag */
- spinlock_t d0i3_tx_lock;
- wait_queue_head_t d0i3_exit_waitq;
wait_queue_head_t rx_sync_waitq;
/* BT-Coex */
@@ -1184,6 +1145,8 @@ struct iwl_mvm {
struct iwl_mvm_sar_profile sar_profiles[ACPI_SAR_PROFILE_NUM];
struct iwl_mvm_geo_profile geo_profiles[ACPI_NUM_GEO_PROFILES];
u32 geo_rev;
+ struct iwl_ppag_table_cmd ppag_table;
+ u32 ppag_rev;
#endif
};
@@ -1201,7 +1164,6 @@ struct iwl_mvm {
* @IWL_MVM_STATUS_ROC_RUNNING: remain-on-channel is running
* @IWL_MVM_STATUS_HW_RESTART_REQUESTED: HW restart was requested
* @IWL_MVM_STATUS_IN_HW_RESTART: HW restart is active
- * @IWL_MVM_STATUS_IN_D0I3: NIC is in D0i3
* @IWL_MVM_STATUS_ROC_AUX_RUNNING: AUX remain-on-channel is running
* @IWL_MVM_STATUS_FIRMWARE_RUNNING: firmware is running
* @IWL_MVM_STATUS_NEED_FLUSH_P2P: need to flush P2P bcast STA
@@ -1212,7 +1174,6 @@ enum iwl_mvm_status {
IWL_MVM_STATUS_ROC_RUNNING,
IWL_MVM_STATUS_HW_RESTART_REQUESTED,
IWL_MVM_STATUS_IN_HW_RESTART,
- IWL_MVM_STATUS_IN_D0I3,
IWL_MVM_STATUS_ROC_AUX_RUNNING,
IWL_MVM_STATUS_FIRMWARE_RUNNING,
IWL_MVM_STATUS_NEED_FLUSH_P2P,
@@ -1291,13 +1252,6 @@ iwl_mvm_rcu_dereference_vif_id(struct iwl_mvm *mvm, u8 vif_id, bool rcu)
lockdep_is_held(&mvm->mutex));
}
-static inline bool iwl_mvm_is_d0i3_supported(struct iwl_mvm *mvm)
-{
- return !iwlwifi_mod_params.d0i3_disable &&
- fw_has_capa(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_CAPA_D0I3_SUPPORT);
-}
-
static inline bool iwl_mvm_is_adaptive_dwell_supported(struct iwl_mvm *mvm)
{
return fw_has_api(&mvm->fw->ucode_capa,
@@ -1333,19 +1287,6 @@ static inline bool iwl_mvm_is_short_beacon_notif_supported(struct iwl_mvm *mvm)
IWL_UCODE_TLV_API_SHORT_BEACON_NOTIF);
}
-static inline bool iwl_mvm_enter_d0i3_on_suspend(struct iwl_mvm *mvm)
-{
- /* For now we only use this mode to differentiate between
- * slave transports, which handle D0i3 entry in suspend by
- * themselves in conjunction with runtime PM D0i3. So, this
- * function is used to check whether we need to do anything
- * when entering suspend or if the transport layer has already
- * done it.
- */
- return (mvm->trans->system_pm_mode == IWL_PLAT_PM_MODE_D0I3) &&
- (mvm->trans->runtime_pm_mode != IWL_PLAT_PM_MODE_D0I3);
-}
-
static inline bool iwl_mvm_is_dqa_data_queue(struct iwl_mvm *mvm, u8 queue)
{
return (queue >= IWL_MVM_DQA_MIN_DATA_QUEUE) &&
@@ -1424,13 +1365,13 @@ static inline bool iwl_mvm_has_new_rx_api(struct iwl_mvm *mvm)
static inline bool iwl_mvm_has_new_tx_api(struct iwl_mvm *mvm)
{
/* TODO - replace with TLV once defined */
- return mvm->trans->cfg->use_tfh;
+ return mvm->trans->trans_cfg->use_tfh;
}
static inline bool iwl_mvm_has_unified_ucode(struct iwl_mvm *mvm)
{
/* TODO - better define this */
- return mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000;
+ return mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000;
}
static inline bool iwl_mvm_is_cdb_supported(struct iwl_mvm *mvm)
@@ -1455,7 +1396,13 @@ static inline bool iwl_mvm_cdb_scan_api(struct iwl_mvm *mvm)
* but then there's a little bit of code in scan that won't make
* any sense...
*/
- return mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000;
+ return mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000;
+}
+
+static inline bool iwl_mvm_is_scan_ext_chan_supported(struct iwl_mvm *mvm)
+{
+ return fw_has_api(&mvm->fw->ucode_capa,
+ IWL_UCODE_TLV_API_SCAN_EXT_CHAN_VER);
}
static inline bool iwl_mvm_has_new_rx_stats_api(struct iwl_mvm *mvm)
@@ -1487,7 +1434,6 @@ iwl_mvm_get_agg_status(struct iwl_mvm *mvm, void *tx_resp)
static inline bool iwl_mvm_is_tt_in_fw(struct iwl_mvm *mvm)
{
-#ifdef CONFIG_THERMAL
/* these two TLV are redundant since the responsibility to CT-kill by
* FW happens only after we send at least one command of
* temperature THs report.
@@ -1496,9 +1442,6 @@ static inline bool iwl_mvm_is_tt_in_fw(struct iwl_mvm *mvm)
IWL_UCODE_TLV_CAPA_CT_KILL_BY_FW) &&
fw_has_capa(&mvm->fw->ucode_capa,
IWL_UCODE_TLV_CAPA_TEMP_THS_REPORT_SUPPORT);
-#else /* CONFIG_THERMAL */
- return false;
-#endif /* CONFIG_THERMAL */
}
static inline bool iwl_mvm_is_ctdp_supported(struct iwl_mvm *mvm)
@@ -1663,6 +1606,8 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi,
struct iwl_rx_cmd_buffer *rxb, int queue);
void iwl_mvm_rx_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
struct iwl_rx_cmd_buffer *rxb, int queue);
+void iwl_mvm_rx_bar_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
+ struct iwl_rx_cmd_buffer *rxb, int queue);
int iwl_mvm_notify_rx_queue(struct iwl_mvm *mvm, u32 rxq_mask,
const u8 *data, u32 count, bool async);
void iwl_mvm_rx_queue_notif(struct iwl_mvm *mvm, struct napi_struct *napi,
@@ -1863,30 +1808,9 @@ void iwl_mvm_set_default_unicast_key(struct ieee80211_hw *hw,
extern const struct file_operations iwl_dbgfs_d3_test_ops;
struct iwl_wowlan_status *iwl_mvm_send_wowlan_get_status(struct iwl_mvm *mvm);
#ifdef CONFIG_PM
-int iwl_mvm_wowlan_config_key_params(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- bool host_awake,
- u32 cmd_flags);
-void iwl_mvm_d0i3_update_keys(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- struct iwl_wowlan_status *status);
void iwl_mvm_set_last_nonqos_seq(struct iwl_mvm *mvm,
struct ieee80211_vif *vif);
#else
-static inline int iwl_mvm_wowlan_config_key_params(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- bool host_awake,
- u32 cmd_flags)
-{
- return 0;
-}
-
-static inline void iwl_mvm_d0i3_update_keys(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- struct iwl_wowlan_status *status)
-{
-}
-
static inline void
iwl_mvm_set_last_nonqos_seq(struct iwl_mvm *mvm, struct ieee80211_vif *vif)
{
@@ -1900,19 +1824,6 @@ int iwl_mvm_send_proto_offload(struct iwl_mvm *mvm,
bool offload_ns,
u32 cmd_flags);
-/* D0i3 */
-void iwl_mvm_ref(struct iwl_mvm *mvm, enum iwl_mvm_ref_type ref_type);
-void iwl_mvm_unref(struct iwl_mvm *mvm, enum iwl_mvm_ref_type ref_type);
-int iwl_mvm_ref_sync(struct iwl_mvm *mvm, enum iwl_mvm_ref_type ref_type);
-bool iwl_mvm_ref_taken(struct iwl_mvm *mvm);
-
-#ifdef CONFIG_PM
-void iwl_mvm_d0i3_enable_tx(struct iwl_mvm *mvm, __le16 *qos_seq);
-int iwl_mvm_enter_d0i3(struct iwl_op_mode *op_mode);
-int iwl_mvm_exit_d0i3(struct iwl_op_mode *op_mode);
-int _iwl_mvm_exit_d0i3(struct iwl_mvm *mvm);
-#endif
-
/* BT Coex */
int iwl_mvm_send_bt_init_conf(struct iwl_mvm *mvm);
void iwl_mvm_rx_bt_coex_notif(struct iwl_mvm *mvm,
@@ -1943,9 +1854,6 @@ iwl_mvm_beacon_filter_debugfs_parameters(struct ieee80211_vif *vif,
struct iwl_beacon_filter_cmd *cmd)
{}
#endif
-int iwl_mvm_update_d0i3_power_mode(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- bool enable, u32 flags);
int iwl_mvm_enable_beacon_filter(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
u32 flags);
@@ -2025,7 +1933,7 @@ void iwl_mvm_vif_set_low_latency(struct iwl_mvm_vif *mvmvif, bool set,
*/
static inline u32 iwl_mvm_flushable_queues(struct iwl_mvm *mvm)
{
- return ((BIT(mvm->cfg->base_params->num_of_queues) - 1) &
+ return ((BIT(mvm->trans->trans_cfg->base_params->num_of_queues) - 1) &
~BIT(IWL_MVM_DQA_CMD_QUEUE));
}
@@ -2034,7 +1942,8 @@ static inline void iwl_mvm_stop_device(struct iwl_mvm *mvm)
lockdep_assert_held(&mvm->mutex);
iwl_fw_cancel_timestamp(&mvm->fwrt);
clear_bit(IWL_MVM_STATUS_FIRMWARE_RUNNING, &mvm->status);
- iwl_fwrt_stop_device(&mvm->fwrt);
+ iwl_fw_dbg_stop_sync(&mvm->fwrt);
+ iwl_trans_stop_device(mvm->trans);
iwl_free_fw_paging(&mvm->fwrt);
iwl_fw_dump_conf_clear(&mvm->fwrt);
}
@@ -2154,6 +2063,7 @@ void iwl_mvm_event_frame_timeout_callback(struct iwl_mvm *mvm,
int iwl_mvm_sar_select_profile(struct iwl_mvm *mvm, int prof_a, int prof_b);
int iwl_mvm_get_sar_geo_profile(struct iwl_mvm *mvm);
+int iwl_mvm_ppag_send_cmd(struct iwl_mvm *mvm);
#ifdef CONFIG_IWLWIFI_DEBUGFS
void iwl_mvm_sta_add_debugfs(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c b/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c
index a9bb43a..945c1ea 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c
@@ -249,7 +249,7 @@ static int iwl_nvm_read_section(struct iwl_mvm *mvm, u16 section,
while (ret == length) {
/* Check no memory assumptions fail and cause an overflow */
if ((size_read + offset + length) >
- mvm->cfg->base_params->eeprom_size) {
+ mvm->trans->trans_cfg->base_params->eeprom_size) {
IWL_ERR(mvm, "EEPROM size is too small for NVM\n");
return -ENOBUFS;
}
@@ -372,7 +372,7 @@ int iwl_nvm_init(struct iwl_mvm *mvm)
/* Read From FW NVM */
IWL_DEBUG_EEPROM(mvm->trans->dev, "Read from NVM\n");
- nvm_buffer = kmalloc(mvm->cfg->base_params->eeprom_size,
+ nvm_buffer = kmalloc(mvm->trans->trans_cfg->base_params->eeprom_size,
GFP_KERNEL);
if (!nvm_buffer)
return -ENOMEM;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
index 4888054..3acbd5b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
@@ -173,7 +173,7 @@ static void iwl_mvm_nic_config(struct iwl_op_mode *op_mode)
* unrelated errors. Need to further investigate this, but for now
* we'll separate cases.
*/
- if (mvm->trans->cfg->device_family < IWL_DEVICE_FAMILY_8000)
+ if (mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_8000)
reg_val |= CSR_HW_IF_CONFIG_REG_BIT_RADIO_SI;
if (iwl_fw_dbg_is_d3_debug_enabled(&mvm->fwrt))
@@ -389,6 +389,7 @@ static const struct iwl_hcmd_names iwl_mvm_legacy_names[] = {
HCMD_NAME(SCAN_ITERATION_COMPLETE_UMAC),
HCMD_NAME(REPLY_RX_PHY_CMD),
HCMD_NAME(REPLY_RX_MPDU_CMD),
+ HCMD_NAME(BAR_FRAME_RELEASE),
HCMD_NAME(FRAME_RELEASE),
HCMD_NAME(BA_NOTIF),
HCMD_NAME(MCC_UPDATE_CMD),
@@ -414,6 +415,7 @@ static const struct iwl_hcmd_names iwl_mvm_legacy_names[] = {
HCMD_NAME(SCAN_ITERATION_COMPLETE),
HCMD_NAME(D0I3_END_CMD),
HCMD_NAME(LTR_CONFIG),
+ HCMD_NAME(LDBG_CONFIG_CMD),
};
/* Please keep this array *SORTED* by hex value.
@@ -465,6 +467,8 @@ static const struct iwl_hcmd_names iwl_mvm_data_path_names[] = {
* Access is done through binary search
*/
static const struct iwl_hcmd_names iwl_mvm_debug_names[] = {
+ HCMD_NAME(DBGC_SUSPEND_RESUME),
+ HCMD_NAME(BUFFER_ALLOCATION),
HCMD_NAME(MFU_ASSERT_DUMP_NTF),
};
@@ -514,9 +518,6 @@ static const struct iwl_hcmd_arr iwl_mvm_groups[] = {
/* this forward declaration can avoid to export the function */
static void iwl_mvm_async_handlers_wk(struct work_struct *wk);
-#ifdef CONFIG_PM
-static void iwl_mvm_d0i3_exit_work(struct work_struct *wk);
-#endif
static u32 iwl_mvm_min_backoff(struct iwl_mvm *mvm)
{
@@ -564,23 +565,16 @@ static void iwl_mvm_tx_unblock_dwork(struct work_struct *work)
static int iwl_mvm_fwrt_dump_start(void *ctx)
{
struct iwl_mvm *mvm = ctx;
- int ret = 0;
mutex_lock(&mvm->mutex);
- ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_FW_DBG_COLLECT);
- if (ret)
- mutex_unlock(&mvm->mutex);
-
- return ret;
+ return 0;
}
static void iwl_mvm_fwrt_dump_end(void *ctx)
{
struct iwl_mvm *mvm = ctx;
- iwl_mvm_unref(mvm, IWL_MVM_REF_FW_DBG_COLLECT);
-
mutex_unlock(&mvm->mutex);
}
@@ -672,7 +666,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
if (iwl_mvm_has_new_rx_api(mvm)) {
op_mode->ops = &iwl_mvm_ops_mq;
trans->rx_mpdu_cmd_hdr_size =
- (trans->cfg->device_family >=
+ (trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560) ?
sizeof(struct iwl_rx_mpdu_desc) :
IWL_RX_DESC_SIZE_V1;
@@ -700,7 +694,6 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
mvm->drop_bcn_ap_mode = true;
mutex_init(&mvm->mutex);
- mutex_init(&mvm->d0i3_suspend_mutex);
spin_lock_init(&mvm->async_handlers_lock);
INIT_LIST_HEAD(&mvm->time_event_list);
INIT_LIST_HEAD(&mvm->aux_roc_te_list);
@@ -710,18 +703,11 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
INIT_WORK(&mvm->async_handlers_wk, iwl_mvm_async_handlers_wk);
INIT_WORK(&mvm->roc_done_wk, iwl_mvm_roc_done_wk);
-#ifdef CONFIG_PM
- INIT_WORK(&mvm->d0i3_exit_work, iwl_mvm_d0i3_exit_work);
-#endif
INIT_DELAYED_WORK(&mvm->tdls_cs.dwork, iwl_mvm_tdls_ch_switch_work);
INIT_DELAYED_WORK(&mvm->scan_timeout_dwork, iwl_mvm_scan_timeout_wk);
INIT_WORK(&mvm->add_stream_wk, iwl_mvm_add_new_dqa_stream_wk);
INIT_LIST_HEAD(&mvm->add_stream_txqs);
- spin_lock_init(&mvm->d0i3_tx_lock);
- spin_lock_init(&mvm->refs_lock);
- skb_queue_head_init(&mvm->d0i3_tx);
- init_waitqueue_head(&mvm->d0i3_exit_waitq);
init_waitqueue_head(&mvm->rx_sync_waitq);
atomic_set(&mvm->queue_sync_counter, 0);
@@ -744,7 +730,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
trans_cfg.no_reclaim_cmds = no_reclaim_cmds;
trans_cfg.n_no_reclaim_cmds = ARRAY_SIZE(no_reclaim_cmds);
- if (mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
rb_size_default = IWL_AMSDU_2K;
else
rb_size_default = IWL_AMSDU_4K;
@@ -768,12 +754,9 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
trans_cfg.rx_buf_size = rb_size_default;
}
- BUILD_BUG_ON(sizeof(struct iwl_ldbg_config_cmd) !=
- LDBG_CFG_COMMAND_SIZE);
-
trans->wide_cmd_header = true;
trans_cfg.bc_table_dword =
- mvm->trans->cfg->device_family < IWL_DEVICE_FAMILY_22560;
+ mvm->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_22560;
trans_cfg.command_groups = iwl_mvm_groups;
trans_cfg.command_groups_size = ARRAY_SIZE(iwl_mvm_groups);
@@ -832,13 +815,11 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
goto out_free;
mutex_lock(&mvm->mutex);
- iwl_mvm_ref(mvm, IWL_MVM_REF_INIT_UCODE);
err = iwl_run_init_mvm_ucode(mvm, true);
if (err && err != -ERFKILL)
iwl_fw_dbg_error_collect(&mvm->fwrt, FW_DBG_TRIGGER_DRIVER);
if (!iwlmvm_mod_params.init_dbg || !err)
iwl_mvm_stop_device(mvm);
- iwl_mvm_unref(mvm, IWL_MVM_REF_INIT_UCODE);
mutex_unlock(&mvm->mutex);
if (err < 0) {
IWL_ERR(mvm, "Failed to run INIT ucode: %d\n", err);
@@ -870,11 +851,6 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg,
else
memset(&mvm->rx_stats, 0, sizeof(struct mvm_statistics_rx));
- /* The transport always starts with a taken reference, we can
- * release it now if d0i3 is supported */
- if (iwl_mvm_is_d0i3_supported(mvm))
- iwl_trans_unref(mvm->trans);
-
iwl_mvm_toggle_tx_ant(mvm, &mvm->mgmt_last_antenna_idx);
return op_mode;
@@ -898,13 +874,6 @@ static void iwl_op_mode_mvm_stop(struct iwl_op_mode *op_mode)
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
int i;
- /* If d0i3 is supported, we have released the reference that
- * the transport started with, so we should take it back now
- * that we are leaving.
- */
- if (iwl_mvm_is_d0i3_supported(mvm))
- iwl_trans_ref(mvm->trans);
-
iwl_mvm_leds_exit(mvm);
iwl_mvm_thermal_exit(mvm);
@@ -931,7 +900,6 @@ static void iwl_op_mode_mvm_stop(struct iwl_op_mode *op_mode)
iwl_fw_runtime_free(&mvm->fwrt);
mutex_destroy(&mvm->mutex);
- mutex_destroy(&mvm->d0i3_suspend_mutex);
ieee80211_free_hw(mvm->hw);
}
@@ -1020,7 +988,10 @@ static void iwl_mvm_rx_common(struct iwl_mvm *mvm,
struct iwl_rx_packet *pkt)
{
int i;
+ union iwl_dbg_tlv_tp_data tp_data = { .fw_pkt = pkt };
+ iwl_dbg_tlv_time_point(&mvm->fwrt,
+ IWL_FW_INI_TIME_POINT_FW_RSP_OR_NOTIF, &tp_data);
iwl_mvm_rx_check_trigger(mvm, pkt);
/*
@@ -1091,6 +1062,8 @@ static void iwl_mvm_rx_mq(struct iwl_op_mode *op_mode,
iwl_mvm_rx_queue_notif(mvm, napi, rxb, 0);
else if (cmd == WIDE_ID(LEGACY_GROUP, FRAME_RELEASE))
iwl_mvm_rx_frame_release(mvm, napi, rxb, 0);
+ else if (cmd == WIDE_ID(LEGACY_GROUP, BAR_FRAME_RELEASE))
+ iwl_mvm_rx_bar_frame_release(mvm, napi, rxb, 0);
else if (cmd == WIDE_ID(DATA_PATH_GROUP, RX_NO_DATA_NOTIF))
iwl_mvm_rx_monitor_no_data(mvm, napi, rxb, 0);
else
@@ -1234,8 +1207,7 @@ static bool iwl_mvm_set_hw_rfkill_state(struct iwl_op_mode *op_mode, bool state)
* Stop the device if we run OPERATIONAL firmware or if we are in the
* middle of the calibrations.
*/
- return state && (mvm->fwrt.cur_fw_img != IWL_UCODE_INIT ||
- rfkill_safe_init_done);
+ return state && rfkill_safe_init_done;
}
static void iwl_mvm_free_skb(struct iwl_op_mode *op_mode, struct sk_buff *skb)
@@ -1267,7 +1239,7 @@ static void iwl_mvm_reprobe_wk(struct work_struct *wk)
void iwl_mvm_nic_restart(struct iwl_mvm *mvm, bool fw_error)
{
iwl_abort_notification_waits(&mvm->notif_wait);
- del_timer(&mvm->fwrt.dump.periodic_trig);
+ iwl_dbg_tlv_del_timers(mvm->trans);
/*
* This is a bit racy, but worst case we tell mac80211 about
@@ -1319,9 +1291,6 @@ void iwl_mvm_nic_restart(struct iwl_mvm *mvm, bool fw_error)
} else if (mvm->fwrt.cur_fw_img == IWL_UCODE_REGULAR &&
mvm->hw_registered &&
!test_bit(STATUS_TRANS_DEAD, &mvm->trans->status)) {
- /* don't let the transport/FW power down */
- iwl_mvm_ref(mvm, IWL_MVM_REF_UCODE_DOWN);
-
if (mvm->fw->ucode_capa.error_log_size) {
u32 src_size = mvm->fw->ucode_capa.error_log_size;
u32 src_addr = mvm->fw->ucode_capa.error_log_addr;
@@ -1363,422 +1332,6 @@ static void iwl_mvm_cmd_queue_full(struct iwl_op_mode *op_mode)
iwl_mvm_nic_restart(mvm, true);
}
-#ifdef CONFIG_PM
-struct iwl_d0i3_iter_data {
- struct iwl_mvm *mvm;
- struct ieee80211_vif *connected_vif;
- u8 ap_sta_id;
- u8 vif_count;
- u8 offloading_tid;
- bool disable_offloading;
-};
-
-static bool iwl_mvm_disallow_offloading(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- struct iwl_d0i3_iter_data *iter_data)
-{
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- struct iwl_mvm_sta *mvmsta;
- u32 available_tids = 0;
- u8 tid;
-
- if (WARN_ON(vif->type != NL80211_IFTYPE_STATION ||
- mvmvif->ap_sta_id == IWL_MVM_INVALID_STA))
- return false;
-
- mvmsta = iwl_mvm_sta_from_staid_rcu(mvm, mvmvif->ap_sta_id);
- if (!mvmsta)
- return false;
-
- spin_lock_bh(&mvmsta->lock);
- for (tid = 0; tid < IWL_MAX_TID_COUNT; tid++) {
- struct iwl_mvm_tid_data *tid_data = &mvmsta->tid_data[tid];
-
- /*
- * in case of pending tx packets, don't use this tid
- * for offloading in order to prevent reuse of the same
- * qos seq counters.
- */
- if (iwl_mvm_tid_queued(mvm, tid_data))
- continue;
-
- if (tid_data->state != IWL_AGG_OFF)
- continue;
-
- available_tids |= BIT(tid);
- }
- spin_unlock_bh(&mvmsta->lock);
-
- /*
- * disallow protocol offloading if we have no available tid
- * (with no pending frames and no active aggregation,
- * as we don't handle "holes" properly - the scheduler needs the
- * frame's seq number and TFD index to match)
- */
- if (!available_tids)
- return true;
-
- /* for simplicity, just use the first available tid */
- iter_data->offloading_tid = ffs(available_tids) - 1;
- return false;
-}
-
-static void iwl_mvm_enter_d0i3_iterator(void *_data, u8 *mac,
- struct ieee80211_vif *vif)
-{
- struct iwl_d0i3_iter_data *data = _data;
- struct iwl_mvm *mvm = data->mvm;
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- u32 flags = CMD_ASYNC | CMD_HIGH_PRIO | CMD_SEND_IN_IDLE;
-
- IWL_DEBUG_RPM(mvm, "entering D0i3 - vif %pM\n", vif->addr);
- if (vif->type != NL80211_IFTYPE_STATION ||
- !vif->bss_conf.assoc)
- return;
-
- /*
- * in case of pending tx packets or active aggregations,
- * avoid offloading features in order to prevent reuse of
- * the same qos seq counters.
- */
- if (iwl_mvm_disallow_offloading(mvm, vif, data))
- data->disable_offloading = true;
-
- iwl_mvm_update_d0i3_power_mode(mvm, vif, true, flags);
- iwl_mvm_send_proto_offload(mvm, vif, data->disable_offloading,
- false, flags);
-
- /*
- * on init/association, mvm already configures POWER_TABLE_CMD
- * and REPLY_MCAST_FILTER_CMD, so currently don't
- * reconfigure them (we might want to use different
- * params later on, though).
- */
- data->ap_sta_id = mvmvif->ap_sta_id;
- data->vif_count++;
-
- /*
- * no new commands can be sent at this stage, so it's safe
- * to save the vif pointer during d0i3 entrance.
- */
- data->connected_vif = vif;
-}
-
-static void iwl_mvm_set_wowlan_data(struct iwl_mvm *mvm,
- struct iwl_wowlan_config_cmd *cmd,
- struct iwl_d0i3_iter_data *iter_data)
-{
- struct ieee80211_sta *ap_sta;
- struct iwl_mvm_sta *mvm_ap_sta;
-
- if (iter_data->ap_sta_id == IWL_MVM_INVALID_STA)
- return;
-
- rcu_read_lock();
-
- ap_sta = rcu_dereference(mvm->fw_id_to_mac_id[iter_data->ap_sta_id]);
- if (IS_ERR_OR_NULL(ap_sta))
- goto out;
-
- mvm_ap_sta = iwl_mvm_sta_from_mac80211(ap_sta);
- cmd->is_11n_connection = ap_sta->ht_cap.ht_supported;
- cmd->offloading_tid = iter_data->offloading_tid;
- cmd->flags = ENABLE_L3_FILTERING | ENABLE_NBNS_FILTERING |
- ENABLE_DHCP_FILTERING | ENABLE_STORE_BEACON;
- /*
- * The d0i3 uCode takes care of the nonqos counters,
- * so configure only the qos seq ones.
- */
- iwl_mvm_set_wowlan_qos_seq(mvm_ap_sta, cmd);
-out:
- rcu_read_unlock();
-}
-
-int iwl_mvm_enter_d0i3(struct iwl_op_mode *op_mode)
-{
- struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
- u32 flags = CMD_ASYNC | CMD_HIGH_PRIO | CMD_SEND_IN_IDLE;
- int ret;
- struct iwl_d0i3_iter_data d0i3_iter_data = {
- .mvm = mvm,
- };
- struct iwl_wowlan_config_cmd wowlan_config_cmd = {
- .wakeup_filter = cpu_to_le32(IWL_WOWLAN_WAKEUP_RX_FRAME |
- IWL_WOWLAN_WAKEUP_BEACON_MISS |
- IWL_WOWLAN_WAKEUP_LINK_CHANGE),
- };
- struct iwl_d3_manager_config d3_cfg_cmd = {
- .min_sleep_time = cpu_to_le32(1000),
- .wakeup_flags = cpu_to_le32(IWL_WAKEUP_D3_CONFIG_FW_ERROR),
- };
-
- IWL_DEBUG_RPM(mvm, "MVM entering D0i3\n");
-
- if (WARN_ON_ONCE(mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR))
- return -EINVAL;
-
- set_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status);
-
- /*
- * iwl_mvm_ref_sync takes a reference before checking the flag.
- * so by checking there is no held reference we prevent a state
- * in which iwl_mvm_ref_sync continues successfully while we
- * configure the firmware to enter d0i3
- */
- if (iwl_mvm_ref_taken(mvm)) {
- IWL_DEBUG_RPM(mvm->trans, "abort d0i3 due to taken ref\n");
- clear_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status);
- wake_up(&mvm->d0i3_exit_waitq);
- return 1;
- }
-
- ieee80211_iterate_active_interfaces_atomic(mvm->hw,
- IEEE80211_IFACE_ITER_NORMAL,
- iwl_mvm_enter_d0i3_iterator,
- &d0i3_iter_data);
- if (d0i3_iter_data.vif_count == 1) {
- mvm->d0i3_ap_sta_id = d0i3_iter_data.ap_sta_id;
- mvm->d0i3_offloading = !d0i3_iter_data.disable_offloading;
- } else {
- WARN_ON_ONCE(d0i3_iter_data.vif_count > 1);
- mvm->d0i3_ap_sta_id = IWL_MVM_INVALID_STA;
- mvm->d0i3_offloading = false;
- }
-
- iwl_mvm_pause_tcm(mvm, true);
- /* make sure we have no running tx while configuring the seqno */
- synchronize_net();
-
- /* Flush the hw queues, in case something got queued during entry */
- /* TODO new tx api */
- if (iwl_mvm_has_new_tx_api(mvm)) {
- WARN_ONCE(1, "d0i3: Need to implement flush TX queue\n");
- } else {
- ret = iwl_mvm_flush_tx_path(mvm, iwl_mvm_flushable_queues(mvm),
- flags);
- if (ret)
- return ret;
- }
-
- /* configure wowlan configuration only if needed */
- if (mvm->d0i3_ap_sta_id != IWL_MVM_INVALID_STA) {
- /* wake on beacons only if beacon storing isn't supported */
- if (!fw_has_capa(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_CAPA_BEACON_STORING))
- wowlan_config_cmd.wakeup_filter |=
- cpu_to_le32(IWL_WOWLAN_WAKEUP_BCN_FILTERING);
-
- iwl_mvm_wowlan_config_key_params(mvm,
- d0i3_iter_data.connected_vif,
- true, flags);
-
- iwl_mvm_set_wowlan_data(mvm, &wowlan_config_cmd,
- &d0i3_iter_data);
-
- ret = iwl_mvm_send_cmd_pdu(mvm, WOWLAN_CONFIGURATION, flags,
- sizeof(wowlan_config_cmd),
- &wowlan_config_cmd);
- if (ret)
- return ret;
- }
-
- return iwl_mvm_send_cmd_pdu(mvm, D3_CONFIG_CMD,
- flags | CMD_MAKE_TRANS_IDLE,
- sizeof(d3_cfg_cmd), &d3_cfg_cmd);
-}
-
-static void iwl_mvm_exit_d0i3_iterator(void *_data, u8 *mac,
- struct ieee80211_vif *vif)
-{
- struct iwl_mvm *mvm = _data;
- u32 flags = CMD_ASYNC | CMD_HIGH_PRIO;
-
- IWL_DEBUG_RPM(mvm, "exiting D0i3 - vif %pM\n", vif->addr);
- if (vif->type != NL80211_IFTYPE_STATION ||
- !vif->bss_conf.assoc)
- return;
-
- iwl_mvm_update_d0i3_power_mode(mvm, vif, false, flags);
-}
-
-struct iwl_mvm_d0i3_exit_work_iter_data {
- struct iwl_mvm *mvm;
- struct iwl_wowlan_status *status;
- u32 wakeup_reasons;
-};
-
-static void iwl_mvm_d0i3_exit_work_iter(void *_data, u8 *mac,
- struct ieee80211_vif *vif)
-{
- struct iwl_mvm_d0i3_exit_work_iter_data *data = _data;
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- u32 reasons = data->wakeup_reasons;
-
- /* consider only the relevant station interface */
- if (vif->type != NL80211_IFTYPE_STATION || !vif->bss_conf.assoc ||
- data->mvm->d0i3_ap_sta_id != mvmvif->ap_sta_id)
- return;
-
- if (reasons & IWL_WOWLAN_WAKEUP_BY_DISCONNECTION_ON_DEAUTH)
- iwl_mvm_connection_loss(data->mvm, vif, "D0i3");
- else if (reasons & IWL_WOWLAN_WAKEUP_BY_DISCONNECTION_ON_MISSED_BEACON)
- ieee80211_beacon_loss(vif);
- else
- iwl_mvm_d0i3_update_keys(data->mvm, vif, data->status);
-}
-
-void iwl_mvm_d0i3_enable_tx(struct iwl_mvm *mvm, __le16 *qos_seq)
-{
- struct ieee80211_sta *sta = NULL;
- struct iwl_mvm_sta *mvm_ap_sta;
- int i;
- bool wake_queues = false;
-
- lockdep_assert_held(&mvm->mutex);
-
- spin_lock_bh(&mvm->d0i3_tx_lock);
-
- if (mvm->d0i3_ap_sta_id == IWL_MVM_INVALID_STA)
- goto out;
-
- IWL_DEBUG_RPM(mvm, "re-enqueue packets\n");
-
- /* get the sta in order to update seq numbers and re-enqueue skbs */
- sta = rcu_dereference_protected(
- mvm->fw_id_to_mac_id[mvm->d0i3_ap_sta_id],
- lockdep_is_held(&mvm->mutex));
-
- if (IS_ERR_OR_NULL(sta)) {
- sta = NULL;
- goto out;
- }
-
- if (mvm->d0i3_offloading && qos_seq) {
- /* update qos seq numbers if offloading was enabled */
- mvm_ap_sta = iwl_mvm_sta_from_mac80211(sta);
- for (i = 0; i < IWL_MAX_TID_COUNT; i++) {
- u16 seq = le16_to_cpu(qos_seq[i]);
- /* firmware stores last-used one, we store next one */
- seq += 0x10;
- mvm_ap_sta->tid_data[i].seq_number = seq;
- }
- }
-out:
- /* re-enqueue (or drop) all packets */
- while (!skb_queue_empty(&mvm->d0i3_tx)) {
- struct sk_buff *skb = __skb_dequeue(&mvm->d0i3_tx);
-
- if (!sta || iwl_mvm_tx_skb(mvm, skb, sta))
- ieee80211_free_txskb(mvm->hw, skb);
-
- /* if the skb_queue is not empty, we need to wake queues */
- wake_queues = true;
- }
- clear_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status);
- wake_up(&mvm->d0i3_exit_waitq);
- mvm->d0i3_ap_sta_id = IWL_MVM_INVALID_STA;
- if (wake_queues)
- ieee80211_wake_queues(mvm->hw);
-
- spin_unlock_bh(&mvm->d0i3_tx_lock);
-}
-
-static void iwl_mvm_d0i3_exit_work(struct work_struct *wk)
-{
- struct iwl_mvm *mvm = container_of(wk, struct iwl_mvm, d0i3_exit_work);
- struct iwl_mvm_d0i3_exit_work_iter_data iter_data = {
- .mvm = mvm,
- };
-
- struct iwl_wowlan_status *status;
- u32 wakeup_reasons = 0;
- __le16 *qos_seq = NULL;
-
- mutex_lock(&mvm->mutex);
-
- status = iwl_mvm_send_wowlan_get_status(mvm);
- if (IS_ERR_OR_NULL(status)) {
- /* set to NULL so we don't need to check before kfree'ing */
- status = NULL;
- goto out;
- }
-
- wakeup_reasons = le32_to_cpu(status->wakeup_reasons);
- qos_seq = status->qos_seq_ctr;
-
- IWL_DEBUG_RPM(mvm, "wakeup reasons: 0x%x\n", wakeup_reasons);
-
- iter_data.wakeup_reasons = wakeup_reasons;
- iter_data.status = status;
- ieee80211_iterate_active_interfaces(mvm->hw,
- IEEE80211_IFACE_ITER_NORMAL,
- iwl_mvm_d0i3_exit_work_iter,
- &iter_data);
-out:
- iwl_mvm_d0i3_enable_tx(mvm, qos_seq);
-
- IWL_DEBUG_INFO(mvm, "d0i3 exit completed (wakeup reasons: 0x%x)\n",
- wakeup_reasons);
-
- /* qos_seq might point inside resp_pkt, so free it only now */
- kfree(status);
-
- /* the FW might have updated the regdomain */
- iwl_mvm_update_changed_regdom(mvm);
-
- iwl_mvm_resume_tcm(mvm);
- iwl_mvm_unref(mvm, IWL_MVM_REF_EXIT_WORK);
- mutex_unlock(&mvm->mutex);
-}
-
-int _iwl_mvm_exit_d0i3(struct iwl_mvm *mvm)
-{
- u32 flags = CMD_ASYNC | CMD_HIGH_PRIO | CMD_SEND_IN_IDLE |
- CMD_WAKE_UP_TRANS;
- int ret;
-
- IWL_DEBUG_RPM(mvm, "MVM exiting D0i3\n");
-
- if (WARN_ON_ONCE(mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR))
- return -EINVAL;
-
- mutex_lock(&mvm->d0i3_suspend_mutex);
- if (test_bit(D0I3_DEFER_WAKEUP, &mvm->d0i3_suspend_flags)) {
- IWL_DEBUG_RPM(mvm, "Deferring d0i3 exit until resume\n");
- __set_bit(D0I3_PENDING_WAKEUP, &mvm->d0i3_suspend_flags);
- mutex_unlock(&mvm->d0i3_suspend_mutex);
- return 0;
- }
- mutex_unlock(&mvm->d0i3_suspend_mutex);
-
- ret = iwl_mvm_send_cmd_pdu(mvm, D0I3_END_CMD, flags, 0, NULL);
- if (ret)
- goto out;
-
- ieee80211_iterate_active_interfaces_atomic(mvm->hw,
- IEEE80211_IFACE_ITER_NORMAL,
- iwl_mvm_exit_d0i3_iterator,
- mvm);
-out:
- schedule_work(&mvm->d0i3_exit_work);
- return ret;
-}
-
-int iwl_mvm_exit_d0i3(struct iwl_op_mode *op_mode)
-{
- struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
-
- iwl_mvm_ref(mvm, IWL_MVM_REF_EXIT_WORK);
- return _iwl_mvm_exit_d0i3(mvm);
-}
-
-#define IWL_MVM_D0I3_OPS \
- .enter_d0i3 = iwl_mvm_enter_d0i3, \
- .exit_d0i3 = iwl_mvm_exit_d0i3,
-#else /* CONFIG_PM */
-#define IWL_MVM_D0I3_OPS
-#endif /* CONFIG_PM */
-
#define IWL_MVM_COMMON_OPS \
/* these could be differentiated */ \
.async_cb = iwl_mvm_async_cb, \
@@ -1789,7 +1342,6 @@ int iwl_mvm_exit_d0i3(struct iwl_op_mode *op_mode)
.nic_error = iwl_mvm_nic_error, \
.cmd_queue_full = iwl_mvm_cmd_queue_full, \
.nic_config = iwl_mvm_nic_config, \
- IWL_MVM_D0I3_OPS \
/* as we only register one, these MUST be common! */ \
.start = iwl_op_mode_mvm_start, \
.stop = iwl_op_mode_mvm_stop
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c
index 86e40bae5..0243dbe 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/phy-ctxt.c
@@ -289,8 +289,17 @@ void iwl_mvm_phy_ctxt_unref(struct iwl_mvm *mvm, struct iwl_mvm_phy_ctxt *ctxt)
if (ctxt->ref == 0) {
struct ieee80211_channel *chan;
struct cfg80211_chan_def chandef;
+ struct ieee80211_supported_band *sband = NULL;
+ enum nl80211_band band = NL80211_BAND_2GHZ;
- chan = &mvm->hw->wiphy->bands[NL80211_BAND_2GHZ]->channels[0];
+ while (!sband && band < NUM_NL80211_BANDS)
+ sband = mvm->hw->wiphy->bands[band++];
+
+ if (WARN_ON(!sband))
+ return;
+
+ chan = &sband->channels[0];
+
cfg80211_chandef_create(&chandef, chan, NL80211_CHAN_NO_HT);
iwl_mvm_phy_ctxt_changed(mvm, ctxt, &chandef, 1, 1);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/power.c b/drivers/net/wireless/intel/iwlwifi/mvm/power.c
index 36f5fa1..22136e4 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/power.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/power.c
@@ -127,12 +127,11 @@ int iwl_mvm_beacon_filter_send_cmd(struct iwl_mvm *mvm,
static
void iwl_mvm_beacon_filter_set_cqm_params(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
- struct iwl_beacon_filter_cmd *cmd,
- bool d0i3)
+ struct iwl_beacon_filter_cmd *cmd)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- if (vif->bss_conf.cqm_rssi_thold && !d0i3) {
+ if (vif->bss_conf.cqm_rssi_thold) {
cmd->bf_energy_delta =
cpu_to_le32(vif->bss_conf.cqm_rssi_hyst);
/* fw uses an absolute value for this */
@@ -849,8 +848,7 @@ iwl_mvm_beacon_filter_debugfs_parameters(struct ieee80211_vif *vif,
static int _iwl_mvm_enable_beacon_filter(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
struct iwl_beacon_filter_cmd *cmd,
- u32 cmd_flags,
- bool d0i3)
+ u32 cmd_flags)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
int ret;
@@ -859,13 +857,11 @@ static int _iwl_mvm_enable_beacon_filter(struct iwl_mvm *mvm,
vif->type != NL80211_IFTYPE_STATION || vif->p2p)
return 0;
- iwl_mvm_beacon_filter_set_cqm_params(mvm, vif, cmd, d0i3);
- if (!d0i3)
- iwl_mvm_beacon_filter_debugfs_parameters(vif, cmd);
+ iwl_mvm_beacon_filter_set_cqm_params(mvm, vif, cmd);
+ iwl_mvm_beacon_filter_debugfs_parameters(vif, cmd);
ret = iwl_mvm_beacon_filter_send_cmd(mvm, cmd, cmd_flags);
- /* don't change bf_enabled in case of temporary d0i3 configuration */
- if (!ret && !d0i3)
+ if (!ret)
mvmvif->bf_data.bf_enabled = true;
return ret;
@@ -880,12 +876,12 @@ int iwl_mvm_enable_beacon_filter(struct iwl_mvm *mvm,
.bf_enable_beacon_filter = cpu_to_le32(1),
};
- return _iwl_mvm_enable_beacon_filter(mvm, vif, &cmd, flags, false);
+ return _iwl_mvm_enable_beacon_filter(mvm, vif, &cmd, flags);
}
static int _iwl_mvm_disable_beacon_filter(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
- u32 flags, bool d0i3)
+ u32 flags)
{
struct iwl_beacon_filter_cmd cmd = {};
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
@@ -896,8 +892,7 @@ static int _iwl_mvm_disable_beacon_filter(struct iwl_mvm *mvm,
ret = iwl_mvm_beacon_filter_send_cmd(mvm, &cmd, flags);
- /* don't change bf_enabled in case of temporary d0i3 configuration */
- if (!ret && !d0i3)
+ if (!ret)
mvmvif->bf_data.bf_enabled = false;
return ret;
@@ -907,7 +902,7 @@ int iwl_mvm_disable_beacon_filter(struct iwl_mvm *mvm,
struct ieee80211_vif *vif,
u32 flags)
{
- return _iwl_mvm_disable_beacon_filter(mvm, vif, flags, false);
+ return _iwl_mvm_disable_beacon_filter(mvm, vif, flags);
}
static int iwl_mvm_power_set_ps(struct iwl_mvm *mvm)
@@ -958,7 +953,7 @@ static int iwl_mvm_power_set_ba(struct iwl_mvm *mvm,
!vif->bss_conf.ps ||
iwl_mvm_vif_low_latency(mvmvif));
- return _iwl_mvm_enable_beacon_filter(mvm, vif, &cmd, 0, false);
+ return _iwl_mvm_enable_beacon_filter(mvm, vif, &cmd, 0);
}
int iwl_mvm_power_update_ps(struct iwl_mvm *mvm)
@@ -1022,58 +1017,3 @@ int iwl_mvm_power_update_mac(struct iwl_mvm *mvm)
return 0;
}
-
-int iwl_mvm_update_d0i3_power_mode(struct iwl_mvm *mvm,
- struct ieee80211_vif *vif,
- bool enable, u32 flags)
-{
- int ret;
- struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- struct iwl_mac_power_cmd cmd = {};
-
- if (vif->type != NL80211_IFTYPE_STATION || vif->p2p)
- return 0;
-
- if (!vif->bss_conf.assoc)
- return 0;
-
- iwl_mvm_power_build_cmd(mvm, vif, &cmd, !enable);
-
- iwl_mvm_power_log(mvm, &cmd);
-#ifdef CONFIG_IWLWIFI_DEBUGFS
- memcpy(&mvmvif->mac_pwr_cmd, &cmd, sizeof(cmd));
-#endif
- ret = iwl_mvm_send_cmd_pdu(mvm, MAC_PM_POWER_TABLE, flags,
- sizeof(cmd), &cmd);
- if (ret)
- return ret;
-
- /* configure beacon filtering */
- if (mvmvif != mvm->bf_allowed_vif)
- return 0;
-
- if (enable) {
- struct iwl_beacon_filter_cmd cmd_bf = {
- IWL_BF_CMD_CONFIG_D0I3,
- .bf_enable_beacon_filter = cpu_to_le32(1),
- };
- /*
- * When beacon storing is supported - disable beacon filtering
- * altogether - the latest beacon will be sent when exiting d0i3
- */
- if (fw_has_capa(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_CAPA_BEACON_STORING))
- ret = _iwl_mvm_disable_beacon_filter(mvm, vif, flags,
- true);
- else
- ret = _iwl_mvm_enable_beacon_filter(mvm, vif, &cmd_bf,
- flags, true);
- } else {
- if (mvmvif->bf_data.bf_enabled)
- ret = iwl_mvm_enable_beacon_filter(mvm, vif, flags);
- else
- ret = iwl_mvm_disable_beacon_filter(mvm, vif, flags);
- }
-
- return ret;
-}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
index 08b6781..8f50e2b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c
@@ -193,7 +193,7 @@ rs_fw_vht_set_enabled_rates(const struct ieee80211_sta *sta,
int i, highest_mcs;
for (i = 0; i < sta->rx_nss; i++) {
- if (i == MAX_NSS)
+ if (i == IWL_TLC_NSS_MAX)
break;
highest_mcs = rs_fw_vht_highest_rx_mcs_index(vht_cap, i + 1);
@@ -204,9 +204,10 @@ rs_fw_vht_set_enabled_rates(const struct ieee80211_sta *sta,
if (sta->bandwidth == IEEE80211_STA_RX_BW_20)
supp &= ~BIT(IWL_TLC_MNG_HT_RATE_MCS9);
- cmd->ht_rates[i][0] = cpu_to_le16(supp);
+ cmd->ht_rates[i][IWL_TLC_HT_BW_NONE_160] = cpu_to_le16(supp);
if (sta->bandwidth == IEEE80211_STA_RX_BW_160)
- cmd->ht_rates[i][1] = cmd->ht_rates[i][0];
+ cmd->ht_rates[i][IWL_TLC_HT_BW_160] =
+ cmd->ht_rates[i][IWL_TLC_HT_BW_NONE_160];
}
}
@@ -241,7 +242,7 @@ rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta,
le16_to_cpu(sband->iftype_data->he_cap.he_mcs_nss_supp.tx_mcs_160);
int i;
- for (i = 0; i < sta->rx_nss && i < MAX_NSS; i++) {
+ for (i = 0; i < sta->rx_nss && i < IWL_TLC_NSS_MAX; i++) {
u16 _mcs_160 = (mcs_160 >> (2 * i)) & 0x3;
u16 _mcs_80 = (mcs_80 >> (2 * i)) & 0x3;
u16 _tx_mcs_160 = (tx_mcs_160 >> (2 * i)) & 0x3;
@@ -255,7 +256,7 @@ rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta,
}
if (_mcs_80 > _tx_mcs_80)
_mcs_80 = _tx_mcs_80;
- cmd->ht_rates[i][0] =
+ cmd->ht_rates[i][IWL_TLC_HT_BW_NONE_160] =
cpu_to_le16(rs_fw_he_ieee80211_mcs_to_rs_mcs(_mcs_80));
/* If one side doesn't support - mark both as not supporting */
@@ -266,7 +267,7 @@ rs_fw_he_set_enabled_rates(const struct ieee80211_sta *sta,
}
if (_mcs_160 > _tx_mcs_160)
_mcs_160 = _tx_mcs_160;
- cmd->ht_rates[i][1] =
+ cmd->ht_rates[i][IWL_TLC_HT_BW_160] =
cpu_to_le16(rs_fw_he_ieee80211_mcs_to_rs_mcs(_mcs_160));
}
}
@@ -300,8 +301,10 @@ static void rs_fw_set_supp_rates(struct ieee80211_sta *sta,
rs_fw_vht_set_enabled_rates(sta, vht_cap, cmd);
} else if (ht_cap->ht_supported) {
cmd->mode = IWL_TLC_MNG_MODE_HT;
- cmd->ht_rates[0][0] = cpu_to_le16(ht_cap->mcs.rx_mask[0]);
- cmd->ht_rates[1][0] = cpu_to_le16(ht_cap->mcs.rx_mask[1]);
+ cmd->ht_rates[IWL_TLC_NSS_1][IWL_TLC_HT_BW_NONE_160] =
+ cpu_to_le16(ht_cap->mcs.rx_mask[0]);
+ cmd->ht_rates[IWL_TLC_NSS_2][IWL_TLC_HT_BW_NONE_160] =
+ cpu_to_le16(ht_cap->mcs.rx_mask[1]);
}
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs.c
index e4415e5..42d525e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rs.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs.c
@@ -3079,11 +3079,7 @@ static void __iwl_mvm_rs_tx_status(struct iwl_mvm *mvm,
struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
struct iwl_lq_sta *lq_sta = &mvmsta->lq_sta.rs_drv;
- /* Treat uninitialized rate scaling data same as non-existing. */
- if (!lq_sta) {
- IWL_DEBUG_RATE(mvm, "Station rate scaling not created yet.\n");
- return;
- } else if (!lq_sta->pers.drv) {
+ if (!lq_sta->pers.drv) {
IWL_DEBUG_RATE(mvm, "Rate scaling not initialized yet.\n");
return;
}
@@ -3342,7 +3338,7 @@ static void rs_build_rates_table_from_fixed(struct iwl_mvm *mvm,
if (num_of_ant(ant) == 1)
lq_cmd->single_stream_ant_msk = ant;
- if (!mvm->trans->cfg->gen2)
+ if (!mvm->trans->trans_cfg->gen2)
lq_cmd->agg_frame_cnt_limit = LINK_QUAL_AGG_FRAME_LIMIT_DEF;
else
lq_cmd->agg_frame_cnt_limit =
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
index 160b0db..0ad8ed2 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
@@ -8,6 +8,7 @@
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 - 2019 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -29,6 +30,8 @@
*
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
+ * Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 - 2019 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -349,7 +352,6 @@ void iwl_mvm_rx_rx_mpdu(struct iwl_mvm *mvm, struct napi_struct *napi,
u32 rate_n_flags;
u32 rx_pkt_status;
u8 crypt_len = 0;
- bool take_ref;
phy_info = &mvm->last_phy_info;
rx_res = (struct iwl_rx_mpdu_res_start *)pkt->data;
@@ -557,22 +559,8 @@ void iwl_mvm_rx_rx_mpdu(struct iwl_mvm *mvm, struct napi_struct *napi,
ieee80211_is_probe_resp(hdr->frame_control)))
rx_status->boottime_ns = ktime_get_boottime_ns();
- /* Take a reference briefly to kick off a d0i3 entry delay so
- * we can handle bursts of RX packets without toggling the
- * state too often. But don't do this for beacons if we are
- * going to idle because the beacon filtering changes we make
- * cause the firmware to send us collateral beacons. */
- take_ref = !(test_bit(STATUS_TRANS_GOING_IDLE, &mvm->trans->status) &&
- ieee80211_is_beacon(hdr->frame_control));
-
- if (take_ref)
- iwl_mvm_ref(mvm, IWL_MVM_REF_RX);
-
iwl_mvm_pass_packet_to_mac80211(mvm, sta, napi, skb, hdr, len,
crypt_len, rxb);
-
- if (take_ref)
- iwl_mvm_unref(mvm, IWL_MVM_REF_RX);
}
struct iwl_mvm_stat_data {
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
index 854edd7..77b03b7 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
@@ -349,7 +349,7 @@ static int iwl_mvm_rx_crypto(struct iwl_mvm *mvm, struct ieee80211_hdr *hdr,
!(status & IWL_RX_MPDU_RES_STATUS_TTAK_OK))
return 0;
- if (mvm->trans->cfg->gen2 &&
+ if (mvm->trans->trans_cfg->gen2 &&
!(status & RX_MPDU_RES_STATUS_MIC_OK))
stats->flag |= RX_FLAG_MMIC_ERROR;
@@ -366,7 +366,7 @@ static int iwl_mvm_rx_crypto(struct iwl_mvm *mvm, struct ieee80211_hdr *hdr,
if (pkt_flags & FH_RSCSR_RADA_EN) {
stats->flag |= RX_FLAG_ICV_STRIPPED;
- if (mvm->trans->cfg->gen2)
+ if (mvm->trans->trans_cfg->gen2)
stats->flag |= RX_FLAG_MMIC_STRIPPED;
}
@@ -377,8 +377,16 @@ static int iwl_mvm_rx_crypto(struct iwl_mvm *mvm, struct ieee80211_hdr *hdr,
stats->flag |= RX_FLAG_DECRYPTED;
return 0;
default:
- /* Expected in monitor (not having the keys) */
- if (!mvm->monitor_on)
+ /*
+ * Sometimes we can get frames that were not decrypted
+ * because the firmware didn't have the keys yet. This can
+ * happen after connection where we can get multicast frames
+ * before the GTK is installed.
+ * Silently drop those frames.
+ * Also drop un-decrypted frames in monitor mode.
+ */
+ if (!is_multicast_ether_addr(hdr->addr1) &&
+ !mvm->monitor_on && net_ratelimit())
IWL_ERR(mvm, "Unhandled alg: 0x%x\n", status);
}
@@ -781,6 +789,55 @@ void iwl_mvm_rx_queue_notif(struct iwl_mvm *mvm, struct napi_struct *napi,
wake_up(&mvm->rx_sync_waitq);
}
+static void iwl_mvm_oldsn_workaround(struct iwl_mvm *mvm,
+ struct ieee80211_sta *sta, int tid,
+ struct iwl_mvm_reorder_buffer *buffer,
+ u32 reorder, u32 gp2, int queue)
+{
+ struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
+
+ if (gp2 != buffer->consec_oldsn_ampdu_gp2) {
+ /* we have a new (A-)MPDU ... */
+
+ /*
+ * reset counter to 0 if we didn't have any oldsn in
+ * the last A-MPDU (as detected by GP2 being identical)
+ */
+ if (!buffer->consec_oldsn_prev_drop)
+ buffer->consec_oldsn_drops = 0;
+
+ /* either way, update our tracking state */
+ buffer->consec_oldsn_ampdu_gp2 = gp2;
+ } else if (buffer->consec_oldsn_prev_drop) {
+ /*
+ * tracking state didn't change, and we had an old SN
+ * indication before - do nothing in this case, we
+ * already noted this one down and are waiting for the
+ * next A-MPDU (by GP2)
+ */
+ return;
+ }
+
+ /* return unless this MPDU has old SN */
+ if (!(reorder & IWL_RX_MPDU_REORDER_BA_OLD_SN))
+ return;
+
+ /* update state */
+ buffer->consec_oldsn_prev_drop = 1;
+ buffer->consec_oldsn_drops++;
+
+ /* if limit is reached, send del BA and reset state */
+ if (buffer->consec_oldsn_drops == IWL_MVM_AMPDU_CONSEC_DROPS_DELBA) {
+ IWL_WARN(mvm,
+ "reached %d old SN frames from %pM on queue %d, stopping BA session on TID %d\n",
+ IWL_MVM_AMPDU_CONSEC_DROPS_DELBA,
+ sta->addr, queue, tid);
+ ieee80211_stop_rx_ba_session(mvmsta->vif, BIT(tid), sta->addr);
+ buffer->consec_oldsn_prev_drop = 0;
+ buffer->consec_oldsn_drops = 0;
+ }
+}
+
/*
* Returns true if the MPDU was buffered\dropped, false if it should be passed
* to upper layer.
@@ -792,6 +849,7 @@ static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
struct sk_buff *skb,
struct iwl_rx_mpdu_desc *desc)
{
+ struct ieee80211_rx_status *rx_status = IEEE80211_SKB_RXCB(skb);
struct ieee80211_hdr *hdr = iwl_mvm_skb_get_hdr(skb);
struct iwl_mvm_sta *mvm_sta;
struct iwl_mvm_baid_data *baid_data;
@@ -894,6 +952,9 @@ static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
min_sn, IWL_MVM_RELEASE_SEND_RSS_SYNC);
}
+ iwl_mvm_oldsn_workaround(mvm, sta, tid, buffer, reorder,
+ rx_status->device_timestamp, queue);
+
/* drop any oudated packets */
if (ieee80211_sn_less(sn, buffer->head_sn))
goto drop;
@@ -1504,7 +1565,7 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
if (unlikely(test_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status)))
return;
- if (mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
rate_n_flags = le32_to_cpu(desc->v3.rate_n_flags);
channel = desc->v3.channel;
gp2_on_air_rise = le32_to_cpu(desc->v3.gp2_on_air_rise);
@@ -1605,7 +1666,8 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
if (likely(!(phy_info & IWL_RX_MPDU_PHY_TSF_OVERLOAD))) {
u64 tsf_on_air_rise;
- if (mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (mvm->trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_22560)
tsf_on_air_rise = le64_to_cpu(desc->v3.tsf_on_air_rise);
else
tsf_on_air_rise = le64_to_cpu(desc->v1.tsf_on_air_rise);
@@ -1731,7 +1793,7 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
*qc &= ~IEEE80211_QOS_CTL_A_MSDU_PRESENT;
- if (mvm->trans->cfg->device_family ==
+ if (mvm->trans->trans_cfg->device_family ==
IWL_DEVICE_FAMILY_9000) {
iwl_mvm_flip_address(hdr->addr3);
@@ -1960,3 +2022,42 @@ void iwl_mvm_rx_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
le16_to_cpu(release->nssn),
queue, 0);
}
+
+void iwl_mvm_rx_bar_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
+ struct iwl_rx_cmd_buffer *rxb, int queue)
+{
+ struct iwl_rx_packet *pkt = rxb_addr(rxb);
+ struct iwl_bar_frame_release *release = (void *)pkt->data;
+ unsigned int baid = le32_get_bits(release->ba_info,
+ IWL_BAR_FRAME_RELEASE_BAID_MASK);
+ unsigned int nssn = le32_get_bits(release->ba_info,
+ IWL_BAR_FRAME_RELEASE_NSSN_MASK);
+ unsigned int sta_id = le32_get_bits(release->sta_tid,
+ IWL_BAR_FRAME_RELEASE_STA_MASK);
+ unsigned int tid = le32_get_bits(release->sta_tid,
+ IWL_BAR_FRAME_RELEASE_TID_MASK);
+ struct iwl_mvm_baid_data *baid_data;
+
+ if (WARN_ON_ONCE(baid == IWL_RX_REORDER_DATA_INVALID_BAID ||
+ baid >= ARRAY_SIZE(mvm->baid_map)))
+ return;
+
+ rcu_read_lock();
+ baid_data = rcu_dereference(mvm->baid_map[baid]);
+ if (!baid_data) {
+ IWL_DEBUG_RX(mvm,
+ "Got valid BAID %d but not allocated, invalid BAR release!\n",
+ baid);
+ goto out;
+ }
+
+ if (WARN(tid != baid_data->tid || sta_id != baid_data->sta_id,
+ "baid 0x%x is mapped to sta:%d tid:%d, but BAR release received for sta:%d tid:%d\n",
+ baid, baid_data->sta_id, baid_data->tid, sta_id,
+ tid))
+ goto out;
+
+ iwl_mvm_release_frames_from_notif(mvm, napi, baid, nssn, queue, 0);
+out:
+ rcu_read_unlock();
+}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
index c284e69..f6b3045 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c
@@ -77,7 +77,10 @@
#define IWL_SCAN_DWELL_FRAGMENTED 44
#define IWL_SCAN_DWELL_EXTENDED 90
#define IWL_SCAN_NUM_OF_FRAGS 3
+#define IWL_SCAN_LAST_2_4_CHN 14
+#define IWL_SCAN_BAND_5_2 0
+#define IWL_SCAN_BAND_2_4 1
/* adaptive dwell max budget time [TU] for full scan */
#define IWL_SCAN_ADWELL_MAX_BUDGET_FULL_SCAN 300
@@ -512,7 +515,6 @@ void iwl_mvm_rx_lmac_scan_complete_notif(struct iwl_mvm *mvm,
mvm->scan_status &= ~IWL_MVM_SCAN_REGULAR;
ieee80211_scan_completed(mvm->hw, &info);
- iwl_mvm_unref(mvm, IWL_MVM_REF_SCAN);
cancel_delayed_work(&mvm->scan_timeout_dwork);
iwl_mvm_resume_tcm(mvm);
} else {
@@ -957,11 +959,24 @@ static int iwl_mvm_scan_lmac_flags(struct iwl_mvm *mvm,
return flags;
}
+static void
+iwl_mvm_scan_set_legacy_probe_req(struct iwl_scan_probe_req_v1 *p_req,
+ struct iwl_scan_probe_req *src_p_req)
+{
+ int i;
+
+ p_req->mac_header = src_p_req->mac_header;
+ for (i = 0; i < SCAN_NUM_BAND_PROBE_DATA_V_1; i++)
+ p_req->band_data[i] = src_p_req->band_data[i];
+ p_req->common_data = src_p_req->common_data;
+ memcpy(p_req->buf, src_p_req->buf, sizeof(p_req->buf));
+}
+
static int iwl_mvm_scan_lmac(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
struct iwl_mvm_scan_params *params)
{
struct iwl_scan_req_lmac *cmd = mvm->scan_cmd;
- struct iwl_scan_probe_req *preq =
+ struct iwl_scan_probe_req_v1 *preq =
(void *)(cmd->data + sizeof(struct iwl_scan_channel_cfg_lmac) *
mvm->fw->ucode_capa.n_scan_channels);
u32 ssid_bitmap = 0;
@@ -1031,7 +1046,7 @@ static int iwl_mvm_scan_lmac(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
iwl_mvm_lmac_scan_cfg_channels(mvm, params->channels,
params->n_channels, ssid_bitmap, cmd);
- *preq = params->preq;
+ iwl_mvm_scan_set_legacy_probe_req(preq, ¶ms->preq);
return 0;
}
@@ -1205,7 +1220,7 @@ int iwl_mvm_config_scan(struct iwl_mvm *mvm)
cmd_size = sizeof(struct iwl_scan_config);
else
cmd_size = sizeof(struct iwl_scan_config_v1);
- cmd_size += mvm->fw->ucode_capa.n_scan_channels;
+ cmd_size += num_channels;
cfg = kzalloc(cmd_size, GFP_KERNEL);
if (!cfg)
@@ -1385,9 +1400,17 @@ iwl_mvm_umac_scan_cfg_channels(struct iwl_mvm *mvm,
for (i = 0; i < n_channels; i++) {
channel_cfg[i].flags = cpu_to_le32(ssid_bitmap);
- channel_cfg[i].channel_num = channels[i]->hw_value;
- channel_cfg[i].iter_count = 1;
- channel_cfg[i].iter_interval = 0;
+ channel_cfg[i].v1.channel_num = channels[i]->hw_value;
+ if (iwl_mvm_is_scan_ext_chan_supported(mvm)) {
+ channel_cfg[i].v2.band =
+ channels[i]->hw_value <= IWL_SCAN_LAST_2_4_CHN ?
+ IWL_SCAN_BAND_2_4 : IWL_SCAN_BAND_5_2;
+ channel_cfg[i].v2.iter_count = 1;
+ channel_cfg[i].v2.iter_interval = 0;
+ } else {
+ channel_cfg[i].v1.iter_count = 1;
+ channel_cfg[i].v1.iter_interval = 0;
+ }
}
}
@@ -1477,9 +1500,12 @@ static int iwl_mvm_scan_umac(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
struct iwl_scan_req_umac *cmd = mvm->scan_cmd;
struct iwl_scan_umac_chan_param *chan_param;
void *cmd_data = iwl_mvm_get_scan_req_umac_data(mvm);
- struct iwl_scan_req_umac_tail *sec_part = cmd_data +
- sizeof(struct iwl_scan_channel_cfg_umac) *
- mvm->fw->ucode_capa.n_scan_channels;
+ void *sec_part = cmd_data + sizeof(struct iwl_scan_channel_cfg_umac) *
+ mvm->fw->ucode_capa.n_scan_channels;
+ struct iwl_scan_req_umac_tail_v2 *tail_v2 =
+ (struct iwl_scan_req_umac_tail_v2 *)sec_part;
+ struct iwl_scan_req_umac_tail_v1 *tail_v1;
+ struct iwl_ssid_ie *direct_scan;
int uid, i;
u32 ssid_bitmap = 0;
u8 channel_flags = 0;
@@ -1541,18 +1567,12 @@ static int iwl_mvm_scan_umac(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
chan_param->flags = channel_flags;
chan_param->count = params->n_channels;
- iwl_scan_build_ssids(params, sec_part->direct_scan, &ssid_bitmap);
-
- iwl_mvm_umac_scan_cfg_channels(mvm, params->channels,
- params->n_channels, ssid_bitmap,
- cmd_data);
-
for (i = 0; i < params->n_scan_plans; i++) {
struct cfg80211_sched_scan_plan *scan_plan =
¶ms->scan_plans[i];
- sec_part->schedule[i].iter_count = scan_plan->iterations;
- sec_part->schedule[i].interval =
+ tail_v2->schedule[i].iter_count = scan_plan->iterations;
+ tail_v2->schedule[i].interval =
cpu_to_le16(scan_plan->interval);
}
@@ -1562,12 +1582,24 @@ static int iwl_mvm_scan_umac(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
* For example, when regular scan is requested the driver sets one scan
* plan with one iteration.
*/
- if (!sec_part->schedule[i - 1].iter_count)
- sec_part->schedule[i - 1].iter_count = 0xff;
+ if (!tail_v2->schedule[i - 1].iter_count)
+ tail_v2->schedule[i - 1].iter_count = 0xff;
- sec_part->delay = cpu_to_le16(params->delay);
- sec_part->preq = params->preq;
+ tail_v2->delay = cpu_to_le16(params->delay);
+ if (iwl_mvm_is_scan_ext_chan_supported(mvm)) {
+ tail_v2->preq = params->preq;
+ direct_scan = tail_v2->direct_scan;
+ } else {
+ tail_v1 = (struct iwl_scan_req_umac_tail_v1 *)sec_part;
+ iwl_mvm_scan_set_legacy_probe_req(&tail_v1->preq,
+ ¶ms->preq);
+ direct_scan = tail_v1->direct_scan;
+ }
+ iwl_scan_build_ssids(params, direct_scan, &ssid_bitmap);
+ iwl_mvm_umac_scan_cfg_channels(mvm, params->channels,
+ params->n_channels, ssid_bitmap,
+ cmd_data);
return 0;
}
@@ -1758,7 +1790,6 @@ int iwl_mvm_reg_scan_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
IWL_DEBUG_SCAN(mvm, "Scan request was sent successfully\n");
mvm->scan_status |= IWL_MVM_SCAN_REGULAR;
mvm->scan_vif = iwl_mvm_vif_from_mac80211(vif);
- iwl_mvm_ref(mvm, IWL_MVM_REF_SCAN);
schedule_delayed_work(&mvm->scan_timeout_dwork,
msecs_to_jiffies(SCAN_TIMEOUT));
@@ -1884,7 +1915,6 @@ void iwl_mvm_rx_umac_scan_complete_notif(struct iwl_mvm *mvm,
memcpy(info.tsf_bssid, mvm->scan_vif->bssid, ETH_ALEN);
ieee80211_scan_completed(mvm->hw, &info);
mvm->scan_vif = NULL;
- iwl_mvm_unref(mvm, IWL_MVM_REF_SCAN);
cancel_delayed_work(&mvm->scan_timeout_dwork);
iwl_mvm_resume_tcm(mvm);
} else if (mvm->scan_uid_status[uid] == IWL_MVM_SCAN_SCHED) {
@@ -1909,8 +1939,6 @@ void iwl_mvm_rx_umac_scan_complete_notif(struct iwl_mvm *mvm,
mvm->last_ebs_successful = false;
mvm->scan_uid_status[uid] = 0;
-
- iwl_fw_dbg_apply_point(&mvm->fwrt, IWL_FW_INI_APPLY_SCAN_COMPLETE);
}
void iwl_mvm_rx_umac_scan_iter_complete_notif(struct iwl_mvm *mvm,
@@ -1999,6 +2027,7 @@ static int iwl_mvm_scan_stop_wait(struct iwl_mvm *mvm, int type)
int iwl_mvm_scan_size(struct iwl_mvm *mvm)
{
int base_size = IWL_SCAN_REQ_UMAC_SIZE_V1;
+ int tail_size;
if (iwl_mvm_is_adaptive_dwell_v2_supported(mvm))
base_size = IWL_SCAN_REQ_UMAC_SIZE_V8;
@@ -2007,16 +2036,21 @@ int iwl_mvm_scan_size(struct iwl_mvm *mvm)
else if (iwl_mvm_cdb_scan_api(mvm))
base_size = IWL_SCAN_REQ_UMAC_SIZE_V6;
- if (fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_UMAC_SCAN))
+ if (fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_UMAC_SCAN)) {
+ if (iwl_mvm_is_scan_ext_chan_supported(mvm))
+ tail_size = sizeof(struct iwl_scan_req_umac_tail_v2);
+ else
+ tail_size = sizeof(struct iwl_scan_req_umac_tail_v1);
+
return base_size +
sizeof(struct iwl_scan_channel_cfg_umac) *
mvm->fw->ucode_capa.n_scan_channels +
- sizeof(struct iwl_scan_req_umac_tail);
-
+ tail_size;
+ }
return sizeof(struct iwl_scan_req_lmac) +
sizeof(struct iwl_scan_channel_cfg_lmac) *
mvm->fw->ucode_capa.n_scan_channels +
- sizeof(struct iwl_scan_probe_req);
+ sizeof(struct iwl_scan_probe_req_v1);
}
/*
@@ -2099,10 +2133,6 @@ int iwl_mvm_scan_stop(struct iwl_mvm *mvm, int type, bool notify)
mvm->scan_status &= ~type;
if (type == IWL_MVM_SCAN_REGULAR) {
- /* Since the rx handler won't do anything now, we have
- * to release the scan reference here.
- */
- iwl_mvm_unref(mvm, IWL_MVM_REF_SCAN);
cancel_delayed_work(&mvm->scan_timeout_dwork);
if (notify) {
struct cfg80211_scan_info info = {
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
index 10f1853..0bedba4 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
@@ -67,14 +67,6 @@
#include "sta.h"
#include "rs.h"
-static int iwl_mvm_set_fw_key_idx(struct iwl_mvm *mvm);
-
-static int iwl_mvm_send_sta_key(struct iwl_mvm *mvm,
- u32 sta_id,
- struct ieee80211_key_conf *key, bool mcast,
- u32 tkip_iv32, u16 *tkip_p1k, u32 cmd_flags,
- u8 key_offset, bool mfp);
-
/*
* New version of ADD_STA_sta command added new fields at the end of the
* structure, so sending the size of the relevant API's structure is enough to
@@ -1612,7 +1604,7 @@ int iwl_mvm_add_sta(struct iwl_mvm *mvm,
mvm_sta->mac_id_n_color = FW_CMD_ID_AND_COLOR(mvmvif->id,
mvmvif->color);
mvm_sta->vif = vif;
- if (!mvm->trans->cfg->gen2)
+ if (!mvm->trans->trans_cfg->gen2)
mvm_sta->max_agg_bufsize = LINK_QUAL_AGG_FRAME_LIMIT_DEF;
else
mvm_sta->max_agg_bufsize = LINK_QUAL_AGG_FRAME_LIMIT_GEN2_DEF;
@@ -1895,10 +1887,6 @@ int iwl_mvm_rm_sta(struct iwl_mvm *mvm,
/* unassoc - go ahead - remove the AP STA now */
mvmvif->ap_sta_id = IWL_MVM_INVALID_STA;
-
- /* clear d0i3_ap_sta_id if no longer relevant */
- if (mvm->d0i3_ap_sta_id == sta_id)
- mvm->d0i3_ap_sta_id = IWL_MVM_INVALID_STA;
}
/*
@@ -1966,8 +1954,8 @@ static void iwl_mvm_enable_aux_snif_queue(struct iwl_mvm *mvm, u16 *queue,
u8 sta_id, u8 fifo)
{
unsigned int wdg_timeout = iwlmvm_mod_params.tfd_q_hang_detect ?
- mvm->cfg->base_params->wd_timeout :
- IWL_WATCHDOG_DISABLED;
+ mvm->trans->trans_cfg->base_params->wd_timeout :
+ IWL_WATCHDOG_DISABLED;
if (iwl_mvm_has_new_tx_api(mvm)) {
int tvqm_queue =
@@ -2771,13 +2759,6 @@ int iwl_mvm_sta_tx_agg_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
spin_lock_bh(&mvmsta->lock);
- /* possible race condition - we entered D0i3 while starting agg */
- if (test_bit(IWL_MVM_STATUS_IN_D0I3, &mvm->status)) {
- spin_unlock_bh(&mvmsta->lock);
- IWL_ERR(mvm, "Entered D0i3 while starting Tx agg\n");
- return -EIO;
- }
-
/*
* Note the possible cases:
* 1. An enabled TXQ - TXQ needs to become agg'ed
@@ -2832,7 +2813,7 @@ int iwl_mvm_sta_tx_agg_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
* to align the wrap around of ssn so we compare relevant values.
*/
normalized_ssn = tid_data->ssn;
- if (mvm->trans->cfg->gen2)
+ if (mvm->trans->trans_cfg->gen2)
normalized_ssn &= 0xff;
if (normalized_ssn == tid_data->next_reclaimed) {
@@ -3872,7 +3853,7 @@ u16 iwl_mvm_tid_queued(struct iwl_mvm *mvm, struct iwl_mvm_tid_data *tid_data)
* In 22000 HW, the next_reclaimed index is only 8 bit, so we'll need
* to align the wrap around of ssn so we compare relevant values.
*/
- if (mvm->trans->cfg->gen2)
+ if (mvm->trans->trans_cfg->gen2)
sn &= 0xff;
return ieee80211_sn_sub(sn, tid_data->next_reclaimed);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c b/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c
index 9df21a8..1851719 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c
@@ -205,19 +205,10 @@ void iwl_mvm_mac_mgd_protect_tdls_discover(struct ieee80211_hw *hw,
struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
u32 duration = 2 * vif->bss_conf.dtim_period * vif->bss_conf.beacon_int;
- /*
- * iwl_mvm_protect_session() reads directly from the device
- * (the system time), so make sure it is available.
- */
- if (iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PROTECT_TDLS))
- return;
-
mutex_lock(&mvm->mutex);
/* Protect the session to hear the TDLS setup response on the channel */
iwl_mvm_protect_session(mvm, vif, duration, duration, 100, true);
mutex_unlock(&mvm->mutex);
-
- iwl_mvm_unref(mvm, IWL_MVM_REF_PROTECT_TDLS);
}
static const char *
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c b/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c
index 4d34e5a..a06bc63 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/time-event.c
@@ -106,10 +106,8 @@ void iwl_mvm_roc_done_wk(struct work_struct *wk)
* in the case that the time event actually completed in the firmware
* (which is handled in iwl_mvm_te_handle_notif).
*/
- if (test_and_clear_bit(IWL_MVM_STATUS_ROC_RUNNING, &mvm->status))
- iwl_mvm_unref(mvm, IWL_MVM_REF_ROC);
- if (test_and_clear_bit(IWL_MVM_STATUS_ROC_AUX_RUNNING, &mvm->status))
- iwl_mvm_unref(mvm, IWL_MVM_REF_ROC_AUX);
+ clear_bit(IWL_MVM_STATUS_ROC_RUNNING, &mvm->status);
+ clear_bit(IWL_MVM_STATUS_ROC_AUX_RUNNING, &mvm->status);
synchronize_net();
@@ -357,7 +355,6 @@ static void iwl_mvm_te_handle_notif(struct iwl_mvm *mvm,
if (te_data->vif->type == NL80211_IFTYPE_P2P_DEVICE) {
set_bit(IWL_MVM_STATUS_ROC_RUNNING, &mvm->status);
- iwl_mvm_ref(mvm, IWL_MVM_REF_ROC);
ieee80211_ready_on_channel(mvm->hw);
} else if (te_data->id == TE_CHANNEL_SWITCH_PERIOD) {
iwl_mvm_te_handle_notify_csa(mvm, te_data, notif);
@@ -405,7 +402,6 @@ static int iwl_mvm_aux_roc_te_handle_notif(struct iwl_mvm *mvm,
} else if (le32_to_cpu(notif->action) == TE_V2_NOTIF_HOST_EVENT_START) {
set_bit(IWL_MVM_STATUS_ROC_AUX_RUNNING, &mvm->status);
te_data->running = true;
- iwl_mvm_ref(mvm, IWL_MVM_REF_ROC_AUX);
ieee80211_ready_on_channel(mvm->hw); /* Start TE */
} else {
IWL_DEBUG_TE(mvm,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tt.c b/drivers/net/wireless/intel/iwlwifi/mvm/tt.c
index 0b3e5c9..32a7083 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tt.c
@@ -5,7 +5,7 @@
*
* GPL LICENSE SUMMARY
*
- * Copyright(c) 2013 - 2014 Intel Corporation. All rights reserved.
+ * Copyright(c) 2013 - 2014, 2019 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2015 - 2016 Intel Deutschland GmbH
*
@@ -27,7 +27,7 @@
*
* BSD LICENSE
*
- * Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
+ * Copyright(c) 2012 - 2014, 2019 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2014 Intel Mobile Communications GmbH
* Copyright(c) 2015 - 2016 Intel Deutschland GmbH
* All rights reserved.
@@ -298,16 +298,8 @@ static void check_exit_ctkill(struct work_struct *work)
if (__iwl_mvm_mac_start(mvm))
goto reschedule;
- /* make sure the device is available for direct read/writes */
- if (iwl_mvm_ref_sync(mvm, IWL_MVM_REF_CHECK_CTKILL)) {
- __iwl_mvm_mac_stop(mvm);
- goto reschedule;
- }
-
ret = iwl_mvm_get_temp(mvm, &temp);
- iwl_mvm_unref(mvm, IWL_MVM_REF_CHECK_CTKILL);
-
__iwl_mvm_mac_stop(mvm);
if (ret)
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
index 6ac114a..8a059da 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
@@ -546,7 +546,7 @@ iwl_mvm_set_tx_params(struct iwl_mvm *mvm, struct sk_buff *skb,
hdr->frame_control);
}
- if (mvm->trans->cfg->device_family >=
+ if (mvm->trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560) {
struct iwl_tx_cmd_gen3 *cmd = (void *)dev_cmd->payload;
@@ -1169,8 +1169,9 @@ static int iwl_mvm_tx_mpdu(struct iwl_mvm *mvm, struct sk_buff *skb,
schedule_work(&mvm->add_stream_wk);
}
- IWL_DEBUG_TX(mvm, "TX to [%d|%d] Q:%d - seq: 0x%x\n", mvmsta->sta_id,
- tid, txq_id, IEEE80211_SEQ_TO_SN(seq_number));
+ IWL_DEBUG_TX(mvm, "TX to [%d|%d] Q:%d - seq: 0x%x len %d\n",
+ mvmsta->sta_id, tid, txq_id,
+ IEEE80211_SEQ_TO_SN(seq_number), skb->len);
/* From now on, we cannot access info->control */
iwl_mvm_skb_prepare_status(skb, dev_cmd);
@@ -1271,7 +1272,7 @@ static void iwl_mvm_check_ratid_empty(struct iwl_mvm *mvm,
* to align the wrap around of ssn so we compare relevant values.
*/
normalized_ssn = tid_data->ssn;
- if (mvm->trans->cfg->gen2)
+ if (mvm->trans->trans_cfg->gen2)
normalized_ssn &= 0xff;
if (normalized_ssn != tid_data->next_reclaimed)
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
index b8e20a0..8686107 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
@@ -88,17 +88,11 @@ int iwl_mvm_send_cmd(struct iwl_mvm *mvm, struct iwl_host_cmd *cmd)
* the mutex, this ensures we don't try to send two
* (or more) synchronous commands at a time.
*/
- if (!(cmd->flags & CMD_ASYNC)) {
+ if (!(cmd->flags & CMD_ASYNC))
lockdep_assert_held(&mvm->mutex);
- if (!(cmd->flags & CMD_SEND_IN_IDLE))
- iwl_mvm_ref(mvm, IWL_MVM_REF_SENDING_CMD);
- }
ret = iwl_trans_send_cmd(mvm->trans, cmd);
- if (!(cmd->flags & (CMD_ASYNC | CMD_SEND_IN_IDLE)))
- iwl_mvm_unref(mvm, IWL_MVM_REF_SENDING_CMD);
-
/*
* If the caller wants the SKB, then don't hide any problems, the
* caller might access the response buffer which will be NULL if
@@ -537,7 +531,7 @@ static void iwl_mvm_dump_lmac_error_log(struct iwl_mvm *mvm, u8 lmac_num)
/* reset the device */
iwl_trans_sw_reset(trans);
- err = iwl_finish_nic_init(trans);
+ err = iwl_finish_nic_init(trans, trans->trans_cfg);
if (err)
return;
}
@@ -945,8 +939,9 @@ unsigned int iwl_mvm_get_wd_timeout(struct iwl_mvm *mvm,
{
struct iwl_fw_dbg_trigger_tlv *trigger;
struct iwl_fw_dbg_trigger_txq_timer *txq_timer;
- unsigned int default_timeout =
- cmd_q ? IWL_DEF_WD_TIMEOUT : mvm->cfg->base_params->wd_timeout;
+ unsigned int default_timeout = cmd_q ?
+ IWL_DEF_WD_TIMEOUT :
+ mvm->trans->trans_cfg->base_params->wd_timeout;
if (!iwl_fw_dbg_trigger_enabled(mvm->fw, FW_DBG_TRIGGER_TXQ_TIMERS)) {
/*
@@ -990,7 +985,7 @@ unsigned int iwl_mvm_get_wd_timeout(struct iwl_mvm *mvm,
return default_timeout;
default:
WARN_ON(1);
- return mvm->cfg->base_params->wd_timeout;
+ return mvm->trans->trans_cfg->base_params->wd_timeout;
}
}
@@ -1436,7 +1431,7 @@ u32 iwl_mvm_get_systime(struct iwl_mvm *mvm)
{
u32 reg_addr = DEVICE_SYSTEM_TIME_REG;
- if (mvm->trans->cfg->device_family >= IWL_DEVICE_FAMILY_22000 &&
+ if (mvm->trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22000 &&
mvm->trans->cfg->gp2_reg_addr)
reg_addr = mvm->trans->cfg->gp2_reg_addr;
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/ctxt-info-gen3.c b/drivers/net/wireless/intel/iwlwifi/pcie/ctxt-info-gen3.c
index 5e86783..75fa8a6 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/ctxt-info-gen3.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/ctxt-info-gen3.c
@@ -96,7 +96,7 @@ int iwl_pcie_ctxt_info_gen3_init(struct iwl_trans *trans,
cpu_to_le64(trans_pcie->rxq->bd_dma);
/* Configure debug, for integration */
- if (!trans->dbg.ini_valid)
+ if (!iwl_trans_dbg_ini_valid(trans))
iwl_pcie_alloc_fw_monitor(trans, 0);
if (trans->dbg.num_blocks) {
prph_sc_ctrl->hwm_cfg.hwm_base_addr =
@@ -180,7 +180,7 @@ int iwl_pcie_ctxt_info_gen3_init(struct iwl_trans *trans,
iwl_set_bit(trans, CSR_CTXT_INFO_BOOT_CTRL,
CSR_AUTO_FUNC_BOOT_ENA);
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210)
iwl_write_umac_prph(trans, UREG_CPU_INIT_RUN, 1);
else
iwl_set_bit(trans, CSR_GP_CNTRL, CSR_AUTO_FUNC_INIT);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
index de711c1..e5ca1f2 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
@@ -65,7 +65,6 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
-#include <linux/pm_runtime.h>
#include <linux/pci.h>
#include <linux/acpi.h>
@@ -73,6 +72,7 @@
#include "iwl-trans.h"
#include "iwl-drv.h"
+#include "iwl-prph.h"
#include "internal.h"
#define IWL_PCI_DEVICE(dev, subdev, cfg) \
@@ -994,15 +994,22 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
const struct iwl_cfg *cfg = (struct iwl_cfg *)(ent->driver_data);
const struct iwl_cfg *cfg_7265d __maybe_unused = NULL;
struct iwl_trans *iwl_trans;
+ unsigned long flags;
int ret;
- if (WARN_ONCE(!cfg->csr, "CSR addresses aren't configured\n"))
- return -EINVAL;
-
- iwl_trans = iwl_trans_pcie_alloc(pdev, ent, cfg);
+ iwl_trans = iwl_trans_pcie_alloc(pdev, ent, &cfg->trans);
if (IS_ERR(iwl_trans))
return PTR_ERR(iwl_trans);
+ /* the trans_cfg should never change, so set it now */
+ iwl_trans->trans_cfg = &cfg->trans;
+
+ if (WARN_ONCE(!iwl_trans->trans_cfg->csr,
+ "CSR addresses aren't configured\n")) {
+ ret = -EINVAL;
+ goto out_free_trans;
+ }
+
#if IS_ENABLED(CONFIG_IWLMVM)
/*
* special-case 7265D, it has the same PCI IDs.
@@ -1018,29 +1025,70 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
else if (cfg == &iwl7265_n_cfg)
cfg_7265d = &iwl7265d_n_cfg;
if (cfg_7265d &&
- (iwl_trans->hw_rev & CSR_HW_REV_TYPE_MSK) == CSR_HW_REV_TYPE_7265D) {
+ (iwl_trans->hw_rev & CSR_HW_REV_TYPE_MSK) == CSR_HW_REV_TYPE_7265D)
cfg = cfg_7265d;
- iwl_trans->cfg = cfg_7265d;
- }
- if (iwl_trans->cfg->rf_id && cfg == &iwl22000_2ac_cfg_hr_cdb &&
- iwl_trans->hw_rev != CSR_HW_REV_TYPE_HR_CDB) {
- u32 rf_id_chp = CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id);
- u32 jf_chp_id = CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_JF);
- u32 hr_chp_id = CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR);
+ iwl_trans->hw_rf_id = iwl_read32(iwl_trans, CSR_HW_RF_ID);
- if (rf_id_chp == jf_chp_id) {
- if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_QNJ)
- cfg = &iwl9560_2ac_cfg_qnj_jf_b0;
- else
- cfg = &iwl22000_2ac_cfg_jf;
- } else if (rf_id_chp == hr_chp_id) {
- if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_QNJ)
- cfg = &iwl22000_2ax_cfg_qnj_hr_a0;
- else
- cfg = &iwl22000_2ac_cfg_hr;
+ if (cfg == &iwlax210_2ax_cfg_so_hr_a0) {
+ if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_TY) {
+ cfg = &iwlax210_2ax_cfg_ty_gf_a0;
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_JF)) {
+ cfg = &iwlax210_2ax_cfg_so_jf_a0;
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_GF)) {
+ cfg = &iwlax211_2ax_cfg_so_gf_a0;
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_GF4)) {
+ cfg = &iwlax411_2ax_cfg_so_gf4_a0;
}
- iwl_trans->cfg = cfg;
+ } else if (cfg == &iwl_ax101_cfg_qu_hr) {
+ if ((CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR) &&
+ iwl_trans->hw_rev == CSR_HW_REV_TYPE_QNJ_B0) ||
+ (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR1))) {
+ cfg = &iwl22000_2ax_cfg_qnj_hr_b0;
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR)) {
+ cfg = &iwl_ax101_cfg_qu_hr;
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_JF)) {
+ cfg = &iwl22000_2ax_cfg_jf;
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HRCDB)) {
+ IWL_ERR(iwl_trans, "RF ID HRCDB is not supported\n");
+ return -EINVAL;
+ } else {
+ IWL_ERR(iwl_trans, "Unrecognized RF ID 0x%08x\n",
+ CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id));
+ return -EINVAL;
+ }
+ } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(iwl_trans->hw_rf_id) ==
+ CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR) &&
+ ((cfg != &iwl_ax200_cfg_cc &&
+ cfg != &killer1650x_2ax_cfg &&
+ cfg != &killer1650w_2ax_cfg &&
+ cfg != &iwl_ax201_cfg_quz_hr) ||
+ iwl_trans->hw_rev == CSR_HW_REV_TYPE_QNJ_B0)) {
+ u32 hw_status;
+
+ hw_status = iwl_read_prph(iwl_trans, UMAG_GEN_HW_STATUS);
+ if (CSR_HW_RF_STEP(iwl_trans->hw_rf_id) == SILICON_B_STEP)
+ /*
+ * b step fw is the same for physical card and fpga
+ */
+ cfg = &iwl22000_2ax_cfg_qnj_hr_b0;
+ else if ((hw_status & UMAG_GEN_HW_IS_FPGA) &&
+ CSR_HW_RF_STEP(iwl_trans->hw_rf_id) == SILICON_A_STEP) {
+ cfg = &iwl22000_2ax_cfg_qnj_hr_a0_f0;
+ } else {
+ /*
+ * a step no FPGA
+ */
+ cfg = &iwl22000_2ac_cfg_hr;
+ }
}
/*
@@ -1050,20 +1098,54 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
* thing to do to support Qu C-step.
*/
if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_QU_C0) {
- if (iwl_trans->cfg == &iwl_ax101_cfg_qu_hr)
- iwl_trans->cfg = &iwl_ax101_cfg_qu_c0_hr_b0;
- else if (iwl_trans->cfg == &iwl_ax201_cfg_qu_hr)
- iwl_trans->cfg = &iwl_ax201_cfg_qu_c0_hr_b0;
- else if (iwl_trans->cfg == &iwl9461_2ac_cfg_qu_b0_jf_b0)
- iwl_trans->cfg = &iwl9461_2ac_cfg_qu_c0_jf_b0;
- else if (iwl_trans->cfg == &iwl9462_2ac_cfg_qu_b0_jf_b0)
- iwl_trans->cfg = &iwl9462_2ac_cfg_qu_c0_jf_b0;
- else if (iwl_trans->cfg == &iwl9560_2ac_cfg_qu_b0_jf_b0)
- iwl_trans->cfg = &iwl9560_2ac_cfg_qu_c0_jf_b0;
- else if (iwl_trans->cfg == &iwl9560_2ac_160_cfg_qu_b0_jf_b0)
- iwl_trans->cfg = &iwl9560_2ac_160_cfg_qu_c0_jf_b0;
+ if (cfg == &iwl_ax101_cfg_qu_hr)
+ cfg = &iwl_ax101_cfg_qu_c0_hr_b0;
+ else if (cfg == &iwl_ax201_cfg_qu_hr)
+ cfg = &iwl_ax201_cfg_qu_c0_hr_b0;
+ else if (cfg == &iwl9461_2ac_cfg_qu_b0_jf_b0)
+ cfg = &iwl9461_2ac_cfg_qu_c0_jf_b0;
+ else if (cfg == &iwl9462_2ac_cfg_qu_b0_jf_b0)
+ cfg = &iwl9462_2ac_cfg_qu_c0_jf_b0;
+ else if (cfg == &iwl9560_2ac_cfg_qu_b0_jf_b0)
+ cfg = &iwl9560_2ac_cfg_qu_c0_jf_b0;
+ else if (cfg == &iwl9560_2ac_160_cfg_qu_b0_jf_b0)
+ cfg = &iwl9560_2ac_160_cfg_qu_c0_jf_b0;
}
+
+ /* same thing for QuZ... */
+ if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_QUZ) {
+ if (cfg == &iwl_ax101_cfg_qu_hr)
+ cfg = &iwl_ax101_cfg_quz_hr;
+ else if (cfg == &iwl_ax201_cfg_qu_hr)
+ cfg = &iwl_ax201_cfg_quz_hr;
+ else if (cfg == &iwl9461_2ac_cfg_qu_b0_jf_b0)
+ cfg = &iwl9461_2ac_cfg_quz_a0_jf_b0_soc;
+ else if (cfg == &iwl9462_2ac_cfg_qu_b0_jf_b0)
+ cfg = &iwl9462_2ac_cfg_quz_a0_jf_b0_soc;
+ else if (cfg == &iwl9560_2ac_cfg_qu_b0_jf_b0)
+ cfg = &iwl9560_2ac_cfg_quz_a0_jf_b0_soc;
+ else if (cfg == &iwl9560_2ac_160_cfg_qu_b0_jf_b0)
+ cfg = &iwl9560_2ac_160_cfg_quz_a0_jf_b0_soc;
+ }
+
#endif
+ /* now set the real cfg we decided to use */
+ iwl_trans->cfg = cfg;
+
+ if (iwl_trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_8000 &&
+ iwl_trans_grab_nic_access(iwl_trans, &flags)) {
+ u32 hw_step;
+
+ hw_step = iwl_read_umac_prph_no_grab(iwl_trans, WFPM_CTRL_REG);
+ hw_step |= ENABLE_WFPM;
+ iwl_write_umac_prph_no_grab(iwl_trans, WFPM_CTRL_REG, hw_step);
+ hw_step = iwl_read_prph_no_grab(iwl_trans, CNVI_AUX_MISC_CHIP);
+ hw_step = (hw_step >> HW_STEP_LOCATION_BITS) & 0xF;
+ if (hw_step == 0x3)
+ iwl_trans->hw_rev = (iwl_trans->hw_rev & 0xFFFFFFF3) |
+ (SILICON_C_STEP << 2);
+ iwl_trans_release_nic_access(iwl_trans, &flags);
+ }
pci_set_drvdata(pdev, iwl_trans);
iwl_trans->drv = iwl_drv_start(iwl_trans);
@@ -1076,25 +1158,6 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* register transport layer debugfs here */
iwl_trans_pcie_dbgfs_register(iwl_trans);
- /* if RTPM is in use, enable it in our device */
- if (iwl_trans->runtime_pm_mode != IWL_PLAT_PM_MODE_DISABLED) {
- /* We explicitly set the device to active here to
- * clear contingent errors.
- */
- pm_runtime_set_active(&pdev->dev);
-
- pm_runtime_set_autosuspend_delay(&pdev->dev,
- iwlwifi_mod_params.d0i3_timeout);
- pm_runtime_use_autosuspend(&pdev->dev);
-
- /* We are not supposed to call pm_runtime_allow() by
- * ourselves, but let userspace enable runtime PM via
- * sysfs. However, since we don't enable this from
- * userspace yet, we need to allow/forbid() ourselves.
- */
- pm_runtime_allow(&pdev->dev);
- }
-
/* The PCI device starts with a reference taken and we are
* supposed to release it here. But to simplify the
* interaction with the opmode, we don't do it now, but let
@@ -1112,15 +1175,6 @@ static void iwl_pci_remove(struct pci_dev *pdev)
{
struct iwl_trans *trans = pci_get_drvdata(pdev);
- /* if RTPM was in use, restore it to the state before probe */
- if (trans->runtime_pm_mode != IWL_PLAT_PM_MODE_DISABLED) {
- /* We should not call forbid here, but we do for now.
- * Check the comment to pm_runtime_allow() in
- * iwl_pci_probe().
- */
- pm_runtime_forbid(trans->dev);
- }
-
iwl_drv_stop(trans->drv);
iwl_trans_pcie_free(trans);
@@ -1178,164 +1232,9 @@ static int iwl_pci_resume(struct device *device)
return 0;
}
-int iwl_pci_fw_enter_d0i3(struct iwl_trans *trans)
-{
- struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- int ret;
-
- if (test_bit(STATUS_FW_ERROR, &trans->status))
- return 0;
-
- set_bit(STATUS_TRANS_GOING_IDLE, &trans->status);
-
- /* config the fw */
- ret = iwl_op_mode_enter_d0i3(trans->op_mode);
- if (ret == 1) {
- IWL_DEBUG_RPM(trans, "aborting d0i3 entrance\n");
- clear_bit(STATUS_TRANS_GOING_IDLE, &trans->status);
- return -EBUSY;
- }
- if (ret)
- goto err;
-
- ret = wait_event_timeout(trans_pcie->d0i3_waitq,
- test_bit(STATUS_TRANS_IDLE, &trans->status),
- msecs_to_jiffies(IWL_TRANS_IDLE_TIMEOUT));
- if (!ret) {
- IWL_ERR(trans, "Timeout entering D0i3\n");
- ret = -ETIMEDOUT;
- goto err;
- }
-
- clear_bit(STATUS_TRANS_GOING_IDLE, &trans->status);
-
- return 0;
-err:
- clear_bit(STATUS_TRANS_GOING_IDLE, &trans->status);
- iwl_trans_fw_error(trans);
- return ret;
-}
-
-int iwl_pci_fw_exit_d0i3(struct iwl_trans *trans)
-{
- struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- int ret;
-
- /* sometimes a D0i3 entry is not followed through */
- if (!test_bit(STATUS_TRANS_IDLE, &trans->status))
- return 0;
-
- /* config the fw */
- ret = iwl_op_mode_exit_d0i3(trans->op_mode);
- if (ret)
- goto err;
-
- /* we clear STATUS_TRANS_IDLE only when D0I3_END command is completed */
-
- ret = wait_event_timeout(trans_pcie->d0i3_waitq,
- !test_bit(STATUS_TRANS_IDLE, &trans->status),
- msecs_to_jiffies(IWL_TRANS_IDLE_TIMEOUT));
- if (!ret) {
- IWL_ERR(trans, "Timeout exiting D0i3\n");
- ret = -ETIMEDOUT;
- goto err;
- }
-
- return 0;
-err:
- clear_bit(STATUS_TRANS_IDLE, &trans->status);
- iwl_trans_fw_error(trans);
- return ret;
-}
-
-#ifdef CONFIG_IWLWIFI_PCIE_RTPM
-static int iwl_pci_runtime_suspend(struct device *device)
-{
- struct pci_dev *pdev = to_pci_dev(device);
- struct iwl_trans *trans = pci_get_drvdata(pdev);
- int ret;
-
- IWL_DEBUG_RPM(trans, "entering runtime suspend\n");
-
- if (test_bit(STATUS_DEVICE_ENABLED, &trans->status)) {
- ret = iwl_pci_fw_enter_d0i3(trans);
- if (ret < 0)
- return ret;
- }
-
- trans->system_pm_mode = IWL_PLAT_PM_MODE_D0I3;
-
- iwl_trans_d3_suspend(trans, false, false);
-
- return 0;
-}
-
-static int iwl_pci_runtime_resume(struct device *device)
-{
- struct pci_dev *pdev = to_pci_dev(device);
- struct iwl_trans *trans = pci_get_drvdata(pdev);
- enum iwl_d3_status d3_status;
-
- IWL_DEBUG_RPM(trans, "exiting runtime suspend (resume)\n");
-
- iwl_trans_d3_resume(trans, &d3_status, false, false);
-
- if (test_bit(STATUS_DEVICE_ENABLED, &trans->status))
- return iwl_pci_fw_exit_d0i3(trans);
-
- return 0;
-}
-
-static int iwl_pci_system_prepare(struct device *device)
-{
- struct pci_dev *pdev = to_pci_dev(device);
- struct iwl_trans *trans = pci_get_drvdata(pdev);
-
- IWL_DEBUG_RPM(trans, "preparing for system suspend\n");
-
- /* This is called before entering system suspend and before
- * the runtime resume is called. Set the suspending flag to
- * prevent the wakelock from being taken.
- */
- trans->suspending = true;
-
- /* Wake the device up from runtime suspend before going to
- * platform suspend. This is needed because we don't know
- * whether wowlan any is set and, if it's not, mac80211 will
- * disconnect (in which case, we can't be in D0i3).
- */
- pm_runtime_resume(device);
-
- return 0;
-}
-
-static void iwl_pci_system_complete(struct device *device)
-{
- struct pci_dev *pdev = to_pci_dev(device);
- struct iwl_trans *trans = pci_get_drvdata(pdev);
-
- IWL_DEBUG_RPM(trans, "completing system suspend\n");
-
- /* This is called as a counterpart to the prepare op. It is
- * called either when suspending fails or when suspend
- * completed successfully. Now there's no risk of grabbing
- * the wakelock anymore, so we can release the suspending
- * flag.
- */
- trans->suspending = false;
-}
-#endif /* CONFIG_IWLWIFI_PCIE_RTPM */
-
static const struct dev_pm_ops iwl_dev_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(iwl_pci_suspend,
iwl_pci_resume)
-#ifdef CONFIG_IWLWIFI_PCIE_RTPM
- SET_RUNTIME_PM_OPS(iwl_pci_runtime_suspend,
- iwl_pci_runtime_resume,
- NULL)
- .prepare = iwl_pci_system_prepare,
- .complete = iwl_pci_system_complete,
-#endif /* CONFIG_IWLWIFI_PCIE_RTPM */
};
#define IWL_PM_OPS (&iwl_dev_pm_ops)
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/internal.h b/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
index 9f5d0fc..1047d48 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
@@ -253,7 +253,8 @@ struct iwl_dma_ptr {
*/
static inline int iwl_queue_inc_wrap(struct iwl_trans *trans, int index)
{
- return ++index & (trans->cfg->base_params->max_tfd_queue_size - 1);
+ return ++index &
+ (trans->trans_cfg->base_params->max_tfd_queue_size - 1);
}
/**
@@ -263,7 +264,7 @@ static inline int iwl_queue_inc_wrap(struct iwl_trans *trans, int index)
static inline __le16 iwl_get_closed_rb_stts(struct iwl_trans *trans,
struct iwl_rxq *rxq)
{
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
__le16 *rb_stts = rxq->rb_stts;
return READ_ONCE(*rb_stts);
@@ -280,7 +281,8 @@ static inline __le16 iwl_get_closed_rb_stts(struct iwl_trans *trans,
*/
static inline int iwl_queue_dec_wrap(struct iwl_trans *trans, int index)
{
- return --index & (trans->cfg->base_params->max_tfd_queue_size - 1);
+ return --index &
+ (trans->trans_cfg->base_params->max_tfd_queue_size - 1);
}
struct iwl_cmd_meta {
@@ -556,9 +558,10 @@ struct iwl_trans_pcie {
void __iomem *hw_base;
bool ucode_write_complete;
+ bool sx_complete;
wait_queue_head_t ucode_write_waitq;
wait_queue_head_t wait_command_queue;
- wait_queue_head_t d0i3_waitq;
+ wait_queue_head_t sx_waitq;
u8 page_offs, dev_cmd_offs;
@@ -581,7 +584,6 @@ struct iwl_trans_pcie {
/*protect hw register */
spinlock_t reg_lock;
bool cmd_hold_nic_awake;
- bool ref_cmd_in_flight;
#ifdef CONFIG_IWLWIFI_DEBUGFS
struct cont_rec fw_mon_data;
@@ -635,9 +637,10 @@ iwl_trans_pcie_get_trans(struct iwl_trans_pcie *trans_pcie)
* Convention: trans API functions: iwl_trans_pcie_XXX
* Other functions: iwl_pcie_XXX
*/
-struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
- const struct pci_device_id *ent,
- const struct iwl_cfg *cfg);
+struct iwl_trans
+*iwl_trans_pcie_alloc(struct pci_dev *pdev,
+ const struct pci_device_id *ent,
+ const struct iwl_cfg_trans_params *cfg_trans);
void iwl_trans_pcie_free(struct iwl_trans *trans);
/*****************************************************
@@ -697,6 +700,7 @@ void iwl_pcie_hcmd_complete(struct iwl_trans *trans,
struct iwl_rx_cmd_buffer *rxb);
void iwl_trans_pcie_reclaim(struct iwl_trans *trans, int txq_id, int ssn,
struct sk_buff_head *skbs);
+void iwl_trans_pcie_set_q_ptrs(struct iwl_trans *trans, int txq_id, int ptr);
void iwl_trans_pcie_tx_reset(struct iwl_trans *trans);
void iwl_pcie_gen2_update_byte_tbl(struct iwl_trans_pcie *trans_pcie,
struct iwl_txq *txq, u16 byte_cnt,
@@ -705,7 +709,7 @@ void iwl_pcie_gen2_update_byte_tbl(struct iwl_trans_pcie *trans_pcie,
static inline u16 iwl_pcie_tfd_tb_get_len(struct iwl_trans *trans, void *_tfd,
u8 idx)
{
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
struct iwl_tfh_tfd *tfd = _tfd;
struct iwl_tfh_tb *tb = &tfd->tbs[idx];
@@ -911,7 +915,7 @@ static inline void *iwl_pcie_get_tfd(struct iwl_trans *trans,
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- if (trans->cfg->use_tfh)
+ if (trans->trans_cfg->use_tfh)
idx = iwl_pcie_get_cmd_index(txq, idx);
return txq->tfds + trans_pcie->tfd_size * idx;
@@ -955,7 +959,7 @@ static inline void iwl_enable_rfkill_int(struct iwl_trans *trans)
MSIX_HW_INT_CAUSES_REG_RF_KILL);
}
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_9000) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_9000) {
/*
* On 9000-series devices this bit isn't enabled by default, so
* when we power down the device we need set the bit to allow it
@@ -1045,7 +1049,7 @@ static inline void __iwl_trans_pcie_set_bit(struct iwl_trans *trans,
static inline bool iwl_pcie_dbg_on(struct iwl_trans *trans)
{
- return (trans->dbg.dest_tlv || trans->dbg.ini_valid);
+ return (trans->dbg.dest_tlv || iwl_trans_dbg_ini_valid(trans));
}
void iwl_trans_pcie_rf_kill(struct iwl_trans *trans, bool state);
@@ -1058,9 +1062,6 @@ void iwl_trans_pcie_dbgfs_register(struct iwl_trans *trans);
static inline void iwl_trans_pcie_dbgfs_register(struct iwl_trans *trans) { }
#endif
-int iwl_pci_fw_exit_d0i3(struct iwl_trans *trans);
-int iwl_pci_fw_enter_d0i3(struct iwl_trans *trans);
-
void iwl_pcie_rx_allocator_work(struct work_struct *data);
/* common functions that are used by gen2 transport */
@@ -1113,10 +1114,11 @@ int iwl_trans_pcie_gen2_tx(struct iwl_trans *trans, struct sk_buff *skb,
struct iwl_device_cmd *dev_cmd, int txq_id);
int iwl_trans_pcie_gen2_send_hcmd(struct iwl_trans *trans,
struct iwl_host_cmd *cmd);
-void iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans,
- bool low_power);
-void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power);
+void iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans);
+void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans);
void iwl_pcie_gen2_txq_unmap(struct iwl_trans *trans, int txq_id);
void iwl_pcie_gen2_tx_free(struct iwl_trans *trans);
void iwl_pcie_gen2_tx_stop(struct iwl_trans *trans);
+void iwl_pcie_d3_complete_suspend(struct iwl_trans *trans,
+ bool test, bool reset);
#endif /* __iwl_trans_int_pcie_h__ */
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
index a2d7096..19dd075 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
@@ -200,12 +200,12 @@ static inline __le32 iwl_pcie_dma_addr2rbd_ptr(dma_addr_t dma_addr)
*/
int iwl_pcie_rx_stop(struct iwl_trans *trans)
{
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
/* TODO: remove this for 22560 once fw does it */
iwl_write_umac_prph(trans, RFH_RXF_DMA_CFG_GEN3, 0);
return iwl_poll_umac_prph_bit(trans, RFH_GEN_STATUS_GEN3,
RXF_DMA_IDLE, RXF_DMA_IDLE, 1000);
- } else if (trans->cfg->mq_rx_supported) {
+ } else if (trans->trans_cfg->mq_rx_supported) {
iwl_write_prph(trans, RFH_RXF_DMA_CFG, 0);
return iwl_poll_prph_bit(trans, RFH_GEN_STATUS,
RXF_DMA_IDLE, RXF_DMA_IDLE, 1000);
@@ -232,7 +232,7 @@ static void iwl_pcie_rxq_inc_wr_ptr(struct iwl_trans *trans,
* 1. shadow registers aren't enabled
* 2. there is a chance that the NIC is asleep
*/
- if (!trans->cfg->base_params->shadow_reg_enable &&
+ if (!trans->trans_cfg->base_params->shadow_reg_enable &&
test_bit(STATUS_TPOWER_PMI, &trans->status)) {
reg = iwl_read32(trans, CSR_UCODE_DRV_GP1);
@@ -240,18 +240,18 @@ static void iwl_pcie_rxq_inc_wr_ptr(struct iwl_trans *trans,
IWL_DEBUG_INFO(trans, "Rx queue requesting wakeup, GP1 = 0x%x\n",
reg);
iwl_set_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
rxq->need_update = true;
return;
}
}
rxq->write_actual = round_down(rxq->write, 8);
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22560)
iwl_write32(trans, HBUS_TARG_WRPTR,
(rxq->write_actual |
((FIRST_RX_QUEUE + rxq->id) << 16)));
- else if (trans->cfg->mq_rx_supported)
+ else if (trans->trans_cfg->mq_rx_supported)
iwl_write32(trans, RFH_Q_FRBDCB_WIDX_TRG(rxq->id),
rxq->write_actual);
else
@@ -279,7 +279,7 @@ static void iwl_pcie_restock_bd(struct iwl_trans *trans,
struct iwl_rxq *rxq,
struct iwl_rx_mem_buffer *rxb)
{
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
struct iwl_rx_transfer_desc *bd = rxq->bd;
BUILD_BUG_ON(sizeof(*bd) != 2 * sizeof(u64));
@@ -405,7 +405,7 @@ static void iwl_pcie_rxsq_restock(struct iwl_trans *trans,
static
void iwl_pcie_rxq_restock(struct iwl_trans *trans, struct iwl_rxq *rxq)
{
- if (trans->cfg->mq_rx_supported)
+ if (trans->trans_cfg->mq_rx_supported)
iwl_pcie_rxmq_restock(trans, rxq);
else
iwl_pcie_rxsq_restock(trans, rxq);
@@ -682,7 +682,7 @@ static int iwl_pcie_free_bd_size(struct iwl_trans *trans, bool use_rx_td)
if (use_rx_td)
return sizeof(*rx_td);
else
- return trans->cfg->mq_rx_supported ? sizeof(__le64) :
+ return trans->trans_cfg->mq_rx_supported ? sizeof(__le64) :
sizeof(__le32);
}
@@ -690,7 +690,7 @@ static void iwl_pcie_free_rxq_dma(struct iwl_trans *trans,
struct iwl_rxq *rxq)
{
struct device *dev = trans->dev;
- bool use_rx_td = (trans->cfg->device_family >=
+ bool use_rx_td = (trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560);
int free_size = iwl_pcie_free_bd_size(trans, use_rx_td);
@@ -712,7 +712,7 @@ static void iwl_pcie_free_rxq_dma(struct iwl_trans *trans,
rxq->used_bd_dma = 0;
rxq->used_bd = NULL;
- if (trans->cfg->device_family < IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_22560)
return;
if (rxq->tr_tail)
@@ -735,13 +735,13 @@ static int iwl_pcie_alloc_rxq_dma(struct iwl_trans *trans,
struct device *dev = trans->dev;
int i;
int free_size;
- bool use_rx_td = (trans->cfg->device_family >=
+ bool use_rx_td = (trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560);
size_t rb_stts_size = use_rx_td ? sizeof(__le16) :
sizeof(struct iwl_rb_status);
spin_lock_init(&rxq->lock);
- if (trans->cfg->mq_rx_supported)
+ if (trans->trans_cfg->mq_rx_supported)
rxq->queue_size = MQ_RX_TABLE_SIZE;
else
rxq->queue_size = RX_QUEUE_SIZE;
@@ -757,7 +757,7 @@ static int iwl_pcie_alloc_rxq_dma(struct iwl_trans *trans,
if (!rxq->bd)
goto err;
- if (trans->cfg->mq_rx_supported) {
+ if (trans->trans_cfg->mq_rx_supported) {
rxq->used_bd = dma_alloc_coherent(dev,
(use_rx_td ? sizeof(*rxq->cd) : sizeof(__le32)) * rxq->queue_size,
&rxq->used_bd_dma,
@@ -807,7 +807,7 @@ int iwl_pcie_rx_alloc(struct iwl_trans *trans)
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
struct iwl_rb_allocator *rba = &trans_pcie->rba;
int i, ret;
- size_t rb_stts_size = trans->cfg->device_family >=
+ size_t rb_stts_size = trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560 ?
sizeof(__le16) : sizeof(struct iwl_rb_status);
@@ -1074,8 +1074,8 @@ int _iwl_pcie_rx_init(struct iwl_trans *trans)
rxq->read = 0;
rxq->write = 0;
rxq->write_actual = 0;
- memset(rxq->rb_stts, 0,
- (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) ?
+ memset(rxq->rb_stts, 0, (trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_22560) ?
sizeof(__le16) : sizeof(struct iwl_rb_status));
iwl_pcie_rx_init_rxb_lists(rxq);
@@ -1088,7 +1088,7 @@ int _iwl_pcie_rx_init(struct iwl_trans *trans)
}
/* move the pool to the default queue and allocator ownerships */
- queue_size = trans->cfg->mq_rx_supported ?
+ queue_size = trans->trans_cfg->mq_rx_supported ?
MQ_RX_NUM_RBDS : RX_QUEUE_SIZE;
allocator_pool_size = trans->num_rx_queues *
(RX_CLAIM_REQ_ALLOC - RX_POST_REQ_ALLOC);
@@ -1120,7 +1120,7 @@ int iwl_pcie_rx_init(struct iwl_trans *trans)
if (ret)
return ret;
- if (trans->cfg->mq_rx_supported)
+ if (trans->trans_cfg->mq_rx_supported)
iwl_pcie_rx_mq_hw_init(trans);
else
iwl_pcie_rx_hw_init(trans, trans_pcie->rxq);
@@ -1151,7 +1151,7 @@ void iwl_pcie_rx_free(struct iwl_trans *trans)
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
struct iwl_rb_allocator *rba = &trans_pcie->rba;
int i;
- size_t rb_stts_size = trans->cfg->device_family >=
+ size_t rb_stts_size = trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560 ?
sizeof(__le16) : sizeof(struct iwl_rb_status);
@@ -1347,7 +1347,7 @@ static void iwl_pcie_rx_handle_rb(struct iwl_trans *trans,
}
page_stolen |= rxcb._page_stolen;
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
break;
offset += ALIGN(len, FH_RSCSR_FRAME_ALIGN);
}
@@ -1392,14 +1392,14 @@ static struct iwl_rx_mem_buffer *iwl_pcie_get_rxb(struct iwl_trans *trans,
BUILD_BUG_ON(sizeof(struct iwl_rx_completion_desc) != 32);
- if (!trans->cfg->mq_rx_supported) {
+ if (!trans->trans_cfg->mq_rx_supported) {
rxb = rxq->queue[i];
rxq->queue[i] = NULL;
return rxb;
}
/* used_bd is a 32/16 bit but only 12 are used to retrieve the vid */
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
vid = le16_to_cpu(rxq->cd[i].rbid) & 0x0FFF;
else
vid = le32_to_cpu(rxq->bd_32[i]) & 0x0FFF;
@@ -1515,7 +1515,7 @@ static void iwl_pcie_rx_handle(struct iwl_trans *trans, int queue)
/* Backtrack one entry */
rxq->read = i;
/* update cr tail with the rxq read pointer */
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
*rxq->cr_tail = cpu_to_le16(r);
spin_unlock(&rxq->lock);
@@ -1597,7 +1597,7 @@ static void iwl_pcie_irq_handle_error(struct iwl_trans *trans)
return;
}
- for (i = 0; i < trans->cfg->base_params->num_of_queues; i++) {
+ for (i = 0; i < trans->trans_cfg->base_params->num_of_queues; i++) {
if (!trans_pcie->txq[i])
continue;
del_timer(&trans_pcie->txq[i]->stuck_timer);
@@ -1838,7 +1838,7 @@ irqreturn_t iwl_pcie_irq_handler(int irq, void *dev_id)
if (inta & CSR_INT_BIT_ALIVE) {
IWL_DEBUG_ISR(trans, "Alive interrupt\n");
isr_stats->alive++;
- if (trans->cfg->gen2) {
+ if (trans->trans_cfg->gen2) {
/*
* We can restock, since firmware configured
* the RFH
@@ -2179,13 +2179,13 @@ irqreturn_t iwl_pcie_irq_msix_handler(int irq, void *dev_id)
if (inta_hw & MSIX_HW_INT_CAUSES_REG_ALIVE) {
IWL_DEBUG_ISR(trans, "Alive interrupt\n");
isr_stats->alive++;
- if (trans->cfg->gen2) {
+ if (trans->trans_cfg->gen2) {
/* We can restock, since firmware configured the RFH */
iwl_pcie_rxmq_restock(trans, trans_pcie->rxq);
}
}
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_22560 &&
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_22560 &&
inta_hw & MSIX_HW_INT_CAUSES_REG_IPC) {
/* Reflect IML transfer status */
int res = iwl_read32(trans, CSR_IML_RESP_ADDR);
@@ -2196,12 +2196,23 @@ irqreturn_t iwl_pcie_irq_msix_handler(int irq, void *dev_id)
iwl_pcie_irq_handle_error(trans);
}
} else if (inta_hw & MSIX_HW_INT_CAUSES_REG_WAKEUP) {
- /* uCode wakes up after power-down sleep */
- IWL_DEBUG_ISR(trans, "Wakeup interrupt\n");
- iwl_pcie_rxq_check_wrptr(trans);
- iwl_pcie_txq_check_wrptrs(trans);
+ u32 sleep_notif =
+ le32_to_cpu(trans_pcie->prph_info->sleep_notif);
+ if (sleep_notif == IWL_D3_SLEEP_STATUS_SUSPEND ||
+ sleep_notif == IWL_D3_SLEEP_STATUS_RESUME) {
+ IWL_DEBUG_ISR(trans,
+ "Sx interrupt: sleep notification = 0x%x\n",
+ sleep_notif);
+ trans_pcie->sx_complete = true;
+ wake_up(&trans_pcie->sx_waitq);
+ } else {
+ /* uCode wakes up after power-down sleep */
+ IWL_DEBUG_ISR(trans, "Wakeup interrupt\n");
+ iwl_pcie_rxq_check_wrptr(trans);
+ iwl_pcie_txq_check_wrptrs(trans);
- isr_stats->wakeup++;
+ isr_stats->wakeup++;
+ }
}
if (inta_hw & MSIX_HW_INT_CAUSES_REG_IML) {
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
index 8d17e68..df8455f 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c
@@ -92,7 +92,7 @@ int iwl_pcie_gen2_apm_init(struct iwl_trans *trans)
iwl_pcie_apm_config(trans);
- ret = iwl_finish_nic_init(trans);
+ ret = iwl_finish_nic_init(trans, trans->trans_cfg);
if (ret)
return ret;
@@ -133,10 +133,10 @@ static void iwl_pcie_gen2_apm_stop(struct iwl_trans *trans, bool op_mode_leave)
* D0A* (powered-up Active) --> D0U* (Uninitialized) state.
*/
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_init_done));
+ BIT(trans->trans_cfg->csr->flag_init_done));
}
-void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power)
+void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
@@ -147,9 +147,6 @@ void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power)
trans_pcie->is_down = true;
- /* Stop dbgc before stopping device */
- iwl_fw_dbg_stop_recording(trans, NULL);
-
/* tell the device to stop sending interrupts */
iwl_disable_interrupts(trans);
@@ -171,14 +168,14 @@ void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power)
}
iwl_pcie_ctxt_info_free_paging(trans);
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
iwl_pcie_ctxt_info_gen3_free(trans);
else
iwl_pcie_ctxt_info_free(trans);
/* Make sure (redundant) we've released our request to stay awake */
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
/* Stop the device, and put it in low power state */
iwl_pcie_gen2_apm_stop(trans, false);
@@ -218,7 +215,7 @@ void _iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power)
iwl_pcie_prepare_card_hw(trans);
}
-void iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power)
+void iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
bool was_in_rfkill;
@@ -226,7 +223,7 @@ void iwl_trans_pcie_gen2_stop_device(struct iwl_trans *trans, bool low_power)
mutex_lock(&trans_pcie->mutex);
trans_pcie->opmode_down = true;
was_in_rfkill = test_bit(STATUS_RFKILL_OPMODE, &trans->status);
- _iwl_trans_pcie_gen2_stop_device(trans, low_power);
+ _iwl_trans_pcie_gen2_stop_device(trans);
iwl_trans_pcie_handle_stop_rfkill(trans, was_in_rfkill);
mutex_unlock(&trans_pcie->mutex);
}
@@ -343,7 +340,7 @@ int iwl_trans_pcie_gen2_start_fw(struct iwl_trans *trans,
goto out;
}
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
ret = iwl_pcie_ctxt_info_gen3_init(trans, fw);
else
ret = iwl_pcie_ctxt_info_init(trans, fw);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
index f5df5b3..5ab87a8 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
@@ -69,7 +69,6 @@
#include <linux/bitops.h>
#include <linux/gfp.h>
#include <linux/vmalloc.h>
-#include <linux/pm_runtime.h>
#include <linux/module.h>
#include <linux/wait.h>
@@ -185,8 +184,8 @@ void iwl_trans_pcie_dump_regs(struct iwl_trans *trans)
static void iwl_trans_pcie_sw_reset(struct iwl_trans *trans)
{
/* Reset entire device - do controller reset (results in SHRD_HW_RST) */
- iwl_set_bit(trans, trans->cfg->csr->addr_sw_reset,
- BIT(trans->cfg->csr->flag_sw_reset));
+ iwl_set_bit(trans, trans->trans_cfg->csr->addr_sw_reset,
+ BIT(trans->trans_cfg->csr->flag_sw_reset));
usleep_range(5000, 6000);
}
@@ -216,8 +215,7 @@ static void iwl_pcie_alloc_fw_monitor_block(struct iwl_trans *trans,
for (power = max_power; power >= min_power; power--) {
size = BIT(power);
cpu_addr = dma_alloc_coherent(trans->dev, size, &phys,
- GFP_KERNEL | __GFP_NOWARN |
- __GFP_ZERO | __GFP_COMP);
+ GFP_KERNEL | __GFP_NOWARN);
if (!cpu_addr)
continue;
@@ -343,7 +341,7 @@ static int iwl_pcie_apm_init(struct iwl_trans *trans)
*/
/* Disable L0S exit timer (platform NMI Work/Around) */
- if (trans->cfg->device_family < IWL_DEVICE_FAMILY_8000)
+ if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_8000)
iwl_set_bit(trans, CSR_GIO_CHICKEN_BITS,
CSR_GIO_CHICKEN_BITS_REG_BIT_DIS_L0S_EXIT_TIMER);
@@ -367,10 +365,10 @@ static int iwl_pcie_apm_init(struct iwl_trans *trans)
iwl_pcie_apm_config(trans);
/* Configure analog phase-lock-loop before activating to D0A */
- if (trans->cfg->base_params->pll_cfg)
+ if (trans->trans_cfg->base_params->pll_cfg)
iwl_set_bit(trans, CSR_ANA_PLL_CFG, CSR50_ANA_PLL_CFG_VAL);
- ret = iwl_finish_nic_init(trans);
+ ret = iwl_finish_nic_init(trans, trans->trans_cfg);
if (ret)
return ret;
@@ -442,7 +440,7 @@ static void iwl_pcie_apm_lp_xtal_enable(struct iwl_trans *trans)
iwl_trans_pcie_sw_reset(trans);
- ret = iwl_finish_nic_init(trans);
+ ret = iwl_finish_nic_init(trans, trans->trans_cfg);
if (WARN_ON(ret)) {
/* Release XTAL ON request */
__iwl_trans_pcie_clear_bit(trans, CSR_GP_CNTRL,
@@ -492,7 +490,7 @@ static void iwl_pcie_apm_lp_xtal_enable(struct iwl_trans *trans)
* D0A* (powered-up Active) --> D0U* (Uninitialized) state.
*/
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_init_done));
+ BIT(trans->trans_cfg->csr->flag_init_done));
/* Activates XTAL resources monitor */
__iwl_trans_pcie_set_bit(trans, CSR_MONITOR_CFG_REG,
@@ -514,12 +512,12 @@ void iwl_pcie_apm_stop_master(struct iwl_trans *trans)
int ret;
/* stop device's busmaster DMA activity */
- iwl_set_bit(trans, trans->cfg->csr->addr_sw_reset,
- BIT(trans->cfg->csr->flag_stop_master));
+ iwl_set_bit(trans, trans->trans_cfg->csr->addr_sw_reset,
+ BIT(trans->trans_cfg->csr->flag_stop_master));
- ret = iwl_poll_bit(trans, trans->cfg->csr->addr_sw_reset,
- BIT(trans->cfg->csr->flag_master_dis),
- BIT(trans->cfg->csr->flag_master_dis), 100);
+ ret = iwl_poll_bit(trans, trans->trans_cfg->csr->addr_sw_reset,
+ BIT(trans->trans_cfg->csr->flag_master_dis),
+ BIT(trans->trans_cfg->csr->flag_master_dis), 100);
if (ret < 0)
IWL_WARN(trans, "Master Disable Timed Out, 100 usec\n");
@@ -535,10 +533,11 @@ static void iwl_pcie_apm_stop(struct iwl_trans *trans, bool op_mode_leave)
iwl_pcie_apm_init(trans);
/* inform ME that we are leaving */
- if (trans->cfg->device_family == IWL_DEVICE_FAMILY_7000)
+ if (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_7000)
iwl_set_bits_prph(trans, APMG_PCIDEV_STT_REG,
APMG_PCIDEV_STT_VAL_WAKE_ME);
- else if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_8000) {
+ else if (trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_8000) {
iwl_set_bit(trans, CSR_DBG_LINK_PWR_MGMT_REG,
CSR_RESET_LINK_PWR_MGMT_DISABLED);
iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG,
@@ -568,7 +567,7 @@ static void iwl_pcie_apm_stop(struct iwl_trans *trans, bool op_mode_leave)
* D0A* (powered-up Active) --> D0U* (Uninitialized) state.
*/
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_init_done));
+ BIT(trans->trans_cfg->csr->flag_init_done));
}
static int iwl_pcie_nic_init(struct iwl_trans *trans)
@@ -595,7 +594,7 @@ static int iwl_pcie_nic_init(struct iwl_trans *trans)
if (iwl_pcie_tx_init(trans))
return -ENOMEM;
- if (trans->cfg->base_params->shadow_reg_enable) {
+ if (trans->trans_cfg->base_params->shadow_reg_enable) {
/* enable shadow regs in HW */
iwl_set_bit(trans, CSR_MAC_SHADOW_REG_CTRL, 0x800FFFFF);
IWL_DEBUG_INFO(trans, "Enabling shadow registers in device\n");
@@ -833,7 +832,7 @@ static int iwl_pcie_load_cpu_sections_8000(struct iwl_trans *trans,
iwl_enable_interrupts(trans);
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
if (cpu == 1)
iwl_write_prph(trans, UREG_UCODE_LOAD_STATUS,
0xFFFF);
@@ -898,12 +897,12 @@ void iwl_pcie_apply_destination(struct iwl_trans *trans)
const struct iwl_fw_dbg_dest_tlv_v1 *dest = trans->dbg.dest_tlv;
int i;
- if (trans->dbg.ini_valid) {
+ if (iwl_trans_dbg_ini_valid(trans)) {
if (!trans->dbg.num_blocks)
return;
IWL_DEBUG_FW(trans,
- "WRT: applying DRAM buffer[0] destination\n");
+ "WRT: Applying DRAM buffer[0] destination\n");
iwl_write_umac_prph(trans, MON_BUFF_BASE_ADDR_VER2,
trans->dbg.fw_mon[0].physical >>
MON_BUFF_SHIFT_VER2);
@@ -965,7 +964,7 @@ void iwl_pcie_apply_destination(struct iwl_trans *trans)
iwl_write_prph(trans, le32_to_cpu(dest->base_reg),
trans->dbg.fw_mon[0].physical >>
dest->base_shift);
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_8000)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_8000)
iwl_write_prph(trans, le32_to_cpu(dest->end_reg),
(trans->dbg.fw_mon[0].physical +
trans->dbg.fw_mon[0].size - 256) >>
@@ -1007,7 +1006,7 @@ static int iwl_pcie_load_given_ucode(struct iwl_trans *trans,
/* supported for 7000 only for the moment */
if (iwlwifi_mod_params.fw_monitor &&
- trans->cfg->device_family == IWL_DEVICE_FAMILY_7000) {
+ trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_7000) {
iwl_pcie_alloc_fw_monitor(trans, 0);
if (trans->dbg.fw_mon[0].size) {
@@ -1136,7 +1135,7 @@ static void iwl_pcie_map_non_rx_causes(struct iwl_trans *trans)
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int val = trans_pcie->def_irq | MSIX_NON_AUTO_CLEAR_CAUSE;
int i, arr_size =
- (trans->cfg->device_family != IWL_DEVICE_FAMILY_22560) ?
+ (trans->trans_cfg->device_family != IWL_DEVICE_FAMILY_22560) ?
ARRAY_SIZE(causes_list) : ARRAY_SIZE(causes_list_v2);
/*
@@ -1146,7 +1145,8 @@ static void iwl_pcie_map_non_rx_causes(struct iwl_trans *trans)
*/
for (i = 0; i < arr_size; i++) {
struct iwl_causes_list *causes =
- (trans->cfg->device_family != IWL_DEVICE_FAMILY_22560) ?
+ (trans->trans_cfg->device_family !=
+ IWL_DEVICE_FAMILY_22560) ?
causes_list : causes_list_v2;
iwl_write8(trans, CSR_MSIX_IVAR(causes[i].addr), val);
@@ -1190,7 +1190,7 @@ void iwl_pcie_conf_msix_hw(struct iwl_trans_pcie *trans_pcie)
struct iwl_trans *trans = trans_pcie->trans;
if (!trans_pcie->msix_enabled) {
- if (trans->cfg->mq_rx_supported &&
+ if (trans->trans_cfg->mq_rx_supported &&
test_bit(STATUS_DEVICE_ENABLED, &trans->status))
iwl_write_umac_prph(trans, UREG_CHICK,
UREG_CHICK_MSI_ENABLE);
@@ -1231,7 +1231,7 @@ static void iwl_pcie_init_msix(struct iwl_trans_pcie *trans_pcie)
trans_pcie->hw_mask = trans_pcie->hw_init_mask;
}
-static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans, bool low_power)
+static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
@@ -1242,9 +1242,6 @@ static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans, bool low_power)
trans_pcie->is_down = true;
- /* Stop dbgc before stopping device */
- iwl_fw_dbg_stop_recording(trans, NULL);
-
/* tell the device to stop sending interrupts */
iwl_disable_interrupts(trans);
@@ -1274,7 +1271,7 @@ static void _iwl_trans_pcie_stop_device(struct iwl_trans *trans, bool low_power)
/* Make sure (redundant) we've released our request to stay awake */
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
/* Stop the device, and put it in low power state */
iwl_pcie_apm_stop(trans, false);
@@ -1401,7 +1398,7 @@ static int iwl_trans_pcie_start_fw(struct iwl_trans *trans,
iwl_write32(trans, CSR_UCODE_DRV_GP1_CLR, CSR_UCODE_SW_BIT_RFKILL);
/* Load the given image to the HW */
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_8000)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_8000)
ret = iwl_pcie_load_given_ucode_8000(trans, fw);
else
ret = iwl_pcie_load_given_ucode(trans, fw);
@@ -1451,7 +1448,7 @@ void iwl_trans_pcie_handle_stop_rfkill(struct iwl_trans *trans,
iwl_trans_pcie_rf_kill(trans, hw_rfkill);
}
-static void iwl_trans_pcie_stop_device(struct iwl_trans *trans, bool low_power)
+static void iwl_trans_pcie_stop_device(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
bool was_in_rfkill;
@@ -1459,7 +1456,7 @@ static void iwl_trans_pcie_stop_device(struct iwl_trans *trans, bool low_power)
mutex_lock(&trans_pcie->mutex);
trans_pcie->opmode_down = true;
was_in_rfkill = test_bit(STATUS_RFKILL_OPMODE, &trans->status);
- _iwl_trans_pcie_stop_device(trans, low_power);
+ _iwl_trans_pcie_stop_device(trans);
iwl_trans_pcie_handle_stop_rfkill(trans, was_in_rfkill);
mutex_unlock(&trans_pcie->mutex);
}
@@ -1474,22 +1471,16 @@ void iwl_trans_pcie_rf_kill(struct iwl_trans *trans, bool state)
IWL_WARN(trans, "reporting RF_KILL (radio %s)\n",
state ? "disabled" : "enabled");
if (iwl_op_mode_hw_rf_kill(trans->op_mode, state)) {
- if (trans->cfg->gen2)
- _iwl_trans_pcie_gen2_stop_device(trans, true);
+ if (trans->trans_cfg->gen2)
+ _iwl_trans_pcie_gen2_stop_device(trans);
else
- _iwl_trans_pcie_stop_device(trans, true);
+ _iwl_trans_pcie_stop_device(trans);
}
}
-static void iwl_trans_pcie_d3_suspend(struct iwl_trans *trans, bool test,
- bool reset)
+void iwl_pcie_d3_complete_suspend(struct iwl_trans *trans,
+ bool test, bool reset)
{
- if (!reset) {
- /* Enable persistence mode to avoid reset */
- iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG,
- CSR_HW_IF_CONFIG_REG_PERSIST_MODE);
- }
-
iwl_disable_interrupts(trans);
/*
@@ -1504,9 +1495,9 @@ static void iwl_trans_pcie_d3_suspend(struct iwl_trans *trans, bool test,
iwl_pcie_synchronize_irqs(trans);
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_init_done));
+ BIT(trans->trans_cfg->csr->flag_init_done));
if (reset) {
/*
@@ -1520,6 +1511,42 @@ static void iwl_trans_pcie_d3_suspend(struct iwl_trans *trans, bool test,
iwl_pcie_set_pwr(trans, true);
}
+static int iwl_trans_pcie_d3_suspend(struct iwl_trans *trans, bool test,
+ bool reset)
+{
+ int ret;
+ struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
+
+ /*
+ * Family IWL_DEVICE_FAMILY_AX210 and above persist mode is set by FW.
+ */
+ if (!reset && trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_AX210) {
+ /* Enable persistence mode to avoid reset */
+ iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG,
+ CSR_HW_IF_CONFIG_REG_PERSIST_MODE);
+ }
+
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
+ iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
+ UREG_DOORBELL_TO_ISR6_SUSPEND);
+
+ ret = wait_event_timeout(trans_pcie->sx_waitq,
+ trans_pcie->sx_complete, 2 * HZ);
+ /*
+ * Invalidate it toward resume.
+ */
+ trans_pcie->sx_complete = false;
+
+ if (!ret) {
+ IWL_ERR(trans, "Timeout entering D3\n");
+ return -ETIMEDOUT;
+ }
+ }
+ iwl_pcie_d3_complete_suspend(trans, test, reset);
+
+ return 0;
+}
+
static int iwl_trans_pcie_d3_resume(struct iwl_trans *trans,
enum iwl_d3_status *status,
bool test, bool reset)
@@ -1531,13 +1558,13 @@ static int iwl_trans_pcie_d3_resume(struct iwl_trans *trans,
if (test) {
iwl_enable_interrupts(trans);
*status = IWL_D3_STATUS_ALIVE;
- return 0;
+ goto out;
}
iwl_set_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
- ret = iwl_finish_nic_init(trans);
+ ret = iwl_finish_nic_init(trans, trans->trans_cfg);
if (ret)
return ret;
@@ -1557,7 +1584,7 @@ static int iwl_trans_pcie_d3_resume(struct iwl_trans *trans,
if (!reset) {
iwl_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
} else {
iwl_trans_pcie_tx_reset(trans);
@@ -1578,17 +1605,38 @@ static int iwl_trans_pcie_d3_resume(struct iwl_trans *trans,
else
*status = IWL_D3_STATUS_ALIVE;
+out:
+ if (*status == IWL_D3_STATUS_ALIVE &&
+ trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
+ trans_pcie->sx_complete = false;
+ iwl_write_umac_prph(trans, UREG_DOORBELL_TO_ISR6,
+ UREG_DOORBELL_TO_ISR6_RESUME);
+
+ ret = wait_event_timeout(trans_pcie->sx_waitq,
+ trans_pcie->sx_complete, 2 * HZ);
+ /*
+ * Invalidate it toward next suspend.
+ */
+ trans_pcie->sx_complete = false;
+
+ if (!ret) {
+ IWL_ERR(trans, "Timeout exiting D3\n");
+ return -ETIMEDOUT;
+ }
+ }
return 0;
}
-static void iwl_pcie_set_interrupt_capa(struct pci_dev *pdev,
- struct iwl_trans *trans)
+static void
+iwl_pcie_set_interrupt_capa(struct pci_dev *pdev,
+ struct iwl_trans *trans,
+ const struct iwl_cfg_trans_params *cfg_trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int max_irqs, num_irqs, i, ret;
u16 pci_cmd;
- if (!trans->cfg->mq_rx_supported)
+ if (!cfg_trans->mq_rx_supported)
goto enable_msi;
max_irqs = min_t(u32, num_online_cpus() + 2, IWL_MAX_RX_HW_QUEUES);
@@ -1709,7 +1757,7 @@ static int iwl_trans_pcie_clear_persistence_bit(struct iwl_trans *trans)
{
u32 hpm, wprot;
- switch (trans->cfg->device_family) {
+ switch (trans->trans_cfg->device_family) {
case IWL_DEVICE_FAMILY_9000:
wprot = PREG_PRPH_WPROT_9000;
break;
@@ -1736,7 +1784,7 @@ static int iwl_trans_pcie_clear_persistence_bit(struct iwl_trans *trans)
return 0;
}
-static int _iwl_trans_pcie_start_hw(struct iwl_trans *trans, bool low_power)
+static int _iwl_trans_pcie_start_hw(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int err;
@@ -1772,20 +1820,16 @@ static int _iwl_trans_pcie_start_hw(struct iwl_trans *trans, bool low_power)
/* ...rfkill can call stop_device and set it false if needed */
iwl_pcie_check_hw_rf_kill(trans);
- /* Make sure we sync here, because we'll need full access later */
- if (low_power)
- pm_runtime_resume(trans->dev);
-
return 0;
}
-static int iwl_trans_pcie_start_hw(struct iwl_trans *trans, bool low_power)
+static int iwl_trans_pcie_start_hw(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int ret;
mutex_lock(&trans_pcie->mutex);
- ret = _iwl_trans_pcie_start_hw(trans, low_power);
+ ret = _iwl_trans_pcie_start_hw(trans);
mutex_unlock(&trans_pcie->mutex);
return ret;
@@ -1828,7 +1872,7 @@ static u32 iwl_trans_pcie_read32(struct iwl_trans *trans, u32 ofs)
static u32 iwl_trans_pcie_prph_msk(struct iwl_trans *trans)
{
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560)
return 0x00FFFFFF;
else
return 0x000FFFFF;
@@ -1899,7 +1943,7 @@ void iwl_trans_pcie_free(struct iwl_trans *trans)
iwl_pcie_synchronize_irqs(trans);
- if (trans->cfg->gen2)
+ if (trans->trans_cfg->gen2)
iwl_pcie_gen2_tx_free(trans);
else
iwl_pcie_tx_free(trans);
@@ -1981,8 +2025,8 @@ static bool iwl_trans_pcie_grab_nic_access(struct iwl_trans *trans,
/* this bit wakes up the NIC */
__iwl_trans_pcie_set_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_8000)
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_8000)
udelay(2);
/*
@@ -2006,8 +2050,8 @@ static bool iwl_trans_pcie_grab_nic_access(struct iwl_trans *trans,
* and do not save/restore SRAM when power cycling.
*/
ret = iwl_poll_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_val_mac_access_en),
- (BIT(trans->cfg->csr->flag_mac_clock_ready) |
+ BIT(trans->trans_cfg->csr->flag_val_mac_access_en),
+ (BIT(trans->trans_cfg->csr->flag_mac_clock_ready) |
CSR_GP_CNTRL_REG_FLAG_GOING_TO_SLEEP), 15000);
if (unlikely(ret < 0)) {
u32 cntrl = iwl_read32(trans, CSR_GP_CNTRL);
@@ -2089,7 +2133,7 @@ static void iwl_trans_pcie_release_nic_access(struct iwl_trans *trans,
goto out;
__iwl_trans_pcie_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
/*
* Above we read the CSR_GP_CNTRL register, which will flush
* any previous writes, but we need the write that clears the
@@ -2196,7 +2240,7 @@ static void iwl_trans_pcie_block_txq_ptrs(struct iwl_trans *trans, bool block)
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int i;
- for (i = 0; i < trans->cfg->base_params->num_of_queues; i++) {
+ for (i = 0; i < trans->trans_cfg->base_params->num_of_queues; i++) {
struct iwl_txq *txq = trans_pcie->txq[i];
if (i == trans_pcie->cmd_queue)
@@ -2227,7 +2271,7 @@ void iwl_trans_pcie_log_scd_error(struct iwl_trans *trans, struct iwl_txq *txq)
bool active;
u8 fifo;
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
IWL_ERR(trans, "Queue %d is stuck %d %d\n", txq_id,
txq->read_ptr, txq->write_ptr);
/* TODO: access new SCD registers and dump them */
@@ -2244,10 +2288,10 @@ void iwl_trans_pcie_log_scd_error(struct iwl_trans *trans, struct iwl_txq *txq)
jiffies_to_msecs(txq->wd_timeout),
txq->read_ptr, txq->write_ptr,
iwl_read_prph(trans, SCD_QUEUE_RDPTR(txq_id)) &
- (trans->cfg->base_params->max_tfd_queue_size - 1),
- iwl_read_prph(trans, SCD_QUEUE_WRPTR(txq_id)) &
- (trans->cfg->base_params->max_tfd_queue_size - 1),
- iwl_read_direct32(trans, FH_TX_TRB_REG(fifo)));
+ (trans->trans_cfg->base_params->max_tfd_queue_size - 1),
+ iwl_read_prph(trans, SCD_QUEUE_WRPTR(txq_id)) &
+ (trans->trans_cfg->base_params->max_tfd_queue_size - 1),
+ iwl_read_direct32(trans, FH_TX_TRB_REG(fifo)));
}
static int iwl_trans_pcie_rxq_dma_data(struct iwl_trans *trans, int queue,
@@ -2335,7 +2379,9 @@ static int iwl_trans_pcie_wait_txqs_empty(struct iwl_trans *trans, u32 txq_bm)
int ret = 0;
/* waiting for all the tx frames complete might take a while */
- for (cnt = 0; cnt < trans->cfg->base_params->num_of_queues; cnt++) {
+ for (cnt = 0;
+ cnt < trans->trans_cfg->base_params->num_of_queues;
+ cnt++) {
if (cnt == trans_pcie->cmd_queue)
continue;
@@ -2363,37 +2409,6 @@ static void iwl_trans_pcie_set_bits_mask(struct iwl_trans *trans, u32 reg,
spin_unlock_irqrestore(&trans_pcie->reg_lock, flags);
}
-static void iwl_trans_pcie_ref(struct iwl_trans *trans)
-{
- struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
-
- if (iwlwifi_mod_params.d0i3_disable)
- return;
-
- pm_runtime_get(&trans_pcie->pci_dev->dev);
-
-#ifdef CONFIG_PM
- IWL_DEBUG_RPM(trans, "runtime usage count: %d\n",
- atomic_read(&trans_pcie->pci_dev->dev.power.usage_count));
-#endif /* CONFIG_PM */
-}
-
-static void iwl_trans_pcie_unref(struct iwl_trans *trans)
-{
- struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
-
- if (iwlwifi_mod_params.d0i3_disable)
- return;
-
- pm_runtime_mark_last_busy(&trans_pcie->pci_dev->dev);
- pm_runtime_put_autosuspend(&trans_pcie->pci_dev->dev);
-
-#ifdef CONFIG_PM
- IWL_DEBUG_RPM(trans, "runtime usage count: %d\n",
- atomic_read(&trans_pcie->pci_dev->dev.power.usage_count));
-#endif /* CONFIG_PM */
-}
-
static const char *get_csr_string(int cmd)
{
#define IWL_CMD(x) case x: return #x
@@ -2510,7 +2525,8 @@ static ssize_t iwl_dbgfs_tx_queue_read(struct file *file,
int ret;
size_t bufsz;
- bufsz = sizeof(char) * 75 * trans->cfg->base_params->num_of_queues;
+ bufsz = sizeof(char) * 75 *
+ trans->trans_cfg->base_params->num_of_queues;
if (!trans_pcie->txq_memory)
return -EAGAIN;
@@ -2519,7 +2535,9 @@ static ssize_t iwl_dbgfs_tx_queue_read(struct file *file,
if (!buf)
return -ENOMEM;
- for (cnt = 0; cnt < trans->cfg->base_params->num_of_queues; cnt++) {
+ for (cnt = 0;
+ cnt < trans->trans_cfg->base_params->num_of_queues;
+ cnt++) {
txq = trans_pcie->txq[cnt];
pos += scnprintf(buf + pos, bufsz - pos,
"hwq %.2d: read=%u write=%u use=%d stop=%d need_update=%d frozen=%d%s\n",
@@ -2989,7 +3007,7 @@ static u32 iwl_trans_pcie_fh_regs_dump(struct iwl_trans *trans,
(*data)->len = cpu_to_le32(fh_regs_len);
val = (void *)(*data)->data;
- if (!trans->cfg->gen2)
+ if (!trans->trans_cfg->gen2)
for (i = FH_MEM_LOWER_BOUND; i < FH_MEM_UPPER_BOUND;
i += sizeof(u32))
*val++ = cpu_to_le32(iwl_trans_pcie_read32(trans, i));
@@ -3037,7 +3055,7 @@ iwl_trans_pcie_dump_pointers(struct iwl_trans *trans,
{
u32 base, base_high, write_ptr, write_ptr_val, wrap_cnt;
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
base = DBGC_CUR_DBGBUF_BASE_ADDR_LSB;
base_high = DBGC_CUR_DBGBUF_BASE_ADDR_MSB;
write_ptr = DBGC_CUR_DBGBUF_STATUS;
@@ -3057,7 +3075,7 @@ iwl_trans_pcie_dump_pointers(struct iwl_trans *trans,
cpu_to_le32(iwl_read_prph(trans, wrap_cnt));
fw_mon_data->fw_mon_base_ptr =
cpu_to_le32(iwl_read_prph(trans, base));
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210) {
fw_mon_data->fw_mon_base_high_ptr =
cpu_to_le32(iwl_read_prph(trans, base_high));
write_ptr_val &= DBGC_CUR_DBGBUF_STATUS_OFFSET_MSK;
@@ -3074,8 +3092,8 @@ iwl_trans_pcie_dump_monitor(struct iwl_trans *trans,
if (trans->dbg.dest_tlv ||
(trans->dbg.num_blocks &&
- (trans->cfg->device_family == IWL_DEVICE_FAMILY_7000 ||
- trans->cfg->device_family >= IWL_DEVICE_FAMILY_AX210))) {
+ (trans->trans_cfg->device_family == IWL_DEVICE_FAMILY_7000 ||
+ trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_AX210))) {
struct iwl_fw_error_dump_fw_mon *fw_mon_data;
(*data)->type = cpu_to_le32(IWL_FW_ERROR_DUMP_FW_MONITOR);
@@ -3158,7 +3176,7 @@ static int iwl_trans_get_fw_monitor_len(struct iwl_trans *trans, u32 *len)
trans->dbg.dest_tlv->end_shift;
/* Make "end" point to the actual end */
- if (trans->cfg->device_family >=
+ if (trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_8000 ||
trans->dbg.dest_tlv->monitor_mode == MARBH_MODE)
end += (1 << trans->dbg.dest_tlv->end_shift);
@@ -3184,7 +3202,7 @@ static struct iwl_trans_dump_data
u32 len, num_rbs = 0, monitor_len = 0;
int i, ptr;
bool dump_rbs = test_bit(STATUS_FW_ERROR, &trans->status) &&
- !trans->cfg->mq_rx_supported &&
+ !trans->trans_cfg->mq_rx_supported &&
dump_mask & BIT(IWL_FW_ERROR_DUMP_RB);
if (!dump_mask)
@@ -3209,7 +3227,7 @@ static struct iwl_trans_dump_data
/* FH registers */
if (dump_mask & BIT(IWL_FW_ERROR_DUMP_FH_REGS)) {
- if (trans->cfg->gen2)
+ if (trans->trans_cfg->gen2)
len += sizeof(*data) +
(iwl_umac_prph(trans, FH_MEM_UPPER_BOUND_GEN2) -
iwl_umac_prph(trans, FH_MEM_LOWER_BOUND_GEN2));
@@ -3233,7 +3251,7 @@ static struct iwl_trans_dump_data
}
/* Paged memory for gen2 HW */
- if (trans->cfg->gen2 && dump_mask & BIT(IWL_FW_ERROR_DUMP_PAGING))
+ if (trans->trans_cfg->gen2 && dump_mask & BIT(IWL_FW_ERROR_DUMP_PAGING))
for (i = 0; i < trans->init_dram.paging_cnt; i++)
len += sizeof(*data) +
sizeof(struct iwl_fw_error_dump_paging) +
@@ -3288,7 +3306,8 @@ static struct iwl_trans_dump_data
len += iwl_trans_pcie_dump_rbs(trans, &data, num_rbs);
/* Paged memory for gen2 HW */
- if (trans->cfg->gen2 && dump_mask & BIT(IWL_FW_ERROR_DUMP_PAGING)) {
+ if (trans->trans_cfg->gen2 &&
+ dump_mask & BIT(IWL_FW_ERROR_DUMP_PAGING)) {
for (i = 0; i < trans->init_dram.paging_cnt; i++) {
struct iwl_fw_error_dump_paging *paging;
u32 page_len = trans->init_dram.paging[i].size;
@@ -3315,18 +3334,11 @@ static struct iwl_trans_dump_data
#ifdef CONFIG_PM_SLEEP
static int iwl_trans_pcie_suspend(struct iwl_trans *trans)
{
- if (trans->runtime_pm_mode == IWL_PLAT_PM_MODE_D0I3 &&
- (trans->system_pm_mode == IWL_PLAT_PM_MODE_D0I3))
- return iwl_pci_fw_enter_d0i3(trans);
-
return 0;
}
static void iwl_trans_pcie_resume(struct iwl_trans *trans)
{
- if (trans->runtime_pm_mode == IWL_PLAT_PM_MODE_D0I3 &&
- (trans->system_pm_mode == IWL_PLAT_PM_MODE_D0I3))
- iwl_pci_fw_exit_d0i3(trans);
}
#endif /* CONFIG_PM_SLEEP */
@@ -3345,8 +3357,6 @@ static void iwl_trans_pcie_resume(struct iwl_trans *trans)
.grab_nic_access = iwl_trans_pcie_grab_nic_access, \
.release_nic_access = iwl_trans_pcie_release_nic_access, \
.set_bits_mask = iwl_trans_pcie_set_bits_mask, \
- .ref = iwl_trans_pcie_ref, \
- .unref = iwl_trans_pcie_unref, \
.dump_data = iwl_trans_pcie_dump_data, \
.d3_suspend = iwl_trans_pcie_d3_suspend, \
.d3_resume = iwl_trans_pcie_d3_resume, \
@@ -3400,6 +3410,8 @@ static const struct iwl_trans_ops trans_ops_pcie_gen2 = {
.tx = iwl_trans_pcie_gen2_tx,
.reclaim = iwl_trans_pcie_reclaim,
+ .set_q_ptrs = iwl_trans_pcie_set_q_ptrs,
+
.txq_alloc = iwl_trans_pcie_dyn_txq_alloc,
.txq_free = iwl_trans_pcie_dyn_txq_free,
.wait_txq_empty = iwl_trans_pcie_wait_txq_empty,
@@ -3410,8 +3422,8 @@ static const struct iwl_trans_ops trans_ops_pcie_gen2 = {
};
struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
- const struct pci_device_id *ent,
- const struct iwl_cfg *cfg)
+ const struct pci_device_id *ent,
+ const struct iwl_cfg_trans_params *cfg_trans)
{
struct iwl_trans_pcie *trans_pcie;
struct iwl_trans *trans;
@@ -3421,12 +3433,13 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
if (ret)
return ERR_PTR(ret);
- if (cfg->gen2)
+ if (cfg_trans->gen2)
trans = iwl_trans_alloc(sizeof(struct iwl_trans_pcie),
- &pdev->dev, cfg, &trans_ops_pcie_gen2);
+ &pdev->dev, &trans_ops_pcie_gen2);
else
trans = iwl_trans_alloc(sizeof(struct iwl_trans_pcie),
- &pdev->dev, cfg, &trans_ops_pcie);
+ &pdev->dev, &trans_ops_pcie);
+
if (!trans)
return ERR_PTR(-ENOMEM);
@@ -3445,7 +3458,7 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
}
trans_pcie->debug_rfkill = -1;
- if (!cfg->base_params->pcie_l1_allowed) {
+ if (!cfg_trans->base_params->pcie_l1_allowed) {
/*
* W/A - seems to solve weird behavior. We need to remove this
* if we don't want to stay in L1 all the time. This wastes a
@@ -3458,7 +3471,7 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
trans_pcie->def_rx_queue = 0;
- if (cfg->use_tfh) {
+ if (cfg_trans->use_tfh) {
addr_size = 64;
trans_pcie->max_tbs = IWL_TFH_NUM_TBS;
trans_pcie->tfd_size = sizeof(struct iwl_tfh_tfd);
@@ -3520,9 +3533,7 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
* "dash" value). To keep hw_rev backwards compatible - we'll store it
* in the old format.
*/
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_8000) {
- unsigned long flags;
-
+ if (cfg_trans->device_family >= IWL_DEVICE_FAMILY_8000) {
trans->hw_rev = (trans->hw_rev & 0xfff0) |
(CSR_HW_REV_STEP(trans->hw_rev << 2) << 2);
@@ -3536,98 +3547,15 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
* in-order to recognize C step driver should read chip version
* id located at the AUX bus MISC address space.
*/
- ret = iwl_finish_nic_init(trans);
+ ret = iwl_finish_nic_init(trans, cfg_trans);
if (ret)
goto out_no_pci;
- if (iwl_trans_grab_nic_access(trans, &flags)) {
- u32 hw_step;
-
- hw_step = iwl_read_umac_prph_no_grab(trans,
- WFPM_CTRL_REG);
- hw_step |= ENABLE_WFPM;
- iwl_write_umac_prph_no_grab(trans, WFPM_CTRL_REG,
- hw_step);
- hw_step = iwl_read_prph_no_grab(trans,
- CNVI_AUX_MISC_CHIP);
- hw_step = (hw_step >> HW_STEP_LOCATION_BITS) & 0xF;
- if (hw_step == 0x3)
- trans->hw_rev = (trans->hw_rev & 0xFFFFFFF3) |
- (SILICON_C_STEP << 2);
- iwl_trans_release_nic_access(trans, &flags);
- }
}
IWL_DEBUG_INFO(trans, "HW REV: 0x%0x\n", trans->hw_rev);
-#if IS_ENABLED(CONFIG_IWLMVM)
- trans->hw_rf_id = iwl_read32(trans, CSR_HW_RF_ID);
-
- if (cfg == &iwlax210_2ax_cfg_so_hr_a0) {
- if (trans->hw_rev == CSR_HW_REV_TYPE_TY) {
- trans->cfg = &iwlax210_2ax_cfg_ty_gf_a0;
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_JF)) {
- trans->cfg = &iwlax210_2ax_cfg_so_jf_a0;
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_GF)) {
- trans->cfg = &iwlax211_2ax_cfg_so_gf_a0;
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_GF4)) {
- trans->cfg = &iwlax411_2ax_cfg_so_gf4_a0;
- }
- } else if (cfg == &iwl_ax101_cfg_qu_hr) {
- if ((CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR) &&
- trans->hw_rev == CSR_HW_REV_TYPE_QNJ_B0) ||
- (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR1))) {
- trans->cfg = &iwl22000_2ax_cfg_qnj_hr_b0;
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR)) {
- trans->cfg = &iwl_ax101_cfg_qu_hr;
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_JF)) {
- trans->cfg = &iwl22000_2ax_cfg_jf;
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HRCDB)) {
- IWL_ERR(trans, "RF ID HRCDB is not supported\n");
- ret = -EINVAL;
- goto out_no_pci;
- } else {
- IWL_ERR(trans, "Unrecognized RF ID 0x%08x\n",
- CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id));
- ret = -EINVAL;
- goto out_no_pci;
- }
- } else if (CSR_HW_RF_ID_TYPE_CHIP_ID(trans->hw_rf_id) ==
- CSR_HW_RF_ID_TYPE_CHIP_ID(CSR_HW_RF_ID_TYPE_HR) &&
- ((trans->cfg != &iwl_ax200_cfg_cc &&
- trans->cfg != &killer1650x_2ax_cfg &&
- trans->cfg != &killer1650w_2ax_cfg &&
- trans->cfg != &iwl_ax201_cfg_quz_hr) ||
- trans->hw_rev == CSR_HW_REV_TYPE_QNJ_B0)) {
- u32 hw_status;
-
- hw_status = iwl_read_prph(trans, UMAG_GEN_HW_STATUS);
- if (CSR_HW_RF_STEP(trans->hw_rf_id) == SILICON_B_STEP)
- /*
- * b step fw is the same for physical card and fpga
- */
- trans->cfg = &iwl22000_2ax_cfg_qnj_hr_b0;
- else if ((hw_status & UMAG_GEN_HW_IS_FPGA) &&
- CSR_HW_RF_STEP(trans->hw_rf_id) == SILICON_A_STEP) {
- trans->cfg = &iwl22000_2ax_cfg_qnj_hr_a0_f0;
- } else {
- /*
- * a step no FPGA
- */
- trans->cfg = &iwl22000_2ac_cfg_hr;
- }
- }
-#endif
-
- iwl_pcie_set_interrupt_capa(pdev, trans);
+ iwl_pcie_set_interrupt_capa(pdev, trans, cfg_trans);
trans->hw_id = (pdev->device << 16) + pdev->subsystem_device;
snprintf(trans->hw_id_str, sizeof(trans->hw_id_str),
"PCI ID: 0x%04X:0x%04X", pdev->device, pdev->subsystem_device);
@@ -3635,7 +3563,7 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
/* Initialize the wait queue for commands */
init_waitqueue_head(&trans_pcie->wait_command_queue);
- init_waitqueue_head(&trans_pcie->d0i3_waitq);
+ init_waitqueue_head(&trans_pcie->sx_waitq);
if (trans_pcie->msix_enabled) {
ret = iwl_pcie_init_msix_handler(pdev, trans_pcie);
@@ -3661,12 +3589,6 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev,
WQ_HIGHPRI | WQ_UNBOUND, 1);
INIT_WORK(&trans_pcie->rba.rx_alloc, iwl_pcie_rx_allocator_work);
-#ifdef CONFIG_IWLWIFI_PCIE_RTPM
- trans->runtime_pm_mode = IWL_PLAT_PM_MODE_D0I3;
-#else
- trans->runtime_pm_mode = IWL_PLAT_PM_MODE_DISABLED;
-#endif /* CONFIG_IWLWIFI_PCIE_RTPM */
-
#ifdef CONFIG_IWLWIFI_DEBUGFS
trans_pcie->fw_mon_data.state = IWL_FW_MON_DBGFS_STATE_CLOSED;
mutex_init(&trans_pcie->fw_mon_data.mutex);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c b/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c
index 38d1103..8894027 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c
@@ -50,7 +50,6 @@
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
*****************************************************************************/
-#include <linux/pm_runtime.h>
#include <net/tso.h>
#include <linux/tcp.h>
@@ -99,10 +98,7 @@ void iwl_pcie_gen2_update_byte_tbl(struct iwl_trans_pcie *trans_pcie,
u16 len = byte_cnt;
__le16 bc_ent;
- if (trans_pcie->bc_table_dword)
- len = DIV_ROUND_UP(len, 4);
-
- if (WARN_ON(len > 0xFFF || idx >= txq->n_window))
+ if (WARN(idx >= txq->n_window, "%d >= %d\n", idx, txq->n_window))
return;
filled_tfd_size = offsetof(struct iwl_tfh_tfd, tbs) +
@@ -117,11 +113,20 @@ void iwl_pcie_gen2_update_byte_tbl(struct iwl_trans_pcie *trans_pcie,
*/
num_fetch_chunks = DIV_ROUND_UP(filled_tfd_size, 64) - 1;
- bc_ent = cpu_to_le16(len | (num_fetch_chunks << 12));
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
+ /* Starting from 22560, the HW expects bytes */
+ WARN_ON(trans_pcie->bc_table_dword);
+ WARN_ON(len > 0x3FFF);
+ bc_ent = cpu_to_le16(len | (num_fetch_chunks << 14));
scd_bc_tbl_gen3->tfd_offset[idx] = bc_ent;
- else
+ } else {
+ /* Until 22560, the HW expects DW */
+ WARN_ON(!trans_pcie->bc_table_dword);
+ len = DIV_ROUND_UP(len, 4);
+ WARN_ON(len > 0xFFF);
+ bc_ent = cpu_to_le16(len | (num_fetch_chunks << 12));
scd_bc_tbl->tfd_offset[idx] = bc_ent;
+ }
}
/*
@@ -542,7 +547,7 @@ struct iwl_tfh_tfd *iwl_pcie_gen2_build_tfd(struct iwl_trans *trans,
memset(tfd, 0, sizeof(*tfd));
- if (trans->cfg->device_family < IWL_DEVICE_FAMILY_22560)
+ if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_22560)
len = sizeof(struct iwl_tx_cmd_gen2);
else
len = sizeof(struct iwl_tx_cmd_gen3);
@@ -624,7 +629,7 @@ int iwl_trans_pcie_gen2_tx(struct iwl_trans *trans, struct sk_buff *skb,
return -1;
}
- if (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
+ if (trans->trans_cfg->device_family >= IWL_DEVICE_FAMILY_22560) {
struct iwl_tx_cmd_gen3 *tx_cmd_gen3 =
(void *)dev_cmd->payload;
@@ -641,12 +646,8 @@ int iwl_trans_pcie_gen2_tx(struct iwl_trans *trans, struct sk_buff *skb,
iwl_pcie_gen2_get_num_tbs(trans, tfd));
/* start timer if queue currently empty */
- if (txq->read_ptr == txq->write_ptr) {
- if (txq->wd_timeout)
- mod_timer(&txq->stuck_timer, jiffies + txq->wd_timeout);
- IWL_DEBUG_RPM(trans, "Q: %d first tx - take ref\n", txq->id);
- iwl_trans_ref(trans);
- }
+ if (txq->read_ptr == txq->write_ptr && txq->wd_timeout)
+ mod_timer(&txq->stuck_timer, jiffies + txq->wd_timeout);
/* Tell device the write index *just past* this latest filled TFD */
txq->write_ptr = iwl_queue_inc_wrap(trans, txq->write_ptr);
@@ -891,12 +892,6 @@ static int iwl_pcie_gen2_enqueue_hcmd(struct iwl_trans *trans,
mod_timer(&txq->stuck_timer, jiffies + txq->wd_timeout);
spin_lock_irqsave(&trans_pcie->reg_lock, flags);
- if (!(cmd->flags & CMD_SEND_IN_IDLE) &&
- !trans_pcie->ref_cmd_in_flight) {
- trans_pcie->ref_cmd_in_flight = true;
- IWL_DEBUG_RPM(trans, "set ref_cmd_in_flight - ref\n");
- iwl_trans_ref(trans);
- }
/* Increment and update queue's write index */
txq->write_ptr = iwl_queue_inc_wrap(trans, txq->write_ptr);
iwl_pcie_gen2_txq_inc_wr_ptr(trans, txq);
@@ -930,16 +925,6 @@ static int iwl_pcie_gen2_send_hcmd_sync(struct iwl_trans *trans,
IWL_DEBUG_INFO(trans, "Setting HCMD_ACTIVE for command %s\n", cmd_str);
- if (pm_runtime_suspended(&trans_pcie->pci_dev->dev)) {
- ret = wait_event_timeout(trans_pcie->d0i3_waitq,
- pm_runtime_active(&trans_pcie->pci_dev->dev),
- msecs_to_jiffies(IWL_TRANS_IDLE_TIMEOUT));
- if (!ret) {
- IWL_ERR(trans, "Timeout exiting D0i3 before hcmd\n");
- return -ETIMEDOUT;
- }
- }
-
cmd_idx = iwl_pcie_gen2_enqueue_hcmd(trans, cmd);
if (cmd_idx < 0) {
ret = cmd_idx;
@@ -1064,23 +1049,6 @@ void iwl_pcie_gen2_txq_unmap(struct iwl_trans *trans, int txq_id)
}
iwl_pcie_gen2_free_tfd(trans, txq);
txq->read_ptr = iwl_queue_inc_wrap(trans, txq->read_ptr);
-
- if (txq->read_ptr == txq->write_ptr) {
- unsigned long flags;
-
- spin_lock_irqsave(&trans_pcie->reg_lock, flags);
- if (txq_id != trans_pcie->cmd_queue) {
- IWL_DEBUG_RPM(trans, "Q %d - last tx freed\n",
- txq->id);
- iwl_trans_unref(trans);
- } else if (trans_pcie->ref_cmd_in_flight) {
- trans_pcie->ref_cmd_in_flight = false;
- IWL_DEBUG_RPM(trans,
- "clear ref_cmd_in_flight\n");
- iwl_trans_unref(trans);
- }
- spin_unlock_irqrestore(&trans_pcie->reg_lock, flags);
- }
}
while (!skb_queue_empty(&txq->overflow_q)) {
@@ -1161,7 +1129,7 @@ int iwl_trans_pcie_dyn_txq_alloc_dma(struct iwl_trans *trans,
if (!txq)
return -ENOMEM;
ret = iwl_pcie_alloc_dma_ptr(trans, &txq->bc_tbl,
- (trans->cfg->device_family >=
+ (trans->trans_cfg->device_family >=
IWL_DEVICE_FAMILY_22560) ?
sizeof(struct iwl_gen3_bc_tbl) :
sizeof(struct iwlagn_scd_bc_tbl));
@@ -1225,7 +1193,7 @@ int iwl_trans_pcie_txq_alloc_response(struct iwl_trans *trans,
txq->id = qid;
trans_pcie->txq[qid] = txq;
- wr_ptr &= (trans->cfg->base_params->max_tfd_queue_size - 1);
+ wr_ptr &= (trans->trans_cfg->base_params->max_tfd_queue_size - 1);
/* Place first TFD at index corresponding to start sequence number */
txq->read_ptr = wr_ptr;
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
index 2f0ba7e..4806a04 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
@@ -65,7 +65,6 @@
#include <linux/ieee80211.h>
#include <linux/slab.h>
#include <linux/sched.h>
-#include <linux/pm_runtime.h>
#include <net/ip6_checksum.h>
#include <net/tso.h>
@@ -114,17 +113,17 @@ int iwl_queue_space(struct iwl_trans *trans, const struct iwl_txq *q)
* If q->n_window is smaller than max_tfd_queue_size, there is no need
* to reserve any queue entries for this purpose.
*/
- if (q->n_window < trans->cfg->base_params->max_tfd_queue_size)
+ if (q->n_window < trans->trans_cfg->base_params->max_tfd_queue_size)
max = q->n_window;
else
- max = trans->cfg->base_params->max_tfd_queue_size - 1;
+ max = trans->trans_cfg->base_params->max_tfd_queue_size - 1;
/*
* max_tfd_queue_size is a power of 2, so the following is equivalent to
* modulo by max_tfd_queue_size and is well defined.
*/
used = (q->write_ptr - q->read_ptr) &
- (trans->cfg->base_params->max_tfd_queue_size - 1);
+ (trans->trans_cfg->base_params->max_tfd_queue_size - 1);
if (WARN_ON(used > max))
return 0;
@@ -293,7 +292,7 @@ static void iwl_pcie_txq_inc_wr_ptr(struct iwl_trans *trans,
* 2. NIC is woken up for CMD regardless of shadow outside this function
* 3. there is a chance that the NIC is asleep
*/
- if (!trans->cfg->base_params->shadow_reg_enable &&
+ if (!trans->trans_cfg->base_params->shadow_reg_enable &&
txq_id != trans_pcie->cmd_queue &&
test_bit(STATUS_TPOWER_PMI, &trans->status)) {
/*
@@ -307,7 +306,7 @@ static void iwl_pcie_txq_inc_wr_ptr(struct iwl_trans *trans,
IWL_DEBUG_INFO(trans, "Tx queue %d requesting wakeup, GP1 = 0x%x\n",
txq_id, reg);
iwl_set_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
txq->need_update = true;
return;
}
@@ -328,7 +327,7 @@ void iwl_pcie_txq_check_wrptrs(struct iwl_trans *trans)
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
int i;
- for (i = 0; i < trans->cfg->base_params->num_of_queues; i++) {
+ for (i = 0; i < trans->trans_cfg->base_params->num_of_queues; i++) {
struct iwl_txq *txq = trans_pcie->txq[i];
if (!test_bit(i, trans_pcie->queue_used))
@@ -347,7 +346,7 @@ static inline dma_addr_t iwl_pcie_tfd_tb_get_addr(struct iwl_trans *trans,
void *_tfd, u8 idx)
{
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
struct iwl_tfh_tfd *tfd = _tfd;
struct iwl_tfh_tb *tb = &tfd->tbs[idx];
@@ -390,7 +389,7 @@ static inline void iwl_pcie_tfd_set_tb(struct iwl_trans *trans, void *tfd,
static inline u8 iwl_pcie_tfd_get_num_tbs(struct iwl_trans *trans, void *_tfd)
{
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
struct iwl_tfh_tfd *tfd = _tfd;
return le16_to_cpu(tfd->num_tbs) & 0x1f;
@@ -437,7 +436,7 @@ static void iwl_pcie_tfd_unmap(struct iwl_trans *trans,
meta->tbs = 0;
- if (trans->cfg->use_tfh) {
+ if (trans->trans_cfg->use_tfh) {
struct iwl_tfh_tfd *tfd_fh = (void *)tfd;
tfd_fh->num_tbs = 0;
@@ -525,14 +524,14 @@ int iwl_pcie_txq_alloc(struct iwl_trans *trans, struct iwl_txq *txq,
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
size_t tfd_sz = trans_pcie->tfd_size *
- trans->cfg->base_params->max_tfd_queue_size;
+ trans->trans_cfg->base_params->max_tfd_queue_size;
size_t tb0_buf_sz;
int i;
if (WARN_ON(txq->entries || txq->tfds))
return -EINVAL;
- if (trans->cfg->use_tfh)
+ if (trans->trans_cfg->use_tfh)
tfd_sz = trans_pcie->tfd_size * slots_num;
timer_setup(&txq->stuck_timer, iwl_pcie_txq_stuck_timer, 0);
@@ -591,7 +590,8 @@ int iwl_pcie_txq_init(struct iwl_trans *trans, struct iwl_txq *txq,
int slots_num, bool cmd_queue)
{
int ret;
- u32 tfd_queue_max_size = trans->cfg->base_params->max_tfd_queue_size;
+ u32 tfd_queue_max_size =
+ trans->trans_cfg->base_params->max_tfd_queue_size;
txq->need_update = false;
@@ -639,20 +639,14 @@ static void iwl_pcie_clear_cmd_in_flight(struct iwl_trans *trans)
lockdep_assert_held(&trans_pcie->reg_lock);
- if (trans_pcie->ref_cmd_in_flight) {
- trans_pcie->ref_cmd_in_flight = false;
- IWL_DEBUG_RPM(trans, "clear ref_cmd_in_flight - unref\n");
- iwl_trans_unref(trans);
- }
-
- if (!trans->cfg->base_params->apmg_wake_up_wa)
+ if (!trans->trans_cfg->base_params->apmg_wake_up_wa)
return;
if (WARN_ON(!trans_pcie->cmd_hold_nic_awake))
return;
trans_pcie->cmd_hold_nic_awake = false;
__iwl_trans_pcie_clear_bit(trans, CSR_GP_CNTRL,
- BIT(trans->cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
}
/*
@@ -683,13 +677,8 @@ static void iwl_pcie_txq_unmap(struct iwl_trans *trans, int txq_id)
unsigned long flags;
spin_lock_irqsave(&trans_pcie->reg_lock, flags);
- if (txq_id != trans_pcie->cmd_queue) {
- IWL_DEBUG_RPM(trans, "Q %d - last tx freed\n",
- txq->id);
- iwl_trans_unref(trans);
- } else {
+ if (txq_id == trans_pcie->cmd_queue)
iwl_pcie_clear_cmd_in_flight(trans);
- }
spin_unlock_irqrestore(&trans_pcie->reg_lock, flags);
}
}
@@ -737,7 +726,7 @@ static void iwl_pcie_txq_free(struct iwl_trans *trans, int txq_id)
if (txq->tfds) {
dma_free_coherent(dev,
trans_pcie->tfd_size *
- trans->cfg->base_params->max_tfd_queue_size,
+ trans->trans_cfg->base_params->max_tfd_queue_size,
txq->tfds, txq->dma_addr);
txq->dma_addr = 0;
txq->tfds = NULL;
@@ -759,7 +748,7 @@ static void iwl_pcie_txq_free(struct iwl_trans *trans, int txq_id)
void iwl_pcie_tx_start(struct iwl_trans *trans, u32 scd_base_addr)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- int nq = trans->cfg->base_params->num_of_queues;
+ int nq = trans->trans_cfg->base_params->num_of_queues;
int chan;
u32 reg_val;
int clear_dwords = (SCD_TRANS_TBL_OFFSET_QUEUE(nq) -
@@ -786,7 +775,7 @@ void iwl_pcie_tx_start(struct iwl_trans *trans, u32 scd_base_addr)
/* The chain extension of the SCD doesn't work well. This feature is
* enabled by default by the HW, so we need to disable it manually.
*/
- if (trans->cfg->base_params->scd_chain_ext_wa)
+ if (trans->trans_cfg->base_params->scd_chain_ext_wa)
iwl_write_prph(trans, SCD_CHAINEXT_EN, 0);
iwl_trans_ac_txq_enable(trans, trans_pcie->cmd_queue,
@@ -808,7 +797,7 @@ void iwl_pcie_tx_start(struct iwl_trans *trans, u32 scd_base_addr)
reg_val | FH_TX_CHICKEN_BITS_SCD_AUTO_RETRY_EN);
/* Enable L1-Active */
- if (trans->cfg->device_family < IWL_DEVICE_FAMILY_8000)
+ if (trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_8000)
iwl_clear_bits_prph(trans, APMG_PCIDEV_STT_REG,
APMG_PCIDEV_STT_VAL_L1_ACT_DIS);
}
@@ -822,13 +811,13 @@ void iwl_trans_pcie_tx_reset(struct iwl_trans *trans)
* we should never get here in gen2 trans mode return early to avoid
* having invalid accesses
*/
- if (WARN_ON_ONCE(trans->cfg->gen2))
+ if (WARN_ON_ONCE(trans->trans_cfg->gen2))
return;
- for (txq_id = 0; txq_id < trans->cfg->base_params->num_of_queues;
+ for (txq_id = 0; txq_id < trans->trans_cfg->base_params->num_of_queues;
txq_id++) {
struct iwl_txq *txq = trans_pcie->txq[txq_id];
- if (trans->cfg->use_tfh)
+ if (trans->trans_cfg->use_tfh)
iwl_write_direct64(trans,
FH_MEM_CBBC_QUEUE(trans, txq_id),
txq->dma_addr);
@@ -911,7 +900,7 @@ int iwl_pcie_tx_stop(struct iwl_trans *trans)
return 0;
/* Unmap DMA from host system and free skb's */
- for (txq_id = 0; txq_id < trans->cfg->base_params->num_of_queues;
+ for (txq_id = 0; txq_id < trans->trans_cfg->base_params->num_of_queues;
txq_id++)
iwl_pcie_txq_unmap(trans, txq_id);
@@ -933,7 +922,7 @@ void iwl_pcie_tx_free(struct iwl_trans *trans)
/* Tx queues */
if (trans_pcie->txq_memory) {
for (txq_id = 0;
- txq_id < trans->cfg->base_params->num_of_queues;
+ txq_id < trans->trans_cfg->base_params->num_of_queues;
txq_id++) {
iwl_pcie_txq_free(trans, txq_id);
trans_pcie->txq[txq_id] = NULL;
@@ -957,9 +946,10 @@ static int iwl_pcie_tx_alloc(struct iwl_trans *trans)
int ret;
int txq_id, slots_num;
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- u16 bc_tbls_size = trans->cfg->base_params->num_of_queues;
+ u16 bc_tbls_size = trans->trans_cfg->base_params->num_of_queues;
- bc_tbls_size *= (trans->cfg->device_family >= IWL_DEVICE_FAMILY_22560) ?
+ bc_tbls_size *= (trans->trans_cfg->device_family >=
+ IWL_DEVICE_FAMILY_22560) ?
sizeof(struct iwl_gen3_bc_tbl) :
sizeof(struct iwlagn_scd_bc_tbl);
@@ -984,8 +974,9 @@ static int iwl_pcie_tx_alloc(struct iwl_trans *trans)
goto error;
}
- trans_pcie->txq_memory = kcalloc(trans->cfg->base_params->num_of_queues,
- sizeof(struct iwl_txq), GFP_KERNEL);
+ trans_pcie->txq_memory =
+ kcalloc(trans->trans_cfg->base_params->num_of_queues,
+ sizeof(struct iwl_txq), GFP_KERNEL);
if (!trans_pcie->txq_memory) {
IWL_ERR(trans, "Not enough memory for txq\n");
ret = -ENOMEM;
@@ -993,7 +984,7 @@ static int iwl_pcie_tx_alloc(struct iwl_trans *trans)
}
/* Alloc and init all Tx queues, including the command queue (#4/#9) */
- for (txq_id = 0; txq_id < trans->cfg->base_params->num_of_queues;
+ for (txq_id = 0; txq_id < trans->trans_cfg->base_params->num_of_queues;
txq_id++) {
bool cmd_queue = (txq_id == trans_pcie->cmd_queue);
@@ -1047,7 +1038,7 @@ int iwl_pcie_tx_init(struct iwl_trans *trans)
spin_unlock(&trans_pcie->irq_lock);
/* Alloc and init all Tx queues, including the command queue (#4/#9) */
- for (txq_id = 0; txq_id < trans->cfg->base_params->num_of_queues;
+ for (txq_id = 0; txq_id < trans->trans_cfg->base_params->num_of_queues;
txq_id++) {
bool cmd_queue = (txq_id == trans_pcie->cmd_queue);
@@ -1075,7 +1066,7 @@ int iwl_pcie_tx_init(struct iwl_trans *trans)
}
iwl_set_bits_prph(trans, SCD_GP_CTRL, SCD_GP_CTRL_AUTO_ACTIVE_MODE);
- if (trans->cfg->base_params->num_of_queues > 20)
+ if (trans->trans_cfg->base_params->num_of_queues > 20)
iwl_set_bits_prph(trans, SCD_GP_CTRL,
SCD_GP_CTRL_ENABLE_31_QUEUES);
@@ -1147,7 +1138,7 @@ void iwl_trans_pcie_reclaim(struct iwl_trans *trans, int txq_id, int ssn,
IWL_ERR(trans,
"%s: Read index for txq id (%d), last_to_free %d is out of range [0-%d] %d %d.\n",
__func__, txq_id, last_to_free,
- trans->cfg->base_params->max_tfd_queue_size,
+ trans->trans_cfg->base_params->max_tfd_queue_size,
txq->write_ptr, txq->read_ptr);
goto out;
}
@@ -1170,7 +1161,7 @@ void iwl_trans_pcie_reclaim(struct iwl_trans *trans, int txq_id, int ssn,
txq->entries[read_ptr].skb = NULL;
- if (!trans->cfg->use_tfh)
+ if (!trans->trans_cfg->use_tfh)
iwl_pcie_txq_inval_byte_cnt_tbl(trans, txq);
iwl_pcie_txq_free_tfd(trans, txq);
@@ -1225,20 +1216,28 @@ void iwl_trans_pcie_reclaim(struct iwl_trans *trans, int txq_id, int ssn,
txq->overflow_tx = false;
}
- if (txq->read_ptr == txq->write_ptr) {
- IWL_DEBUG_RPM(trans, "Q %d - last tx reclaimed\n", txq->id);
- iwl_trans_unref(trans);
- }
-
out:
spin_unlock_bh(&txq->lock);
}
+/* Set wr_ptr of specific device and txq */
+void iwl_trans_pcie_set_q_ptrs(struct iwl_trans *trans, int txq_id, int ptr)
+{
+ struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
+ struct iwl_txq *txq = trans_pcie->txq[txq_id];
+
+ spin_lock_bh(&txq->lock);
+
+ txq->write_ptr = ptr;
+ txq->read_ptr = txq->write_ptr;
+
+ spin_unlock_bh(&txq->lock);
+}
+
static int iwl_pcie_set_cmd_in_flight(struct iwl_trans *trans,
const struct iwl_host_cmd *cmd)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- const struct iwl_cfg *cfg = trans->cfg;
int ret;
lockdep_assert_held(&trans_pcie->reg_lock);
@@ -1247,32 +1246,25 @@ static int iwl_pcie_set_cmd_in_flight(struct iwl_trans *trans,
if (test_bit(STATUS_TRANS_DEAD, &trans->status))
return -ENODEV;
- if (!(cmd->flags & CMD_SEND_IN_IDLE) &&
- !trans_pcie->ref_cmd_in_flight) {
- trans_pcie->ref_cmd_in_flight = true;
- IWL_DEBUG_RPM(trans, "set ref_cmd_in_flight - ref\n");
- iwl_trans_ref(trans);
- }
-
/*
* wake up the NIC to make sure that the firmware will see the host
* command - we will let the NIC sleep once all the host commands
* returned. This needs to be done only on NICs that have
* apmg_wake_up_wa set.
*/
- if (cfg->base_params->apmg_wake_up_wa &&
+ if (trans->trans_cfg->base_params->apmg_wake_up_wa &&
!trans_pcie->cmd_hold_nic_awake) {
__iwl_trans_pcie_set_bit(trans, CSR_GP_CNTRL,
- BIT(cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
ret = iwl_poll_bit(trans, CSR_GP_CNTRL,
- BIT(cfg->csr->flag_val_mac_access_en),
- (BIT(cfg->csr->flag_mac_clock_ready) |
+ BIT(trans->trans_cfg->csr->flag_val_mac_access_en),
+ (BIT(trans->trans_cfg->csr->flag_mac_clock_ready) |
CSR_GP_CNTRL_REG_FLAG_GOING_TO_SLEEP),
15000);
if (ret < 0) {
__iwl_trans_pcie_clear_bit(trans, CSR_GP_CNTRL,
- BIT(cfg->csr->flag_mac_access_req));
+ BIT(trans->trans_cfg->csr->flag_mac_access_req));
IWL_ERR(trans, "Failed to wake NIC for hcmd\n");
return -EIO;
}
@@ -1302,12 +1294,12 @@ void iwl_pcie_cmdq_reclaim(struct iwl_trans *trans, int txq_id, int idx)
idx = iwl_pcie_get_cmd_index(txq, idx);
r = iwl_pcie_get_cmd_index(txq, txq->read_ptr);
- if (idx >= trans->cfg->base_params->max_tfd_queue_size ||
+ if (idx >= trans->trans_cfg->base_params->max_tfd_queue_size ||
(!iwl_queue_used(txq, idx))) {
WARN_ONCE(test_bit(txq_id, trans_pcie->queue_used),
"%s: Read index for DMA queue txq id (%d), index %d is out of range [0-%d] %d %d.\n",
__func__, txq_id, idx,
- trans->cfg->base_params->max_tfd_queue_size,
+ trans->trans_cfg->base_params->max_tfd_queue_size,
txq->write_ptr, txq->read_ptr);
return;
}
@@ -1421,7 +1413,7 @@ bool iwl_trans_pcie_txq_enable(struct iwl_trans *trans, int txq_id, u16 ssn,
* this sad hardware issue.
* This bug has been fixed on devices 9000 and up.
*/
- scd_bug = !trans->cfg->mq_rx_supported &&
+ scd_bug = !trans->trans_cfg->mq_rx_supported &&
!((ssn - txq->write_ptr) & 0x3f) &&
(ssn != txq->write_ptr);
if (scd_bug)
@@ -1867,20 +1859,6 @@ void iwl_pcie_hcmd_complete(struct iwl_trans *trans,
wake_up(&trans_pcie->wait_command_queue);
}
- if (meta->flags & CMD_MAKE_TRANS_IDLE) {
- IWL_DEBUG_INFO(trans, "complete %s - mark trans as idle\n",
- iwl_get_cmd_string(trans, cmd->hdr.cmd));
- set_bit(STATUS_TRANS_IDLE, &trans->status);
- wake_up(&trans_pcie->d0i3_waitq);
- }
-
- if (meta->flags & CMD_WAKE_UP_TRANS) {
- IWL_DEBUG_INFO(trans, "complete %s - clear trans idle flag\n",
- iwl_get_cmd_string(trans, cmd->hdr.cmd));
- clear_bit(STATUS_TRANS_IDLE, &trans->status);
- wake_up(&trans_pcie->d0i3_waitq);
- }
-
meta->flags = 0;
spin_unlock_bh(&txq->lock);
@@ -1927,16 +1905,6 @@ static int iwl_pcie_send_hcmd_sync(struct iwl_trans *trans,
IWL_DEBUG_INFO(trans, "Setting HCMD_ACTIVE for command %s\n",
iwl_get_cmd_string(trans, cmd->id));
- if (pm_runtime_suspended(&trans_pcie->pci_dev->dev)) {
- ret = wait_event_timeout(trans_pcie->d0i3_waitq,
- pm_runtime_active(&trans_pcie->pci_dev->dev),
- msecs_to_jiffies(IWL_TRANS_IDLE_TIMEOUT));
- if (!ret) {
- IWL_ERR(trans, "Timeout exiting D0i3 before hcmd\n");
- return -ETIMEDOUT;
- }
- }
-
cmd_idx = iwl_pcie_enqueue_hcmd(trans, cmd);
if (cmd_idx < 0) {
ret = cmd_idx;
@@ -2504,22 +2472,18 @@ int iwl_trans_pcie_tx(struct iwl_trans *trans, struct sk_buff *skb,
wait_write_ptr = ieee80211_has_morefrags(fc);
/* start timer if queue currently empty */
- if (txq->read_ptr == txq->write_ptr) {
- if (txq->wd_timeout) {
- /*
- * If the TXQ is active, then set the timer, if not,
- * set the timer in remainder so that the timer will
- * be armed with the right value when the station will
- * wake up.
- */
- if (!txq->frozen)
- mod_timer(&txq->stuck_timer,
- jiffies + txq->wd_timeout);
- else
- txq->frozen_expiry_remainder = txq->wd_timeout;
- }
- IWL_DEBUG_RPM(trans, "Q: %d first tx - take ref\n", txq->id);
- iwl_trans_ref(trans);
+ if (txq->read_ptr == txq->write_ptr && txq->wd_timeout) {
+ /*
+ * If the TXQ is active, then set the timer, if not,
+ * set the timer in remainder so that the timer will
+ * be armed with the right value when the station will
+ * wake up.
+ */
+ if (!txq->frozen)
+ mod_timer(&txq->stuck_timer,
+ jiffies + txq->wd_timeout);
+ else
+ txq->frozen_expiry_remainder = txq->wd_timeout;
}
/* Tell device the write index *just past* this latest filled TFD */
diff --git a/drivers/net/wireless/intersil/hostap/hostap_download.c b/drivers/net/wireless/intersil/hostap/hostap_download.c
index 4507614..8722000 100644
--- a/drivers/net/wireless/intersil/hostap/hostap_download.c
+++ b/drivers/net/wireless/intersil/hostap/hostap_download.c
@@ -407,10 +407,8 @@ static int prism2_enable_genesis(local_info_t *local, int hcr)
hcr);
return 0;
} else {
- printk(KERN_DEBUG "Readback test failed, HCR 0x%02x "
- "write %02x %02x %02x %02x read %02x %02x %02x %02x\n",
- hcr, initseq[0], initseq[1], initseq[2], initseq[3],
- readbuf[0], readbuf[1], readbuf[2], readbuf[3]);
+ printk(KERN_DEBUG "Readback test failed, HCR 0x%02x write %4ph read %4ph\n",
+ hcr, initseq, readbuf);
return 1;
}
}
diff --git a/drivers/net/wireless/intersil/hostap/hostap_plx.c b/drivers/net/wireless/intersil/hostap/hostap_plx.c
index 943070d..841cfc68 100644
--- a/drivers/net/wireless/intersil/hostap/hostap_plx.c
+++ b/drivers/net/wireless/intersil/hostap/hostap_plx.c
@@ -352,8 +352,7 @@ static int prism2_plx_check_cis(void __iomem *attr_mem, int attr_len,
/* read CIS; it is in even offsets in the beginning of attr_mem */
for (i = 0; i < CIS_MAX_LEN; i++)
cis[i] = readb(attr_mem + 2 * i);
- printk(KERN_DEBUG "%s: CIS: %02x %02x %02x %02x %02x %02x ...\n",
- dev_info, cis[0], cis[1], cis[2], cis[3], cis[4], cis[5]);
+ printk(KERN_DEBUG "%s: CIS: %6ph ...\n", dev_info, cis);
/* set reasonable defaults for Prism2 cards just in case CIS parsing
* fails */
diff --git a/drivers/net/wireless/intersil/hostap/hostap_proc.c b/drivers/net/wireless/intersil/hostap/hostap_proc.c
index 703d74c..6151d8d 100644
--- a/drivers/net/wireless/intersil/hostap/hostap_proc.c
+++ b/drivers/net/wireless/intersil/hostap/hostap_proc.c
@@ -234,7 +234,7 @@ static int prism2_io_debug_proc_read(char *page, char **start, off_t off,
{
local_info_t *local = (local_info_t *) data;
int head = local->io_debug_head;
- int start_bytes, left, copy, copied;
+ int start_bytes, left, copy;
if (off + count > PRISM2_IO_DEBUG_SIZE * 4) {
*eof = 1;
@@ -243,7 +243,6 @@ static int prism2_io_debug_proc_read(char *page, char **start, off_t off,
count = PRISM2_IO_DEBUG_SIZE * 4 - off;
}
- copied = 0;
start_bytes = (PRISM2_IO_DEBUG_SIZE - head) * 4;
left = count;
diff --git a/drivers/net/wireless/mediatek/mt76/agg-rx.c b/drivers/net/wireless/mediatek/mt76/agg-rx.c
index 27e3ff0..8f3d36a 100644
--- a/drivers/net/wireless/mediatek/mt76/agg-rx.c
+++ b/drivers/net/wireless/mediatek/mt76/agg-rx.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76.h"
@@ -34,8 +23,9 @@ mt76_aggr_release(struct mt76_rx_tid *tid, struct sk_buff_head *frames, int idx)
}
static void
-mt76_rx_aggr_release_frames(struct mt76_rx_tid *tid, struct sk_buff_head *frames,
- u16 head)
+mt76_rx_aggr_release_frames(struct mt76_rx_tid *tid,
+ struct sk_buff_head *frames,
+ u16 head)
{
int idx;
@@ -74,15 +64,14 @@ mt76_rx_aggr_check_release(struct mt76_rx_tid *tid, struct sk_buff_head *frames)
for (idx = (tid->head + 1) % tid->size;
idx != start && nframes;
idx = (idx + 1) % tid->size) {
-
skb = tid->reorder_buf[idx];
if (!skb)
continue;
nframes--;
- status = (struct mt76_rx_status *) skb->cb;
- if (!time_after(jiffies, status->reorder_time +
- REORDER_TIMEOUT))
+ status = (struct mt76_rx_status *)skb->cb;
+ if (!time_after(jiffies,
+ status->reorder_time + REORDER_TIMEOUT))
continue;
mt76_rx_aggr_release_frames(tid, frames, status->seqno);
@@ -122,8 +111,8 @@ mt76_rx_aggr_reorder_work(struct work_struct *work)
static void
mt76_rx_aggr_check_ctl(struct sk_buff *skb, struct sk_buff_head *frames)
{
- struct mt76_rx_status *status = (struct mt76_rx_status *) skb->cb;
- struct ieee80211_bar *bar = (struct ieee80211_bar *) skb->data;
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
+ struct ieee80211_bar *bar = (struct ieee80211_bar *)skb->data;
struct mt76_wcid *wcid = status->wcid;
struct mt76_rx_tid *tid;
u16 seqno;
@@ -148,8 +137,8 @@ mt76_rx_aggr_check_ctl(struct sk_buff *skb, struct sk_buff_head *frames)
void mt76_rx_aggr_reorder(struct sk_buff *skb, struct sk_buff_head *frames)
{
- struct mt76_rx_status *status = (struct mt76_rx_status *) skb->cb;
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
struct mt76_wcid *wcid = status->wcid;
struct ieee80211_sta *sta;
struct mt76_rx_tid *tid;
@@ -233,7 +222,8 @@ void mt76_rx_aggr_reorder(struct sk_buff *skb, struct sk_buff_head *frames)
tid->nframes++;
mt76_rx_aggr_release_head(tid, frames);
- ieee80211_queue_delayed_work(tid->dev->hw, &tid->reorder_work, REORDER_TIMEOUT);
+ ieee80211_queue_delayed_work(tid->dev->hw, &tid->reorder_work,
+ REORDER_TIMEOUT);
out:
spin_unlock_bh(&tid->lock);
diff --git a/drivers/net/wireless/mediatek/mt76/debugfs.c b/drivers/net/wireless/mediatek/mt76/debugfs.c
index c6a9fe2..d95b73f 100644
--- a/drivers/net/wireless/mediatek/mt76/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/debugfs.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76.h"
diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c
index d8f61e5..c747eb2 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/dma.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/dma-mapping.h>
@@ -442,6 +431,12 @@ mt76_dma_rx_reset(struct mt76_dev *dev, enum mt76_rxq_id qid)
mt76_dma_rx_cleanup(dev, q);
mt76_dma_sync_idx(dev, q);
mt76_dma_rx_fill(dev, q);
+
+ if (!q->rx_head)
+ return;
+
+ dev_kfree_skb(q->rx_head);
+ q->rx_head = NULL;
}
static void
@@ -504,7 +499,7 @@ mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
skb_reserve(skb, q->buf_offset);
if (q == &dev->q_rx[MT_RXQ_MCU]) {
- u32 *rxfce = (u32 *) skb->cb;
+ u32 *rxfce = (u32 *)skb->cb;
*rxfce = info;
}
diff --git a/drivers/net/wireless/mediatek/mt76/dma.h b/drivers/net/wireless/mediatek/mt76/dma.h
index 03dd2ba..e7c2769 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.h
+++ b/drivers/net/wireless/mediatek/mt76/dma.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76_DMA_H
#define __MT76_DMA_H
diff --git a/drivers/net/wireless/mediatek/mt76/eeprom.c b/drivers/net/wireless/mediatek/mt76/eeprom.c
index b7a49ae..804224e 100644
--- a/drivers/net/wireless/mediatek/mt76/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/eeprom.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/of.h>
#include <linux/of_net.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index ec9efb7..1a2c143 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/of.h>
#include "mt76.h"
@@ -294,6 +283,8 @@ mt76_alloc_device(struct device *pdev, unsigned int size,
init_waitqueue_head(&dev->tx_wait);
skb_queue_head_init(&dev->status_list);
+ tasklet_init(&dev->tx_tasklet, mt76_tx_tasklet, (unsigned long)dev);
+
return dev;
}
EXPORT_SYMBOL_GPL(mt76_alloc_device);
@@ -415,11 +406,6 @@ void mt76_set_channel(struct mt76_dev *dev)
bool offchannel = hw->conf.flags & IEEE80211_CONF_OFFCHANNEL;
int timeout = HZ / 5;
- if (offchannel)
- set_bit(MT76_OFFCHANNEL, &dev->state);
- else
- clear_bit(MT76_OFFCHANNEL, &dev->state);
-
wait_event_timeout(dev->tx_wait, !mt76_has_tx_pending(dev), timeout);
if (dev->drv->update_survey)
@@ -487,9 +473,10 @@ void mt76_wcid_key_setup(struct mt76_dev *dev, struct mt76_wcid *wcid,
if (!key)
return;
- if (key->cipher == WLAN_CIPHER_SUITE_CCMP)
- wcid->rx_check_pn = true;
+ if (key->cipher != WLAN_CIPHER_SUITE_CCMP)
+ return;
+ wcid->rx_check_pn = true;
for (i = 0; i < IEEE80211_NUM_TIDS; i++) {
ieee80211_get_key_rx_seq(key, i, &seq);
memcpy(wcid->rx_key_pn[i], seq.ccmp.pn, sizeof(seq.ccmp.pn));
@@ -497,12 +484,12 @@ void mt76_wcid_key_setup(struct mt76_dev *dev, struct mt76_wcid *wcid,
}
EXPORT_SYMBOL(mt76_wcid_key_setup);
-struct ieee80211_sta *mt76_rx_convert(struct sk_buff *skb)
+static struct ieee80211_sta *mt76_rx_convert(struct sk_buff *skb)
{
struct ieee80211_rx_status *status = IEEE80211_SKB_RXCB(skb);
struct mt76_rx_status mstat;
- mstat = *((struct mt76_rx_status *) skb->cb);
+ mstat = *((struct mt76_rx_status *)skb->cb);
memset(status, 0, sizeof(*status));
status->flag = mstat.flag;
@@ -517,17 +504,18 @@ struct ieee80211_sta *mt76_rx_convert(struct sk_buff *skb)
status->chains = mstat.chains;
BUILD_BUG_ON(sizeof(mstat) > sizeof(skb->cb));
- BUILD_BUG_ON(sizeof(status->chain_signal) != sizeof(mstat.chain_signal));
- memcpy(status->chain_signal, mstat.chain_signal, sizeof(mstat.chain_signal));
+ BUILD_BUG_ON(sizeof(status->chain_signal) !=
+ sizeof(mstat.chain_signal));
+ memcpy(status->chain_signal, mstat.chain_signal,
+ sizeof(mstat.chain_signal));
return wcid_to_sta(mstat.wcid);
}
-EXPORT_SYMBOL(mt76_rx_convert);
static int
mt76_check_ccmp_pn(struct sk_buff *skb)
{
- struct mt76_rx_status *status = (struct mt76_rx_status *) skb->cb;
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
struct mt76_wcid *wcid = status->wcid;
struct ieee80211_hdr *hdr;
int ret;
@@ -543,7 +531,7 @@ mt76_check_ccmp_pn(struct sk_buff *skb)
* Validate the first fragment both here and in mac80211
* All further fragments will be validated by mac80211 only.
*/
- hdr = (struct ieee80211_hdr *) skb->data;
+ hdr = (struct ieee80211_hdr *)skb->data;
if (ieee80211_is_frag(hdr) &&
!ieee80211_is_first_frag(hdr->frame_control))
return 0;
@@ -566,8 +554,8 @@ mt76_check_ccmp_pn(struct sk_buff *skb)
static void
mt76_check_sta(struct mt76_dev *dev, struct sk_buff *skb)
{
- struct mt76_rx_status *status = (struct mt76_rx_status *) skb->cb;
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
struct ieee80211_sta *sta;
struct mt76_wcid *wcid = status->wcid;
bool ps;
@@ -576,13 +564,13 @@ mt76_check_sta(struct mt76_dev *dev, struct sk_buff *skb)
if (ieee80211_is_pspoll(hdr->frame_control) && !wcid) {
sta = ieee80211_find_sta_by_ifaddr(dev->hw, hdr->addr2, NULL);
if (sta)
- wcid = status->wcid = (struct mt76_wcid *) sta->drv_priv;
+ wcid = status->wcid = (struct mt76_wcid *)sta->drv_priv;
}
if (!wcid || !wcid->sta)
return;
- sta = container_of((void *) wcid, struct ieee80211_sta, drv_priv);
+ sta = container_of((void *)wcid, struct ieee80211_sta, drv_priv);
if (status->signal <= 0)
ewma_signal_add(&wcid->rssi, -status->signal);
@@ -598,8 +586,8 @@ mt76_check_sta(struct mt76_dev *dev, struct sk_buff *skb)
}
if (ieee80211_has_morefrags(hdr->frame_control) ||
- !(ieee80211_is_mgmt(hdr->frame_control) ||
- ieee80211_is_data(hdr->frame_control)))
+ !(ieee80211_is_mgmt(hdr->frame_control) ||
+ ieee80211_is_data(hdr->frame_control)))
return;
ps = ieee80211_has_pm(hdr->frame_control);
@@ -628,7 +616,7 @@ mt76_check_sta(struct mt76_dev *dev, struct sk_buff *skb)
if (!sta->txq[i])
continue;
- mtxq = (struct mt76_txq *) sta->txq[i]->drv_priv;
+ mtxq = (struct mt76_txq *)sta->txq[i]->drv_priv;
if (!skb_queue_empty(&mtxq->retry_q))
ieee80211_schedule_txq(dev->hw, sta->txq[i]);
}
@@ -714,6 +702,9 @@ void __mt76_sta_remove(struct mt76_dev *dev, struct ieee80211_vif *vif,
rcu_assign_pointer(dev->wcid[idx], NULL);
synchronize_rcu();
+ for (i = 0; i < ARRAY_SIZE(wcid->aggr); i++)
+ mt76_rx_aggr_stop(dev, wcid, i);
+
if (dev->drv->sta_remove)
dev->drv->sta_remove(dev, vif, sta);
@@ -750,7 +741,7 @@ int mt76_sta_state(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
dev->drv->sta_assoc(dev, vif, sta);
if (old_state == IEEE80211_STA_NONE &&
- new_state == IEEE80211_STA_NOTEXIST)
+ new_state == IEEE80211_STA_NOTEXIST)
mt76_sta_remove(dev, vif, sta);
return 0;
@@ -790,7 +781,7 @@ static void
__mt76_csa_finish(void *priv, u8 *mac, struct ieee80211_vif *vif)
{
if (vif->csa_active && ieee80211_csa_is_complete(vif))
- ieee80211_csa_finish(vif);
+ ieee80211_csa_finish(vif);
}
void mt76_csa_finish(struct mt76_dev *dev)
@@ -878,3 +869,20 @@ int mt76_get_rate(struct mt76_dev *dev,
return 0;
}
EXPORT_SYMBOL_GPL(mt76_get_rate);
+
+void mt76_sw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ const u8 *mac)
+{
+ struct mt76_dev *dev = hw->priv;
+
+ set_bit(MT76_SCANNING, &dev->state);
+}
+EXPORT_SYMBOL_GPL(mt76_sw_scan);
+
+void mt76_sw_scan_complete(struct ieee80211_hw *hw, struct ieee80211_vif *vif)
+{
+ struct mt76_dev *dev = hw->priv;
+
+ clear_bit(MT76_SCANNING, &dev->state);
+}
+EXPORT_SYMBOL_GPL(mt76_sw_scan_complete);
diff --git a/drivers/net/wireless/mediatek/mt76/mcu.c b/drivers/net/wireless/mediatek/mt76/mcu.c
index dbb57b5..2a97668 100644
--- a/drivers/net/wireless/mediatek/mt76/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mcu.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2019 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mmio.c b/drivers/net/wireless/mediatek/mt76/mmio.c
index 38368d1..1c974df 100644
--- a/drivers/net/wireless/mediatek/mt76/mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mmio.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76.h"
@@ -40,10 +29,16 @@ static u32 mt76_mmio_rmw(struct mt76_dev *dev, u32 offset, u32 mask, u32 val)
return val;
}
-static void mt76_mmio_copy(struct mt76_dev *dev, u32 offset, const void *data,
- int len)
+static void mt76_mmio_write_copy(struct mt76_dev *dev, u32 offset,
+ const void *data, int len)
{
- __iowrite32_copy(dev->mmio.regs + offset, data, len >> 2);
+ __iowrite32_copy(dev->mmio.regs + offset, data, DIV_ROUND_UP(len, 4));
+}
+
+static void mt76_mmio_read_copy(struct mt76_dev *dev, u32 offset,
+ void *data, int len)
+{
+ __ioread32_copy(data, dev->mmio.regs + offset, DIV_ROUND_UP(len, 4));
}
static int mt76_mmio_wr_rp(struct mt76_dev *dev, u32 base,
@@ -89,7 +84,8 @@ void mt76_mmio_init(struct mt76_dev *dev, void __iomem *regs)
.rr = mt76_mmio_rr,
.rmw = mt76_mmio_rmw,
.wr = mt76_mmio_wr,
- .copy = mt76_mmio_copy,
+ .write_copy = mt76_mmio_write_copy,
+ .read_copy = mt76_mmio_read_copy,
.wr_rp = mt76_mmio_wr_rp,
.rd_rp = mt76_mmio_rd_rp,
.type = MT76_BUS_MMIO,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 989386e..570c159 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76_H
@@ -49,8 +38,10 @@ struct mt76_bus_ops {
u32 (*rr)(struct mt76_dev *dev, u32 offset);
void (*wr)(struct mt76_dev *dev, u32 offset, u32 val);
u32 (*rmw)(struct mt76_dev *dev, u32 offset, u32 mask, u32 val);
- void (*copy)(struct mt76_dev *dev, u32 offset, const void *data,
- int len);
+ void (*write_copy)(struct mt76_dev *dev, u32 offset, const void *data,
+ int len);
+ void (*read_copy)(struct mt76_dev *dev, u32 offset, void *data,
+ int len);
int (*wr_rp)(struct mt76_dev *dev, u32 base,
const struct mt76_reg_pair *rp, int len);
int (*rd_rp)(struct mt76_dev *dev, u32 base,
@@ -213,6 +204,7 @@ struct mt76_wcid {
u8 rx_check_pn;
u8 rx_key_pn[IEEE80211_NUM_TIDS][6];
+ u16 cipher;
u32 tx_info;
bool sw_iv;
@@ -280,7 +272,6 @@ enum {
MT76_STATE_MCU_RUNNING,
MT76_SCANNING,
MT76_RESET,
- MT76_OFFCHANNEL,
MT76_REMOVED,
MT76_READING_STATS,
};
@@ -390,7 +381,10 @@ enum mt76u_out_ep {
#define MCU_RESP_URB_SIZE 1024
struct mt76_usb {
struct mutex usb_ctrl_mtx;
- u8 data[32];
+ union {
+ u8 data[32];
+ __le32 reg_val;
+ };
struct tasklet_struct rx_tasklet;
struct delayed_work stat_work;
@@ -496,6 +490,8 @@ struct mt76_dev {
u8 csa_complete;
+ ktime_t survey_time;
+
u32 rxfilter;
union {
@@ -538,7 +534,8 @@ struct mt76_rx_status {
#define __mt76_rr(dev, ...) (dev)->bus->rr((dev), __VA_ARGS__)
#define __mt76_wr(dev, ...) (dev)->bus->wr((dev), __VA_ARGS__)
#define __mt76_rmw(dev, ...) (dev)->bus->rmw((dev), __VA_ARGS__)
-#define __mt76_wr_copy(dev, ...) (dev)->bus->copy((dev), __VA_ARGS__)
+#define __mt76_wr_copy(dev, ...) (dev)->bus->write_copy((dev), __VA_ARGS__)
+#define __mt76_rr_copy(dev, ...) (dev)->bus->read_copy((dev), __VA_ARGS__)
#define __mt76_set(dev, offset, val) __mt76_rmw(dev, offset, 0, val)
#define __mt76_clear(dev, offset, val) __mt76_rmw(dev, offset, val, 0)
@@ -546,7 +543,8 @@ struct mt76_rx_status {
#define mt76_rr(dev, ...) (dev)->mt76.bus->rr(&((dev)->mt76), __VA_ARGS__)
#define mt76_wr(dev, ...) (dev)->mt76.bus->wr(&((dev)->mt76), __VA_ARGS__)
#define mt76_rmw(dev, ...) (dev)->mt76.bus->rmw(&((dev)->mt76), __VA_ARGS__)
-#define mt76_wr_copy(dev, ...) (dev)->mt76.bus->copy(&((dev)->mt76), __VA_ARGS__)
+#define mt76_wr_copy(dev, ...) (dev)->mt76.bus->write_copy(&((dev)->mt76), __VA_ARGS__)
+#define mt76_rr_copy(dev, ...) (dev)->mt76.bus->read_copy(&((dev)->mt76), __VA_ARGS__)
#define mt76_wr_rp(dev, ...) (dev)->mt76.bus->wr_rp(&((dev)->mt76), __VA_ARGS__)
#define mt76_rd_rp(dev, ...) (dev)->mt76.bus->rd_rp(&((dev)->mt76), __VA_ARGS__)
@@ -675,7 +673,7 @@ static inline struct mt76_tx_cb *mt76_tx_skb_cb(struct sk_buff *skb)
{
BUILD_BUG_ON(sizeof(struct mt76_tx_cb) >
sizeof(IEEE80211_SKB_CB(skb)->status.status_driver_data));
- return ((void *) IEEE80211_SKB_CB(skb)->status.status_driver_data);
+ return ((void *)IEEE80211_SKB_CB(skb)->status.status_driver_data);
}
static inline void mt76_insert_hdr_pad(struct sk_buff *skb)
@@ -710,6 +708,7 @@ void mt76_stop_tx_queues(struct mt76_dev *dev, struct ieee80211_sta *sta,
bool send_bar);
void mt76_txq_schedule(struct mt76_dev *dev, enum mt76_txq_id qid);
void mt76_txq_schedule_all(struct mt76_dev *dev);
+void mt76_tx_tasklet(unsigned long data);
void mt76_release_buffered_frames(struct ieee80211_hw *hw,
struct ieee80211_sta *sta,
u16 tids, int nframes,
@@ -750,8 +749,6 @@ int mt76_sta_state(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
void __mt76_sta_remove(struct mt76_dev *dev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
-struct ieee80211_sta *mt76_rx_convert(struct sk_buff *skb);
-
int mt76_get_min_avg_rssi(struct mt76_dev *dev);
int mt76_get_txpower(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
@@ -765,6 +762,10 @@ void mt76_insert_ccmp_hdr(struct sk_buff *skb, u8 key_id);
int mt76_get_rate(struct mt76_dev *dev,
struct ieee80211_supported_band *sband,
int idx, bool cck);
+void mt76_sw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ const u8 *mac);
+void mt76_sw_scan_complete(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif);
/* internal */
void mt76_tx_free(struct mt76_dev *dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/Kconfig b/drivers/net/wireless/mediatek/mt76/mt7603/Kconfig
index e108bf8..6a0080f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/Kconfig
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/Kconfig
@@ -5,6 +5,8 @@
depends on MAC80211
depends on PCI
help
- This adds support for MT7603E wireless PCIe devices and the WLAN core on
- MT7628/MT7688 SoC devices
+ This adds support for MT7603E wireless PCIe devices and the WLAN core
+ on MT7628/MT7688 SoC devices. This family supports IEEE 802.11n 2x2
+ to 300Mbps PHY rate
+ To compile this driver as a module, choose M here.
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/beacon.c b/drivers/net/wireless/mediatek/mt76/mt7603/beacon.c
index 58e68fb..7a41cdf 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/beacon.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/beacon.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include "mt7603.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/core.c b/drivers/net/wireless/mediatek/mt76/mt7603/core.c
index e7ee58e..e5af4f3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/core.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/core.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include "mt7603.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7603/debugfs.c
index a1bc310..5942fe7 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/debugfs.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include "mt7603.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/dma.c b/drivers/net/wireless/mediatek/mt76/mt7603/dma.c
index 58dc511..24d82a2 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/dma.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include "mt7603.h"
#include "mac.h"
@@ -63,7 +63,7 @@ mt7603_rx_loopback_skb(struct mt7603_dev *dev, struct sk_buff *skb)
txd[0] = cpu_to_le32(val);
sta = container_of(priv, struct ieee80211_sta, drv_priv);
- hdr = (struct ieee80211_hdr *) &skb->data[MT_TXD_SIZE];
+ hdr = (struct ieee80211_hdr *)&skb->data[MT_TXD_SIZE];
tid = *ieee80211_get_qos_ctl(hdr) & IEEE80211_QOS_CTL_TID_MASK;
ieee80211_sta_set_buffered(sta, tid, true);
@@ -135,14 +135,6 @@ mt7603_init_rx_queue(struct mt7603_dev *dev, struct mt76_queue *q,
return 0;
}
-static void
-mt7603_tx_tasklet(unsigned long data)
-{
- struct mt7603_dev *dev = (struct mt7603_dev *)data;
-
- mt76_txq_schedule_all(&dev->mt76);
-}
-
static int mt7603_poll_tx(struct napi_struct *napi, int budget)
{
struct mt7603_dev *dev;
@@ -178,11 +170,6 @@ int mt7603_dma_init(struct mt7603_dev *dev)
mt76_dma_attach(&dev->mt76);
- init_waitqueue_head(&dev->mt76.mmio.mcu.wait);
- skb_queue_head_init(&dev->mt76.mmio.mcu.res_q);
-
- tasklet_init(&dev->mt76.tx_tasklet, mt7603_tx_tasklet, (unsigned long)dev);
-
mt76_clear(dev, MT_WPDMA_GLO_CFG,
MT_WPDMA_GLO_CFG_TX_DMA_EN |
MT_WPDMA_GLO_CFG_RX_DMA_EN |
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7603/eeprom.c
index 8c120e4..2b6a4d8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/eeprom.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include "mt7603.h"
#include "eeprom.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/init.c b/drivers/net/wireless/mediatek/mt76/mt7603/init.c
index 38834c7..ad2ccdb 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/init.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include <linux/etherdevice.h>
#include "mt7603.h"
@@ -248,8 +248,7 @@ mt7603_mac_init(struct mt7603_dev *dev)
FIELD_PREP(MT_AGG_ARxCR_LIMIT(7), MT7603_RATE_RETRY - 1));
mt76_wr(dev, MT_AGG_ARCR,
- (MT_AGG_ARCR_INIT_RATE1 |
- FIELD_PREP(MT_AGG_ARCR_RTS_RATE_THR, 2) |
+ (FIELD_PREP(MT_AGG_ARCR_RTS_RATE_THR, 2) |
MT_AGG_ARCR_RATE_DOWN_RATIO_EN |
FIELD_PREP(MT_AGG_ARCR_RATE_DOWN_RATIO, 1) |
FIELD_PREP(MT_AGG_ARCR_RATE_UP_EXTRA_TH, 4)));
@@ -507,7 +506,6 @@ mt7603_init_txpower(struct mt7603_dev *dev,
}
}
-
int mt7603_register_device(struct mt7603_dev *dev)
{
struct mt76_bus_ops *bus_ops;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/mac.c b/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
index 40db1cb..c328192 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/mac.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include <linux/etherdevice.h>
#include <linux/timekeeping.h>
@@ -639,9 +639,11 @@ void mt7603_wtbl_set_rates(struct mt7603_dev *dev, struct mt7603_sta *sta,
IEEE80211_TX_RC_40_MHZ_WIDTH)
continue;
+ if (!rates[i].idx)
+ continue;
+
rates[i].idx--;
}
-
}
w9 &= MT_WTBL2_W9_SHORT_GI_20 | MT_WTBL2_W9_SHORT_GI_40 |
@@ -1014,8 +1016,9 @@ mt7603_fill_txs(struct mt7603_dev *dev, struct mt7603_sta *sta,
sta->rate_probe = false;
}
spin_unlock_bh(&dev->mt76.lock);
- } else
+ } else {
info->status.rates[0] = rs->rates[first_idx / 2];
+ }
info->status.rates[0].count = 0;
for (i = 0, idx = first_idx; count && idx <= last_idx; idx++) {
@@ -1470,8 +1473,9 @@ void mt7603_update_channel(struct mt76_dev *mdev)
spin_lock_bh(&dev->mt76.cc_lock);
cur_time = ktime_get_boottime();
state->cc_busy += busy;
- state->cc_active += ktime_to_us(ktime_sub(cur_time, dev->survey_time));
- dev->survey_time = cur_time;
+ state->cc_active += ktime_to_us(ktime_sub(cur_time,
+ dev->mt76.survey_time));
+ dev->mt76.survey_time = cur_time;
spin_unlock_bh(&dev->mt76.cc_lock);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/main.c b/drivers/net/wireless/mediatek/mt76/mt7603/main.c
index e5d4cb6..25d5b160 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/main.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include <linux/etherdevice.h>
#include <linux/platform_device.h>
@@ -14,7 +14,7 @@ mt7603_start(struct ieee80211_hw *hw)
struct mt7603_dev *dev = hw->priv;
mt7603_mac_start(dev);
- dev->survey_time = ktime_get_boottime();
+ dev->mt76.survey_time = ktime_get_boottime();
set_bit(MT76_STATE_RUNNING, &dev->mt76.state);
mt7603_mac_work(&dev->mt76.mac_work.work);
@@ -173,7 +173,7 @@ mt7603_set_channel(struct mt7603_dev *dev, struct cfg80211_chan_def *def)
mt76_txq_schedule_all(&dev->mt76);
ieee80211_queue_delayed_work(mt76_hw(dev), &dev->mt76.mac_work,
- MT7603_WATCHDOG_TIME);
+ msecs_to_jiffies(MT7603_WATCHDOG_TIME));
/* reset channel stats */
mt76_clear(dev, MT_MIB_CTL, MT_MIB_CTL_READ_CLR_DIS);
@@ -182,7 +182,7 @@ mt7603_set_channel(struct mt7603_dev *dev, struct cfg80211_chan_def *def)
mt76_rr(dev, MT_MIB_STAT_PSCCA);
mt7603_cca_stats_reset(dev);
- dev->survey_time = ktime_get_boottime();
+ dev->mt76.survey_time = ktime_get_boottime();
mt7603_init_edcca(dev);
@@ -399,7 +399,7 @@ mt7603_ps_set_more_data(struct sk_buff *skb)
{
struct ieee80211_hdr *hdr;
- hdr = (struct ieee80211_hdr *) &skb->data[MT_TXD_SIZE];
+ hdr = (struct ieee80211_hdr *)&skb->data[MT_TXD_SIZE];
hdr->frame_control |= cpu_to_le16(IEEE80211_FCTL_MOREDATA);
}
@@ -537,23 +537,6 @@ mt7603_conf_tx(struct ieee80211_hw *hw, struct ieee80211_vif *vif, u16 queue,
}
static void
-mt7603_sw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- const u8 *mac)
-{
- struct mt7603_dev *dev = hw->priv;
-
- set_bit(MT76_SCANNING, &dev->mt76.state);
-}
-
-static void
-mt7603_sw_scan_complete(struct ieee80211_hw *hw, struct ieee80211_vif *vif)
-{
- struct mt7603_dev *dev = hw->priv;
-
- clear_bit(MT76_SCANNING, &dev->mt76.state);
-}
-
-static void
mt7603_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
u32 queues, bool drop)
{
@@ -569,7 +552,7 @@ mt7603_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
struct ieee80211_txq *txq = sta->txq[params->tid];
struct mt7603_sta *msta = (struct mt7603_sta *)sta->drv_priv;
u16 tid = params->tid;
- u16 *ssn = ¶ms->ssn;
+ u16 ssn = params->ssn;
u8 ba_size = params->buf_size;
struct mt76_txq *mtxq;
@@ -580,7 +563,7 @@ mt7603_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
switch (action) {
case IEEE80211_AMPDU_RX_START:
- mt76_rx_aggr_start(&dev->mt76, &msta->wcid, tid, *ssn,
+ mt76_rx_aggr_start(&dev->mt76, &msta->wcid, tid, ssn,
params->buf_size);
mt7603_mac_rx_ba_reset(dev, sta->addr, tid);
break;
@@ -595,11 +578,10 @@ mt7603_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
case IEEE80211_AMPDU_TX_STOP_FLUSH:
case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
mtxq->aggr = false;
- ieee80211_send_bar(vif, sta->addr, tid, mtxq->agg_ssn);
mt7603_mac_tx_ba_reset(dev, msta->wcid.idx, tid, -1);
break;
case IEEE80211_AMPDU_TX_START:
- mtxq->agg_ssn = IEEE80211_SN_TO_SEQ(*ssn);
+ mtxq->agg_ssn = IEEE80211_SN_TO_SEQ(ssn);
ieee80211_start_tx_ba_cb_irqsafe(vif, sta->addr, tid);
break;
case IEEE80211_AMPDU_TX_STOP_CONT:
@@ -647,7 +629,8 @@ mt7603_set_coverage_class(struct ieee80211_hw *hw, s16 coverage_class)
mt7603_mac_set_timing(dev);
}
-static void mt7603_tx(struct ieee80211_hw *hw, struct ieee80211_tx_control *control,
+static void mt7603_tx(struct ieee80211_hw *hw,
+ struct ieee80211_tx_control *control,
struct sk_buff *skb)
{
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
@@ -682,8 +665,8 @@ const struct ieee80211_ops mt7603_ops = {
.sta_state = mt76_sta_state,
.set_key = mt7603_set_key,
.conf_tx = mt7603_conf_tx,
- .sw_scan_start = mt7603_sw_scan,
- .sw_scan_complete = mt7603_sw_scan_complete,
+ .sw_scan_start = mt76_sw_scan,
+ .sw_scan_complete = mt76_sw_scan_complete,
.flush = mt7603_flush,
.ampdu_action = mt7603_ampdu_action,
.get_txpower = mt76_get_txpower,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c
index 343ddc5..02b2bd6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/mcu.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include <linux/firmware.h>
#include "mt7603.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/mt7603.h b/drivers/net/wireless/mediatek/mt76/mt7603/mt7603.h
index 2c6f7b4..257300f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/mt7603.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/mt7603.h
@@ -116,7 +116,6 @@ struct mt7603_dev {
s8 tx_power_limit;
- ktime_t survey_time;
ktime_t ed_time;
struct mt76_queue q_rx;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/pci.c b/drivers/net/wireless/mediatek/mt76/mt7603/pci.c
index 4acdbf5..2f2f337 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/pci.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include <linux/kernel.h>
#include <linux/module.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/soc.c b/drivers/net/wireless/mediatek/mt76/mt7603/soc.c
index c6c1ce6..68efb30 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7603/soc.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7603/soc.c
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: ISC */
+// SPDX-License-Identifier: ISC
#include <linux/kernel.h>
#include <linux/module.h>
@@ -9,7 +9,6 @@
static int
mt76_wmac_probe(struct platform_device *pdev)
{
- struct resource *res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
struct mt7603_dev *dev;
void __iomem *mem_base;
struct mt76_dev *mdev;
@@ -20,7 +19,7 @@ mt76_wmac_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
- mem_base = devm_ioremap_resource(&pdev->dev, res);
+ mem_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(mem_base)) {
dev_err(&pdev->dev, "Failed to get memory resource\n");
return PTR_ERR(mem_base);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/Kconfig b/drivers/net/wireless/mediatek/mt76/mt7615/Kconfig
index 2ed47b3..4cabba9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/Kconfig
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/Kconfig
@@ -5,4 +5,9 @@
depends on MAC80211
depends on PCI
help
- This adds support for MT7615-based wireless PCIe devices.
+ This adds support for MT7615-based wireless PCIe devices,
+ which support concurrent dual-band operation at both 5GHz
+ and 2.4GHz, IEEE 802.11ac 4x4:4SS 1733Mbps PHY rate, wave2
+ MU-MIMO up to 4 users/group and 160MHz channels.
+
+ To compile this driver as a module, choose M here.
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/Makefile b/drivers/net/wireless/mediatek/mt76/mt7615/Makefile
index 6397552..5aaac69 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/Makefile
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/Makefile
@@ -2,4 +2,5 @@
obj-$(CONFIG_MT7615E) += mt7615e.o
-mt7615e-y := pci.o init.o dma.o eeprom.o main.o mcu.o mac.o
+mt7615e-y := pci.o init.o dma.o eeprom.o main.o mcu.o mac.o \
+ debugfs.o
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c b/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c
new file mode 100644
index 0000000..2428a46
--- /dev/null
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c
@@ -0,0 +1,91 @@
+// SPDX-License-Identifier: ISC
+
+#include "mt7615.h"
+
+static int
+mt7615_radar_pattern_set(void *data, u64 val)
+{
+ struct mt7615_dev *dev = data;
+
+ return mt7615_mcu_rdd_send_pattern(dev);
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(fops_radar_pattern, NULL,
+ mt7615_radar_pattern_set, "%lld\n");
+
+static int
+mt7615_scs_set(void *data, u64 val)
+{
+ struct mt7615_dev *dev = data;
+
+ mt7615_mac_set_scs(dev, val);
+
+ return 0;
+}
+
+static int
+mt7615_scs_get(void *data, u64 *val)
+{
+ struct mt7615_dev *dev = data;
+
+ *val = dev->scs_en;
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(fops_scs, mt7615_scs_get,
+ mt7615_scs_set, "%lld\n");
+
+static int
+mt7615_radio_read(struct seq_file *s, void *data)
+{
+ struct mt7615_dev *dev = dev_get_drvdata(s->private);
+
+ seq_printf(s, "Sensitivity: ofdm=%d cck=%d\n",
+ dev->ofdm_sensitivity, dev->cck_sensitivity);
+ seq_printf(s, "False CCA: ofdm=%d cck=%d\n",
+ dev->false_cca_ofdm, dev->false_cca_cck);
+
+ return 0;
+}
+
+static int mt7615_read_temperature(struct seq_file *s, void *data)
+{
+ struct mt7615_dev *dev = dev_get_drvdata(s->private);
+ int temp;
+
+ /* cpu */
+ temp = mt7615_mcu_get_temperature(dev, 0);
+ seq_printf(s, "Temperature: %d\n", temp);
+
+ return 0;
+}
+
+int mt7615_init_debugfs(struct mt7615_dev *dev)
+{
+ struct dentry *dir;
+
+ dir = mt76_register_debugfs(&dev->mt76);
+ if (!dir)
+ return -ENOMEM;
+
+ debugfs_create_file("scs", 0600, dir, dev, &fops_scs);
+ debugfs_create_devm_seqfile(dev->mt76.dev, "radio", dir,
+ mt7615_radio_read);
+ debugfs_create_u32("dfs_hw_pattern", 0400, dir, &dev->hw_pattern);
+ /* test pattern knobs */
+ debugfs_create_u8("pattern_len", 0600, dir,
+ &dev->radar_pattern.n_pulses);
+ debugfs_create_u32("pulse_period", 0600, dir,
+ &dev->radar_pattern.period);
+ debugfs_create_u16("pulse_width", 0600, dir,
+ &dev->radar_pattern.width);
+ debugfs_create_u16("pulse_power", 0600, dir,
+ &dev->radar_pattern.power);
+ debugfs_create_file("radar_trigger", 0200, dir, dev,
+ &fops_radar_pattern);
+ debugfs_create_devm_seqfile(dev->mt76.dev, "temperature", dir,
+ mt7615_read_temperature);
+
+ return 0;
+}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/dma.c b/drivers/net/wireless/mediatek/mt76/mt7615/dma.c
index 6a70273..fe532ce 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/dma.c
@@ -76,7 +76,7 @@ void mt7615_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
mt7615_mac_tx_free(dev, skb);
break;
case PKT_TYPE_RX_EVENT:
- mt76_mcu_rx_event(&dev->mt76, skb);
+ mt7615_mcu_rx_event(dev, skb);
break;
case PKT_TYPE_NORMAL:
if (!mt7615_mac_fill_rx(dev, skb)) {
@@ -90,13 +90,6 @@ void mt7615_queue_rx_skb(struct mt76_dev *mdev, enum mt76_rxq_id q,
}
}
-static void mt7615_tx_tasklet(unsigned long data)
-{
- struct mt7615_dev *dev = (struct mt7615_dev *)data;
-
- mt76_txq_schedule_all(&dev->mt76);
-}
-
static int mt7615_poll_tx(struct napi_struct *napi, int budget)
{
static const u8 queue_map[] = {
@@ -128,9 +121,6 @@ int mt7615_dma_init(struct mt7615_dev *dev)
mt76_dma_attach(&dev->mt76);
- tasklet_init(&dev->mt76.tx_tasklet, mt7615_tx_tasklet,
- (unsigned long)dev);
-
mt76_wr(dev, MT_WPDMA_GLO_CFG,
MT_WPDMA_GLO_CFG_TX_WRITEBACK_DONE |
MT_WPDMA_GLO_CFG_FIFO_LITTLE_ENDIAN |
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c
index dc94f52..515bb58 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c
@@ -154,6 +154,42 @@ int mt7615_eeprom_get_power_index(struct mt7615_dev *dev,
return index;
}
+static void mt7615_apply_cal_free_data(struct mt7615_dev *dev)
+{
+ static const u16 ical[] = {
+ 0x53, 0x54, 0x55, 0x56, 0x57, 0x5c, 0x5d, 0x62, 0x63, 0x68,
+ 0x69, 0x6e, 0x6f, 0x73, 0x74, 0x78, 0x79, 0x82, 0x83, 0x87,
+ 0x88, 0x8c, 0x8d, 0x91, 0x92, 0x96, 0x97, 0x9b, 0x9c, 0xa0,
+ 0xa1, 0xaa, 0xab, 0xaf, 0xb0, 0xb4, 0xb5, 0xb9, 0xba, 0xf4,
+ 0xf7, 0xff,
+ 0x140, 0x141, 0x145, 0x146, 0x14a, 0x14b, 0x154, 0x155, 0x159,
+ 0x15a, 0x15e, 0x15f, 0x163, 0x164, 0x168, 0x169, 0x16d, 0x16e,
+ 0x172, 0x173, 0x17c, 0x17d, 0x181, 0x182, 0x186, 0x187, 0x18b,
+ 0x18c
+ };
+ static const u16 ical_nocheck[] = {
+ 0x110, 0x111, 0x112, 0x113, 0x114, 0x115, 0x116, 0x117, 0x118,
+ 0x1b5, 0x1b6, 0x1b7, 0x3ac, 0x3ad, 0x3ae, 0x3af, 0x3b0, 0x3b1,
+ 0x3b2
+ };
+ u8 *eeprom = dev->mt76.eeprom.data;
+ u8 *otp = dev->mt76.otp.data;
+ int i;
+
+ if (!otp)
+ return;
+
+ for (i = 0; i < ARRAY_SIZE(ical); i++)
+ if (!otp[ical[i]])
+ return;
+
+ for (i = 0; i < ARRAY_SIZE(ical); i++)
+ eeprom[ical[i]] = otp[ical[i]];
+
+ for (i = 0; i < ARRAY_SIZE(ical_nocheck); i++)
+ eeprom[ical_nocheck[i]] = otp[ical_nocheck[i]];
+}
+
int mt7615_eeprom_init(struct mt7615_dev *dev)
{
int ret;
@@ -166,6 +202,8 @@ int mt7615_eeprom_init(struct mt7615_dev *dev)
if (ret && dev->mt76.otp.data)
memcpy(dev->mt76.eeprom.data, dev->mt76.otp.data,
MT7615_EEPROM_SIZE);
+ else
+ mt7615_apply_cal_free_data(dev);
mt7615_eeprom_parse_hw_cap(dev);
memcpy(dev->mt76.macaddr, dev->mt76.eeprom.data + MT_EE_MAC_ADDR,
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/init.c b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
index 859de24..1104e4c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
@@ -20,10 +20,24 @@ static void mt7615_phy_init(struct mt7615_dev *dev)
static void mt7615_mac_init(struct mt7615_dev *dev)
{
- /* enable band 0 clk */
- mt76_rmw(dev, MT_CFG_CCR,
- MT_CFG_CCR_MAC_D0_1X_GC_EN | MT_CFG_CCR_MAC_D0_2X_GC_EN,
- MT_CFG_CCR_MAC_D0_1X_GC_EN | MT_CFG_CCR_MAC_D0_2X_GC_EN);
+ u32 val;
+
+ /* enable band 0/1 clk */
+ mt76_set(dev, MT_CFG_CCR,
+ MT_CFG_CCR_MAC_D0_1X_GC_EN | MT_CFG_CCR_MAC_D0_2X_GC_EN |
+ MT_CFG_CCR_MAC_D1_1X_GC_EN | MT_CFG_CCR_MAC_D1_2X_GC_EN);
+
+ val = mt76_rmw(dev, MT_TMAC_TRCR0,
+ MT_TMAC_TRCR_CCA_SEL | MT_TMAC_TRCR_SEC_CCA_SEL,
+ FIELD_PREP(MT_TMAC_TRCR_CCA_SEL, 2) |
+ FIELD_PREP(MT_TMAC_TRCR_SEC_CCA_SEL, 0));
+ mt76_wr(dev, MT_TMAC_TRCR1, val);
+
+ val = MT_AGG_ACR_PKT_TIME_EN | MT_AGG_ACR_NO_BA_AR_RULE |
+ FIELD_PREP(MT_AGG_ACR_CFEND_RATE, 0x49) | /* 24M */
+ FIELD_PREP(MT_AGG_ACR_BAR_RATE, 0x4b); /* 6M */
+ mt76_wr(dev, MT_AGG_ACR0, val);
+ mt76_wr(dev, MT_AGG_ACR1, val);
mt76_rmw_field(dev, MT_TMAC_CTCR0,
MT_TMAC_CTCR0_INS_DDLMT_REFTIME, 0x3f);
@@ -36,6 +50,7 @@ static void mt7615_mac_init(struct mt7615_dev *dev)
MT_TMAC_CTCR0_INS_DDLMT_EN);
mt7615_mcu_set_rts_thresh(dev, 0x92b);
+ mt7615_mac_set_scs(dev, false);
mt76_rmw(dev, MT_AGG_SCR, MT_AGG_SCR_NLNAV_MID_PTEC_DIS,
MT_AGG_SCR_NLNAV_MID_PTEC_DIS);
@@ -45,11 +60,19 @@ static void mt7615_mac_init(struct mt7615_dev *dev)
mt76_wr(dev, MT_DMA_DCR0, MT_DMA_DCR0_RX_VEC_DROP |
FIELD_PREP(MT_DMA_DCR0_MAX_RX_LEN, 3072));
- mt76_wr(dev, MT_AGG_ARUCR, FIELD_PREP(MT_AGG_ARxCR_LIMIT(0), 7));
+ mt76_wr(dev, MT_AGG_ARUCR,
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(0), 7) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(1), 2) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(2), 2) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(3), 2) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(4), 1) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(5), 1) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(6), 1) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(7), 1));
+
mt76_wr(dev, MT_AGG_ARDCR,
- FIELD_PREP(MT_AGG_ARxCR_LIMIT(0), 0) |
- FIELD_PREP(MT_AGG_ARxCR_LIMIT(1),
- max_t(int, 0, MT7615_RATE_RETRY - 2)) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(0), MT7615_RATE_RETRY - 1) |
+ FIELD_PREP(MT_AGG_ARxCR_LIMIT(1), MT7615_RATE_RETRY - 1) |
FIELD_PREP(MT_AGG_ARxCR_LIMIT(2), MT7615_RATE_RETRY - 1) |
FIELD_PREP(MT_AGG_ARxCR_LIMIT(3), MT7615_RATE_RETRY - 1) |
FIELD_PREP(MT_AGG_ARxCR_LIMIT(4), MT7615_RATE_RETRY - 1) |
@@ -58,8 +81,7 @@ static void mt7615_mac_init(struct mt7615_dev *dev)
FIELD_PREP(MT_AGG_ARxCR_LIMIT(7), MT7615_RATE_RETRY - 1));
mt76_wr(dev, MT_AGG_ARCR,
- (MT_AGG_ARCR_INIT_RATE1 |
- FIELD_PREP(MT_AGG_ARCR_RTS_RATE_THR, 2) |
+ (FIELD_PREP(MT_AGG_ARCR_RTS_RATE_THR, 2) |
MT_AGG_ARCR_RATE_DOWN_RATIO_EN |
FIELD_PREP(MT_AGG_ARCR_RATE_DOWN_RATIO, 1) |
FIELD_PREP(MT_AGG_ARCR_RATE_UP_EXTRA_TH, 4)));
@@ -155,17 +177,6 @@ static const struct ieee80211_iface_combination if_comb[] = {
}
};
-static int mt7615_init_debugfs(struct mt7615_dev *dev)
-{
- struct dentry *dir;
-
- dir = mt76_register_debugfs(&dev->mt76);
- if (!dir)
- return -ENOMEM;
-
- return 0;
-}
-
static void
mt7615_init_txpower(struct mt7615_dev *dev,
struct ieee80211_supported_band *sband)
@@ -208,6 +219,30 @@ mt7615_init_txpower(struct mt7615_dev *dev,
}
}
+static void
+mt7615_regd_notifier(struct wiphy *wiphy,
+ struct regulatory_request *request)
+{
+ struct ieee80211_hw *hw = wiphy_to_ieee80211_hw(wiphy);
+ struct mt7615_dev *dev = hw->priv;
+ struct cfg80211_chan_def *chandef = &dev->mt76.chandef;
+
+ if (request->dfs_region == dev->mt76.region)
+ return;
+
+ dev->mt76.region = request->dfs_region;
+
+ if (!(chandef->chan->flags & IEEE80211_CHAN_RADAR))
+ return;
+
+ mt7615_dfs_stop_radar_detector(dev);
+ if (request->dfs_region == NL80211_DFS_UNSET)
+ mt7615_mcu_rdd_cmd(dev, RDD_CAC_END, MT_HW_RDD0,
+ MT_RX_SEL0, 0);
+ else
+ mt7615_dfs_start_radar_detector(dev);
+}
+
int mt7615_register_device(struct mt7615_dev *dev)
{
struct ieee80211_hw *hw = mt76_hw(dev);
@@ -230,6 +265,8 @@ int mt7615_register_device(struct mt7615_dev *dev)
wiphy->iface_combinations = if_comb;
wiphy->n_iface_combinations = ARRAY_SIZE(if_comb);
+ wiphy->reg_notifier = mt7615_regd_notifier;
+ wiphy->flags |= WIPHY_FLAG_HAS_CHANNEL_SWITCH;
ieee80211_hw_set(hw, SUPPORTS_REORDERING_BUFFER);
ieee80211_hw_set(hw, TX_STATUS_NO_AMPDU_LEN);
@@ -243,6 +280,7 @@ int mt7615_register_device(struct mt7615_dev *dev)
IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160_80PLUS80MHZ;
dev->mt76.chainmask = 0x404;
dev->mt76.antenna_mask = 0xf;
+ dev->dfs_state = -1;
wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION) |
#ifdef CONFIG_MAC80211_MESH
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
index 1eb0e9c..e07ce2c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.c
@@ -232,11 +232,9 @@ void mt7615_tx_complete_skb(struct mt76_dev *mdev, enum mt76_txq_id qid,
struct mt76_txwi_cache *t;
struct mt7615_dev *dev;
struct mt7615_txp *txp;
- u8 *txwi_ptr;
- txwi_ptr = mt76_get_txwi_ptr(mdev, e->txwi);
- txp = (struct mt7615_txp *)(txwi_ptr + MT_TXD_SIZE);
dev = container_of(mdev, struct mt7615_dev, mt76);
+ txp = mt7615_txwi_to_txp(mdev, e->txwi);
spin_lock_bh(&dev->token_lock);
t = idr_remove(&dev->token, le16_to_cpu(txp->token));
@@ -248,12 +246,13 @@ void mt7615_tx_complete_skb(struct mt76_dev *mdev, enum mt76_txq_id qid,
mt76_tx_complete_skb(mdev, e->skb);
}
-u16 mt7615_mac_tx_rate_val(struct mt7615_dev *dev,
- const struct ieee80211_tx_rate *rate,
- bool stbc, u8 *bw)
+static u16
+mt7615_mac_tx_rate_val(struct mt7615_dev *dev,
+ const struct ieee80211_tx_rate *rate,
+ bool stbc, u8 *bw)
{
u8 phy, nss, rate_idx;
- u16 rateval;
+ u16 rateval = 0;
*bw = 0;
@@ -291,12 +290,14 @@ u16 mt7615_mac_tx_rate_val(struct mt7615_dev *dev,
rate_idx = val & 0xff;
}
- rateval = (FIELD_PREP(MT_TX_RATE_IDX, rate_idx) |
- FIELD_PREP(MT_TX_RATE_MODE, phy) |
- FIELD_PREP(MT_TX_RATE_NSS, nss - 1));
-
- if (stbc && nss == 1)
+ if (stbc && nss == 1) {
+ nss++;
rateval |= MT_TX_RATE_STBC;
+ }
+
+ rateval |= (FIELD_PREP(MT_TX_RATE_IDX, rate_idx) |
+ FIELD_PREP(MT_TX_RATE_MODE, phy) |
+ FIELD_PREP(MT_TX_RATE_NSS, nss - 1));
return rateval;
}
@@ -309,9 +310,10 @@ int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
struct ieee80211_tx_rate *rate = &info->control.rates[0];
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ bool multicast = is_multicast_ether_addr(hdr->addr1);
struct ieee80211_vif *vif = info->control.vif;
int tx_count = 8;
- u8 fc_type, fc_stype, p_fmt, q_idx, omac_idx = 0;
+ u8 fc_type, fc_stype, p_fmt, q_idx, omac_idx = 0, wmm_idx = 0;
__le16 fc = hdr->frame_control;
u16 seqno = 0;
u32 val;
@@ -320,6 +322,7 @@ int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
struct mt7615_vif *mvif = (struct mt7615_vif *)vif->drv_priv;
omac_idx = mvif->omac_idx;
+ wmm_idx = mvif->wmm_idx;
}
if (sta) {
@@ -331,8 +334,9 @@ int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
fc_type = (le16_to_cpu(fc) & IEEE80211_FCTL_FTYPE) >> 2;
fc_stype = (le16_to_cpu(fc) & IEEE80211_FCTL_STYPE) >> 4;
- if (ieee80211_is_data(fc)) {
- q_idx = skb_get_queue_mapping(skb);
+ if (ieee80211_is_data(fc) || ieee80211_is_bufferable_mmpdu(fc)) {
+ q_idx = wmm_idx * MT7615_MAX_WMM_SETS +
+ skb_get_queue_mapping(skb);
p_fmt = MT_TX_TYPE_CT;
} else if (ieee80211_is_beacon(fc)) {
q_idx = MT_LMAC_BCN0;
@@ -360,8 +364,18 @@ int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
val = FIELD_PREP(MT_TXD2_FRAME_TYPE, fc_type) |
FIELD_PREP(MT_TXD2_SUB_TYPE, fc_stype) |
- FIELD_PREP(MT_TXD2_MULTICAST,
- is_multicast_ether_addr(hdr->addr1));
+ FIELD_PREP(MT_TXD2_MULTICAST, multicast);
+ if (key) {
+ if (multicast && ieee80211_is_robust_mgmt_frame(skb) &&
+ key->cipher == WLAN_CIPHER_SUITE_AES_CMAC) {
+ val |= MT_TXD2_BIP;
+ txwi[3] = 0;
+ } else {
+ txwi[3] = cpu_to_le32(MT_TXD3_PROTECT_FRAME);
+ }
+ } else {
+ txwi[3] = 0;
+ }
txwi[2] = cpu_to_le32(val);
if (!(info->flags & IEEE80211_TX_CTL_AMPDU))
@@ -418,14 +432,11 @@ int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
}
val |= FIELD_PREP(MT_TXD3_SEQ, seqno);
- txwi[3] = cpu_to_le32(val);
+ txwi[3] |= cpu_to_le32(val);
if (info->flags & IEEE80211_TX_CTL_NO_ACK)
txwi[3] |= cpu_to_le32(MT_TXD3_NO_ACK);
- if (key)
- txwi[3] |= cpu_to_le32(MT_TXD3_PROTECT_FRAME);
-
txwi[7] = FIELD_PREP(MT_TXD7_TYPE, fc_type) |
FIELD_PREP(MT_TXD7_SUB_TYPE, fc_stype);
@@ -436,16 +447,318 @@ void mt7615_txp_skb_unmap(struct mt76_dev *dev,
struct mt76_txwi_cache *t)
{
struct mt7615_txp *txp;
- u8 *txwi;
int i;
- txwi = mt76_get_txwi_ptr(dev, t);
- txp = (struct mt7615_txp *)(txwi + MT_TXD_SIZE);
+ txp = mt7615_txwi_to_txp(dev, t);
for (i = 1; i < txp->nbuf; i++)
dma_unmap_single(dev->dev, le32_to_cpu(txp->buf[i]),
le16_to_cpu(txp->len[i]), DMA_TO_DEVICE);
}
+static u32 mt7615_mac_wtbl_addr(int wcid)
+{
+ return MT_WTBL_BASE + wcid * MT_WTBL_ENTRY_SIZE;
+}
+
+void mt7615_mac_set_rates(struct mt7615_dev *dev, struct mt7615_sta *sta,
+ struct ieee80211_tx_rate *probe_rate,
+ struct ieee80211_tx_rate *rates)
+{
+ struct ieee80211_tx_rate *ref;
+ int wcid = sta->wcid.idx;
+ u32 addr = mt7615_mac_wtbl_addr(wcid);
+ bool stbc = false;
+ int n_rates = sta->n_rates;
+ u8 bw, bw_prev, bw_idx = 0;
+ u16 val[4];
+ u16 probe_val;
+ u32 w5, w27;
+ bool rateset;
+ int i, k;
+
+ if (!mt76_poll(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_BUSY, 0, 5000))
+ return;
+
+ for (i = n_rates; i < 4; i++)
+ rates[i] = rates[n_rates - 1];
+
+ rateset = !(sta->rate_set_tsf & BIT(0));
+ memcpy(sta->rateset[rateset].rates, rates,
+ sizeof(sta->rateset[rateset].rates));
+ if (probe_rate) {
+ sta->rateset[rateset].probe_rate = *probe_rate;
+ ref = &sta->rateset[rateset].probe_rate;
+ } else {
+ sta->rateset[rateset].probe_rate.idx = -1;
+ ref = &sta->rateset[rateset].rates[0];
+ }
+
+ rates = sta->rateset[rateset].rates;
+ for (i = 0; i < ARRAY_SIZE(sta->rateset[rateset].rates); i++) {
+ /*
+ * We don't support switching between short and long GI
+ * within the rate set. For accurate tx status reporting, we
+ * need to make sure that flags match.
+ * For improved performance, avoid duplicate entries by
+ * decrementing the MCS index if necessary
+ */
+ if ((ref->flags ^ rates[i].flags) & IEEE80211_TX_RC_SHORT_GI)
+ rates[i].flags ^= IEEE80211_TX_RC_SHORT_GI;
+
+ for (k = 0; k < i; k++) {
+ if (rates[i].idx != rates[k].idx)
+ continue;
+ if ((rates[i].flags ^ rates[k].flags) &
+ (IEEE80211_TX_RC_40_MHZ_WIDTH |
+ IEEE80211_TX_RC_80_MHZ_WIDTH |
+ IEEE80211_TX_RC_160_MHZ_WIDTH))
+ continue;
+
+ if (!rates[i].idx)
+ continue;
+
+ rates[i].idx--;
+ }
+ }
+
+ val[0] = mt7615_mac_tx_rate_val(dev, &rates[0], stbc, &bw);
+ bw_prev = bw;
+
+ if (probe_rate) {
+ probe_val = mt7615_mac_tx_rate_val(dev, probe_rate, stbc, &bw);
+ if (bw)
+ bw_idx = 1;
+ else
+ bw_prev = 0;
+ } else {
+ probe_val = val[0];
+ }
+
+ val[1] = mt7615_mac_tx_rate_val(dev, &rates[1], stbc, &bw);
+ if (bw_prev) {
+ bw_idx = 3;
+ bw_prev = bw;
+ }
+
+ val[2] = mt7615_mac_tx_rate_val(dev, &rates[2], stbc, &bw);
+ if (bw_prev) {
+ bw_idx = 5;
+ bw_prev = bw;
+ }
+
+ val[3] = mt7615_mac_tx_rate_val(dev, &rates[3], stbc, &bw);
+ if (bw_prev)
+ bw_idx = 7;
+
+ w27 = mt76_rr(dev, addr + 27 * 4);
+ w27 &= ~MT_WTBL_W27_CC_BW_SEL;
+ w27 |= FIELD_PREP(MT_WTBL_W27_CC_BW_SEL, bw);
+
+ w5 = mt76_rr(dev, addr + 5 * 4);
+ w5 &= ~(MT_WTBL_W5_BW_CAP | MT_WTBL_W5_CHANGE_BW_RATE |
+ MT_WTBL_W5_MPDU_OK_COUNT |
+ MT_WTBL_W5_MPDU_FAIL_COUNT |
+ MT_WTBL_W5_RATE_IDX);
+ w5 |= FIELD_PREP(MT_WTBL_W5_BW_CAP, bw) |
+ FIELD_PREP(MT_WTBL_W5_CHANGE_BW_RATE, bw_idx ? bw_idx - 1 : 7);
+
+ mt76_wr(dev, MT_WTBL_RIUCR0, w5);
+
+ mt76_wr(dev, MT_WTBL_RIUCR1,
+ FIELD_PREP(MT_WTBL_RIUCR1_RATE0, probe_val) |
+ FIELD_PREP(MT_WTBL_RIUCR1_RATE1, val[0]) |
+ FIELD_PREP(MT_WTBL_RIUCR1_RATE2_LO, val[1]));
+
+ mt76_wr(dev, MT_WTBL_RIUCR2,
+ FIELD_PREP(MT_WTBL_RIUCR2_RATE2_HI, val[1] >> 8) |
+ FIELD_PREP(MT_WTBL_RIUCR2_RATE3, val[1]) |
+ FIELD_PREP(MT_WTBL_RIUCR2_RATE4, val[2]) |
+ FIELD_PREP(MT_WTBL_RIUCR2_RATE5_LO, val[2]));
+
+ mt76_wr(dev, MT_WTBL_RIUCR3,
+ FIELD_PREP(MT_WTBL_RIUCR3_RATE5_HI, val[2] >> 4) |
+ FIELD_PREP(MT_WTBL_RIUCR3_RATE6, val[3]) |
+ FIELD_PREP(MT_WTBL_RIUCR3_RATE7, val[3]));
+
+ mt76_wr(dev, MT_WTBL_UPDATE,
+ FIELD_PREP(MT_WTBL_UPDATE_WLAN_IDX, wcid) |
+ MT_WTBL_UPDATE_RATE_UPDATE |
+ MT_WTBL_UPDATE_TX_COUNT_CLEAR);
+
+ mt76_wr(dev, addr + 27 * 4, w27);
+
+ mt76_set(dev, MT_LPON_T0CR, MT_LPON_T0CR_MODE); /* TSF read */
+ sta->rate_set_tsf = (mt76_rr(dev, MT_LPON_UTTR0) & ~BIT(0)) | rateset;
+
+ if (!(sta->wcid.tx_info & MT_WCID_TX_INFO_SET))
+ mt76_poll(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_BUSY, 0, 5000);
+
+ sta->rate_count = 2 * MT7615_RATE_RETRY * n_rates;
+ sta->wcid.tx_info |= MT_WCID_TX_INFO_SET;
+}
+
+static enum mt7615_cipher_type
+mt7615_mac_get_cipher(int cipher)
+{
+ switch (cipher) {
+ case WLAN_CIPHER_SUITE_WEP40:
+ return MT_CIPHER_WEP40;
+ case WLAN_CIPHER_SUITE_WEP104:
+ return MT_CIPHER_WEP104;
+ case WLAN_CIPHER_SUITE_TKIP:
+ return MT_CIPHER_TKIP;
+ case WLAN_CIPHER_SUITE_AES_CMAC:
+ return MT_CIPHER_BIP_CMAC_128;
+ case WLAN_CIPHER_SUITE_CCMP:
+ return MT_CIPHER_AES_CCMP;
+ case WLAN_CIPHER_SUITE_CCMP_256:
+ return MT_CIPHER_CCMP_256;
+ case WLAN_CIPHER_SUITE_GCMP:
+ return MT_CIPHER_GCMP;
+ case WLAN_CIPHER_SUITE_GCMP_256:
+ return MT_CIPHER_GCMP_256;
+ case WLAN_CIPHER_SUITE_SMS4:
+ return MT_CIPHER_WAPI;
+ default:
+ return MT_CIPHER_NONE;
+ }
+}
+
+static int
+mt7615_mac_wtbl_update_key(struct mt7615_dev *dev, struct mt76_wcid *wcid,
+ struct ieee80211_key_conf *key,
+ enum mt7615_cipher_type cipher,
+ enum set_key_cmd cmd)
+{
+ u32 addr = mt7615_mac_wtbl_addr(wcid->idx) + 30 * 4;
+ u8 data[32] = {};
+
+ if (key->keylen > sizeof(data))
+ return -EINVAL;
+
+ mt76_rr_copy(dev, addr, data, sizeof(data));
+ if (cmd == SET_KEY) {
+ if (cipher == MT_CIPHER_TKIP) {
+ /* Rx/Tx MIC keys are swapped */
+ memcpy(data + 16, key->key + 24, 8);
+ memcpy(data + 24, key->key + 16, 8);
+ }
+ if (cipher != MT_CIPHER_BIP_CMAC_128 && wcid->cipher)
+ memmove(data + 16, data, 16);
+ if (cipher != MT_CIPHER_BIP_CMAC_128 || !wcid->cipher)
+ memcpy(data, key->key, key->keylen);
+ else if (cipher == MT_CIPHER_BIP_CMAC_128)
+ memcpy(data + 16, key->key, 16);
+ } else {
+ if (wcid->cipher & ~BIT(cipher)) {
+ if (cipher != MT_CIPHER_BIP_CMAC_128)
+ memmove(data, data + 16, 16);
+ memset(data + 16, 0, 16);
+ } else {
+ memset(data, 0, sizeof(data));
+ }
+ }
+ mt76_wr_copy(dev, addr, data, sizeof(data));
+
+ return 0;
+}
+
+static int
+mt7615_mac_wtbl_update_pk(struct mt7615_dev *dev, struct mt76_wcid *wcid,
+ enum mt7615_cipher_type cipher, int keyidx,
+ enum set_key_cmd cmd)
+{
+ u32 addr = mt7615_mac_wtbl_addr(wcid->idx), w0, w1;
+
+ if (!mt76_poll(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_BUSY, 0, 5000))
+ return -ETIMEDOUT;
+
+ w0 = mt76_rr(dev, addr);
+ w1 = mt76_rr(dev, addr + 4);
+ if (cmd == SET_KEY) {
+ w0 |= MT_WTBL_W0_RX_KEY_VALID |
+ FIELD_PREP(MT_WTBL_W0_RX_IK_VALID,
+ cipher == MT_CIPHER_BIP_CMAC_128);
+ if (cipher != MT_CIPHER_BIP_CMAC_128 ||
+ !wcid->cipher)
+ w0 |= FIELD_PREP(MT_WTBL_W0_KEY_IDX, keyidx);
+ } else {
+ if (!(wcid->cipher & ~BIT(cipher)))
+ w0 &= ~(MT_WTBL_W0_RX_KEY_VALID |
+ MT_WTBL_W0_KEY_IDX);
+ if (cipher == MT_CIPHER_BIP_CMAC_128)
+ w0 &= ~MT_WTBL_W0_RX_IK_VALID;
+ }
+ mt76_wr(dev, MT_WTBL_RICR0, w0);
+ mt76_wr(dev, MT_WTBL_RICR1, w1);
+
+ mt76_wr(dev, MT_WTBL_UPDATE,
+ FIELD_PREP(MT_WTBL_UPDATE_WLAN_IDX, wcid->idx) |
+ MT_WTBL_UPDATE_RXINFO_UPDATE);
+
+ if (!mt76_poll(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_BUSY, 0, 5000))
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+static void
+mt7615_mac_wtbl_update_cipher(struct mt7615_dev *dev, struct mt76_wcid *wcid,
+ enum mt7615_cipher_type cipher,
+ enum set_key_cmd cmd)
+{
+ u32 addr = mt7615_mac_wtbl_addr(wcid->idx);
+
+ if (cmd == SET_KEY) {
+ if (cipher != MT_CIPHER_BIP_CMAC_128 || !wcid->cipher)
+ mt76_rmw(dev, addr + 2 * 4, MT_WTBL_W2_KEY_TYPE,
+ FIELD_PREP(MT_WTBL_W2_KEY_TYPE, cipher));
+ } else {
+ if (cipher != MT_CIPHER_BIP_CMAC_128 &&
+ wcid->cipher & BIT(MT_CIPHER_BIP_CMAC_128))
+ mt76_rmw(dev, addr + 2 * 4, MT_WTBL_W2_KEY_TYPE,
+ FIELD_PREP(MT_WTBL_W2_KEY_TYPE,
+ MT_CIPHER_BIP_CMAC_128));
+ else if (!(wcid->cipher & ~BIT(cipher)))
+ mt76_clear(dev, addr + 2 * 4, MT_WTBL_W2_KEY_TYPE);
+ }
+}
+
+int mt7615_mac_wtbl_set_key(struct mt7615_dev *dev,
+ struct mt76_wcid *wcid,
+ struct ieee80211_key_conf *key,
+ enum set_key_cmd cmd)
+{
+ enum mt7615_cipher_type cipher;
+ int err;
+
+ cipher = mt7615_mac_get_cipher(key->cipher);
+ if (cipher == MT_CIPHER_NONE)
+ return -EOPNOTSUPP;
+
+ spin_lock_bh(&dev->mt76.lock);
+
+ mt7615_mac_wtbl_update_cipher(dev, wcid, cipher, cmd);
+ err = mt7615_mac_wtbl_update_key(dev, wcid, key, cipher, cmd);
+ if (err < 0)
+ goto out;
+
+ err = mt7615_mac_wtbl_update_pk(dev, wcid, cipher, key->keyidx,
+ cmd);
+ if (err < 0)
+ goto out;
+
+ if (cmd == SET_KEY)
+ wcid->cipher |= BIT(cipher);
+ else
+ wcid->cipher &= ~BIT(cipher);
+
+out:
+ spin_unlock_bh(&dev->mt76.lock);
+
+ return err;
+}
+
int mt7615_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
enum mt76_txq_id qid, struct mt76_wcid *wcid,
struct ieee80211_sta *sta,
@@ -469,9 +782,9 @@ int mt7615_tx_prepare_skb(struct mt76_dev *mdev, void *txwi_ptr,
if (info->flags & IEEE80211_TX_CTL_RATE_CTRL_PROBE) {
spin_lock_bh(&dev->mt76.lock);
- msta->rate_probe = true;
- mt7615_mcu_set_rates(dev, msta, &info->control.rates[0],
+ mt7615_mac_set_rates(dev, msta, &info->control.rates[0],
msta->rates);
+ msta->rate_probe = true;
spin_unlock_bh(&dev->mt76.lock);
}
@@ -523,9 +836,13 @@ static bool mt7615_fill_txs(struct mt7615_dev *dev, struct mt7615_sta *sta,
struct ieee80211_tx_info *info, __le32 *txs_data)
{
struct ieee80211_supported_band *sband;
- int i, idx, count, final_idx = 0;
+ struct mt7615_rate_set *rs;
+ int first_idx = 0, last_idx;
+ int i, idx, count;
bool fixed_rate, ack_timeout;
bool probe, ampdu, cck = false;
+ bool rs_idx;
+ u32 rate_set_tsf;
u32 final_rate, final_rate_flags, final_nss, txs;
fixed_rate = info->status.rates[0].count;
@@ -536,6 +853,7 @@ static bool mt7615_fill_txs(struct mt7615_dev *dev, struct mt7615_sta *sta,
txs = le32_to_cpu(txs_data[3]);
count = FIELD_GET(MT_TXS3_TX_COUNT, txs);
+ last_idx = FIELD_GET(MT_TXS3_LAST_TX_RATE, txs);
txs = le32_to_cpu(txs_data[0]);
final_rate = FIELD_GET(MT_TXS0_TX_RATE, txs);
@@ -557,38 +875,57 @@ static bool mt7615_fill_txs(struct mt7615_dev *dev, struct mt7615_sta *sta,
if (ampdu || (info->flags & IEEE80211_TX_CTL_AMPDU))
info->flags |= IEEE80211_TX_STAT_AMPDU | IEEE80211_TX_CTL_AMPDU;
+ first_idx = max_t(int, 0, last_idx - (count + 1) / MT7615_RATE_RETRY);
+
if (fixed_rate && !probe) {
info->status.rates[0].count = count;
+ i = 0;
goto out;
}
- for (i = 0, idx = 0; i < ARRAY_SIZE(info->status.rates); i++) {
- int cur_count = min_t(int, count, 2 * MT7615_RATE_RETRY);
+ rate_set_tsf = READ_ONCE(sta->rate_set_tsf);
+ rs_idx = !((u32)(FIELD_GET(MT_TXS4_F0_TIMESTAMP, le32_to_cpu(txs_data[4])) -
+ rate_set_tsf) < 1000000);
+ rs_idx ^= rate_set_tsf & BIT(0);
+ rs = &sta->rateset[rs_idx];
- if (!i && probe) {
- cur_count = 1;
- } else {
- info->status.rates[i] = sta->rates[idx];
- idx++;
+ if (!first_idx && rs->probe_rate.idx >= 0) {
+ info->status.rates[0] = rs->probe_rate;
+
+ spin_lock_bh(&dev->mt76.lock);
+ if (sta->rate_probe) {
+ mt7615_mac_set_rates(dev, sta, NULL, sta->rates);
+ sta->rate_probe = false;
}
+ spin_unlock_bh(&dev->mt76.lock);
+ } else {
+ info->status.rates[0] = rs->rates[first_idx / 2];
+ }
+ info->status.rates[0].count = 0;
- if (i && info->status.rates[i].idx < 0) {
- info->status.rates[i - 1].count += count;
- break;
- }
+ for (i = 0, idx = first_idx; count && idx <= last_idx; idx++) {
+ struct ieee80211_tx_rate *cur_rate;
+ int cur_count;
- if (!count) {
- info->status.rates[i].idx = -1;
- break;
- }
-
- info->status.rates[i].count = cur_count;
- final_idx = i;
+ cur_rate = &rs->rates[idx / 2];
+ cur_count = min_t(int, MT7615_RATE_RETRY, count);
count -= cur_count;
+
+ if (idx && (cur_rate->idx != info->status.rates[i].idx ||
+ cur_rate->flags != info->status.rates[i].flags)) {
+ i++;
+ if (i == ARRAY_SIZE(info->status.rates))
+ break;
+
+ info->status.rates[i] = *cur_rate;
+ info->status.rates[i].count = 0;
+ }
+
+ info->status.rates[i].count += cur_count;
}
out:
- final_rate_flags = info->status.rates[final_idx].flags;
+ final_rate_flags = info->status.rates[i].flags;
switch (FIELD_GET(MT_TX_RATE_MODE, final_rate)) {
case MT_PHY_TYPE_CCK:
@@ -613,6 +950,10 @@ static bool mt7615_fill_txs(struct mt7615_dev *dev, struct mt7615_sta *sta,
break;
case MT_PHY_TYPE_VHT:
final_nss = FIELD_GET(MT_TX_RATE_NSS, final_rate);
+
+ if ((final_rate & MT_TX_RATE_STBC) && final_nss)
+ final_nss--;
+
final_rate_flags |= IEEE80211_TX_RC_VHT_MCS;
final_rate = (final_rate & MT_TX_RATE_IDX) | (final_nss << 4);
break;
@@ -620,8 +961,8 @@ static bool mt7615_fill_txs(struct mt7615_dev *dev, struct mt7615_sta *sta,
return false;
}
- info->status.rates[final_idx].idx = final_rate;
- info->status.rates[final_idx].flags = final_rate_flags;
+ info->status.rates[i].idx = final_rate;
+ info->status.rates[i].flags = final_rate_flags;
return true;
}
@@ -642,16 +983,6 @@ static bool mt7615_mac_add_txs_skb(struct mt7615_dev *dev,
if (skb) {
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
- if (info->flags & IEEE80211_TX_CTL_RATE_CTRL_PROBE) {
- spin_lock_bh(&dev->mt76.lock);
- if (sta->rate_probe) {
- mt7615_mcu_set_rates(dev, sta, NULL,
- sta->rates);
- sta->rate_probe = false;
- }
- spin_unlock_bh(&dev->mt76.lock);
- }
-
if (!mt7615_fill_txs(dev, sta, info, txs_data)) {
ieee80211_tx_info_clear_status(info);
info->status.rates[0].idx = -1;
@@ -735,6 +1066,198 @@ void mt7615_mac_tx_free(struct mt7615_dev *dev, struct sk_buff *skb)
dev_kfree_skb(skb);
}
+static void
+mt7615_mac_set_default_sensitivity(struct mt7615_dev *dev)
+{
+ mt76_rmw(dev, MT_WF_PHY_B0_MIN_PRI_PWR,
+ MT_WF_PHY_B0_PD_OFDM_MASK,
+ MT_WF_PHY_B0_PD_OFDM(0x13c));
+ mt76_rmw(dev, MT_WF_PHY_B1_MIN_PRI_PWR,
+ MT_WF_PHY_B1_PD_OFDM_MASK,
+ MT_WF_PHY_B1_PD_OFDM(0x13c));
+
+ mt76_rmw(dev, MT_WF_PHY_B0_RXTD_CCK_PD,
+ MT_WF_PHY_B0_PD_CCK_MASK,
+ MT_WF_PHY_B0_PD_CCK(0x92));
+ mt76_rmw(dev, MT_WF_PHY_B1_RXTD_CCK_PD,
+ MT_WF_PHY_B1_PD_CCK_MASK,
+ MT_WF_PHY_B1_PD_CCK(0x92));
+
+ dev->ofdm_sensitivity = -98;
+ dev->cck_sensitivity = -110;
+ dev->last_cca_adj = jiffies;
+}
+
+void mt7615_mac_set_scs(struct mt7615_dev *dev, bool enable)
+{
+ mutex_lock(&dev->mt76.mutex);
+
+ if (dev->scs_en == enable)
+ goto out;
+
+ if (enable) {
+ /* DBDC not supported */
+ mt76_set(dev, MT_WF_PHY_B0_MIN_PRI_PWR,
+ MT_WF_PHY_B0_PD_BLK);
+ if (is_mt7622(&dev->mt76)) {
+ mt76_set(dev, MT_MIB_M0_MISC_CR, 0x7 << 8);
+ mt76_set(dev, MT_MIB_M0_MISC_CR, 0x7);
+ }
+ } else {
+ mt76_clear(dev, MT_WF_PHY_B0_MIN_PRI_PWR,
+ MT_WF_PHY_B0_PD_BLK);
+ mt76_clear(dev, MT_WF_PHY_B1_MIN_PRI_PWR,
+ MT_WF_PHY_B1_PD_BLK);
+ }
+
+ mt7615_mac_set_default_sensitivity(dev);
+ dev->scs_en = enable;
+
+out:
+ mutex_unlock(&dev->mt76.mutex);
+}
+
+void mt7615_mac_cca_stats_reset(struct mt7615_dev *dev)
+{
+ mt76_clear(dev, MT_WF_PHY_R0_B0_PHYMUX_5, GENMASK(22, 20));
+ mt76_set(dev, MT_WF_PHY_R0_B0_PHYMUX_5, BIT(22) | BIT(20));
+}
+
+static void
+mt7615_mac_adjust_sensitivity(struct mt7615_dev *dev,
+ u32 rts_err_rate, bool ofdm)
+{
+ int false_cca = ofdm ? dev->false_cca_ofdm : dev->false_cca_cck;
+ u16 def_th = ofdm ? -98 : -110;
+ bool update = false;
+ s8 *sensitivity;
+ int signal;
+
+ sensitivity = ofdm ? &dev->ofdm_sensitivity : &dev->cck_sensitivity;
+ signal = mt76_get_min_avg_rssi(&dev->mt76);
+ if (!signal) {
+ mt7615_mac_set_default_sensitivity(dev);
+ return;
+ }
+
+ signal = min(signal, -72);
+ if (false_cca > 500) {
+ if (rts_err_rate > MT_FRAC(40, 100))
+ return;
+
+ /* decrease coverage */
+ if (*sensitivity == def_th && signal > -90) {
+ *sensitivity = -90;
+ update = true;
+ } else if (*sensitivity + 2 < signal) {
+ *sensitivity += 2;
+ update = true;
+ }
+ } else if ((false_cca > 0 && false_cca < 50) ||
+ rts_err_rate > MT_FRAC(60, 100)) {
+ /* increase coverage */
+ if (*sensitivity - 2 >= def_th) {
+ *sensitivity -= 2;
+ update = true;
+ }
+ }
+
+ if (*sensitivity > signal) {
+ *sensitivity = signal;
+ update = true;
+ }
+
+ if (update) {
+ u16 val;
+
+ if (ofdm) {
+ /* DBDC not supported */
+ val = *sensitivity * 2 + 512;
+ mt76_rmw(dev, MT_WF_PHY_B0_MIN_PRI_PWR,
+ MT_WF_PHY_B0_PD_OFDM_MASK,
+ MT_WF_PHY_B0_PD_OFDM(val));
+ } else {
+ val = *sensitivity + 256;
+ mt76_rmw(dev, MT_WF_PHY_B0_RXTD_CCK_PD,
+ MT_WF_PHY_B0_PD_CCK_MASK,
+ MT_WF_PHY_B0_PD_CCK(val));
+ mt76_rmw(dev, MT_WF_PHY_B1_RXTD_CCK_PD,
+ MT_WF_PHY_B1_PD_CCK_MASK,
+ MT_WF_PHY_B1_PD_CCK(val));
+ }
+ dev->last_cca_adj = jiffies;
+ }
+}
+
+static void
+mt7615_mac_scs_check(struct mt7615_dev *dev)
+{
+ u32 val, rts_cnt = 0, rts_retries_cnt = 0, rts_err_rate = 0;
+ u32 mdrdy_cck, mdrdy_ofdm, pd_cck, pd_ofdm;
+ int i;
+
+ if (!dev->scs_en)
+ return;
+
+ for (i = 0; i < 4; i++) {
+ u32 data;
+
+ val = mt76_rr(dev, MT_MIB_MB_SDR0(i));
+ data = FIELD_GET(MT_MIB_RTS_RETRIES_COUNT_MASK, val);
+ if (data > rts_retries_cnt) {
+ rts_cnt = FIELD_GET(MT_MIB_RTS_COUNT_MASK, val);
+ rts_retries_cnt = data;
+ }
+ }
+
+ val = mt76_rr(dev, MT_WF_PHY_R0_B0_PHYCTRL_STS0);
+ pd_cck = FIELD_GET(MT_WF_PHYCTRL_STAT_PD_CCK, val);
+ pd_ofdm = FIELD_GET(MT_WF_PHYCTRL_STAT_PD_OFDM, val);
+
+ val = mt76_rr(dev, MT_WF_PHY_R0_B0_PHYCTRL_STS5);
+ mdrdy_cck = FIELD_GET(MT_WF_PHYCTRL_STAT_MDRDY_CCK, val);
+ mdrdy_ofdm = FIELD_GET(MT_WF_PHYCTRL_STAT_MDRDY_OFDM, val);
+
+ dev->false_cca_ofdm = pd_ofdm - mdrdy_ofdm;
+ dev->false_cca_cck = pd_cck - mdrdy_cck;
+ mt7615_mac_cca_stats_reset(dev);
+
+ if (rts_cnt + rts_retries_cnt)
+ rts_err_rate = MT_FRAC(rts_retries_cnt,
+ rts_cnt + rts_retries_cnt);
+
+ /* cck */
+ mt7615_mac_adjust_sensitivity(dev, rts_err_rate, false);
+ /* ofdm */
+ mt7615_mac_adjust_sensitivity(dev, rts_err_rate, true);
+
+ if (time_after(jiffies, dev->last_cca_adj + 10 * HZ))
+ mt7615_mac_set_default_sensitivity(dev);
+}
+
+void mt7615_update_channel(struct mt76_dev *mdev)
+{
+ struct mt7615_dev *dev = container_of(mdev, struct mt7615_dev, mt76);
+ struct mt76_channel_state *state;
+ ktime_t cur_time;
+ u32 busy;
+
+ if (!test_bit(MT76_STATE_RUNNING, &mdev->state))
+ return;
+
+ state = mt76_channel_state(mdev, mdev->chandef.chan);
+ /* TODO: add DBDC support */
+ busy = mt76_get_field(dev, MT_MIB_SDR16(0), MT_MIB_BUSY_MASK);
+
+ spin_lock_bh(&mdev->cc_lock);
+ cur_time = ktime_get_boottime();
+ state->cc_busy += busy;
+ state->cc_active += ktime_to_us(ktime_sub(cur_time,
+ mdev->survey_time));
+ mdev->survey_time = cur_time;
+ spin_unlock_bh(&mdev->cc_lock);
+}
+
void mt7615_mac_work(struct work_struct *work)
{
struct mt7615_dev *dev;
@@ -742,7 +1265,103 @@ void mt7615_mac_work(struct work_struct *work)
dev = (struct mt7615_dev *)container_of(work, struct mt76_dev,
mac_work.work);
+ mutex_lock(&dev->mt76.mutex);
+ mt7615_update_channel(&dev->mt76);
+ if (++dev->mac_work_count == 5) {
+ mt7615_mac_scs_check(dev);
+ dev->mac_work_count = 0;
+ }
+ mutex_unlock(&dev->mt76.mutex);
+
mt76_tx_status_check(&dev->mt76, NULL, false);
ieee80211_queue_delayed_work(mt76_hw(dev), &dev->mt76.mac_work,
MT7615_WATCHDOG_TIME);
}
+
+int mt7615_dfs_stop_radar_detector(struct mt7615_dev *dev)
+{
+ struct cfg80211_chan_def *chandef = &dev->mt76.chandef;
+ int err;
+
+ err = mt7615_mcu_rdd_cmd(dev, RDD_STOP, MT_HW_RDD0,
+ MT_RX_SEL0, 0);
+ if (err < 0)
+ return err;
+
+ if (chandef->width == NL80211_CHAN_WIDTH_160 ||
+ chandef->width == NL80211_CHAN_WIDTH_80P80)
+ err = mt7615_mcu_rdd_cmd(dev, RDD_STOP, MT_HW_RDD1,
+ MT_RX_SEL0, 0);
+ return err;
+}
+
+static int mt7615_dfs_start_rdd(struct mt7615_dev *dev, int chain)
+{
+ int err;
+
+ err = mt7615_mcu_rdd_cmd(dev, RDD_START, chain, MT_RX_SEL0, 0);
+ if (err < 0)
+ return err;
+
+ return mt7615_mcu_rdd_cmd(dev, RDD_DET_MODE, chain,
+ MT_RX_SEL0, 1);
+}
+
+int mt7615_dfs_start_radar_detector(struct mt7615_dev *dev)
+{
+ struct cfg80211_chan_def *chandef = &dev->mt76.chandef;
+ int err;
+
+ /* start CAC */
+ err = mt7615_mcu_rdd_cmd(dev, RDD_CAC_START, MT_HW_RDD0,
+ MT_RX_SEL0, 0);
+ if (err < 0)
+ return err;
+
+ /* TODO: DBDC support */
+
+ err = mt7615_dfs_start_rdd(dev, MT_HW_RDD0);
+ if (err < 0)
+ return err;
+
+ if (chandef->width == NL80211_CHAN_WIDTH_160 ||
+ chandef->width == NL80211_CHAN_WIDTH_80P80) {
+ err = mt7615_dfs_start_rdd(dev, MT_HW_RDD1);
+ if (err < 0)
+ return err;
+ }
+
+ return 0;
+}
+
+int mt7615_dfs_init_radar_detector(struct mt7615_dev *dev)
+{
+ struct cfg80211_chan_def *chandef = &dev->mt76.chandef;
+ int err;
+
+ if (dev->mt76.region == NL80211_DFS_UNSET)
+ return 0;
+
+ if (test_bit(MT76_SCANNING, &dev->mt76.state))
+ return 0;
+
+ if (dev->dfs_state == chandef->chan->dfs_state)
+ return 0;
+
+ dev->dfs_state = chandef->chan->dfs_state;
+
+ if (chandef->chan->flags & IEEE80211_CHAN_RADAR) {
+ if (chandef->chan->dfs_state != NL80211_DFS_AVAILABLE)
+ return mt7615_dfs_start_radar_detector(dev);
+ else
+ return mt7615_mcu_rdd_cmd(dev, RDD_CAC_END, MT_HW_RDD0,
+ MT_RX_SEL0, 0);
+ } else {
+ err = mt7615_mcu_rdd_cmd(dev, RDD_NORMAL_START,
+ MT_HW_RDD0, MT_RX_SEL0, 0);
+ if (err < 0)
+ return err;
+
+ return mt7615_dfs_stop_radar_detector(dev);
+ }
+}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mac.h b/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
index b00ce8d..38695d4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mac.h
@@ -302,4 +302,32 @@ struct mt7615_tx_free {
#define MT_TXS6_F1_RCPI_1 GENMASK(15, 8)
#define MT_TXS6_F1_RCPI_0 GENMASK(7, 0)
+enum mt7615_cipher_type {
+ MT_CIPHER_NONE,
+ MT_CIPHER_WEP40,
+ MT_CIPHER_TKIP,
+ MT_CIPHER_TKIP_NO_MIC,
+ MT_CIPHER_AES_CCMP,
+ MT_CIPHER_WEP104,
+ MT_CIPHER_BIP_CMAC_128,
+ MT_CIPHER_WEP128,
+ MT_CIPHER_WAPI,
+ MT_CIPHER_CCMP_256 = 10,
+ MT_CIPHER_GCMP,
+ MT_CIPHER_GCMP_256,
+};
+
+static inline struct mt7615_txp *
+mt7615_txwi_to_txp(struct mt76_dev *dev, struct mt76_txwi_cache *t)
+{
+ u8 *txwi;
+
+ if (!t)
+ return NULL;
+
+ txwi = mt76_get_txwi_ptr(dev, t);
+
+ return (struct mt7615_txp *)(txwi + MT_TXD_SIZE);
+}
+
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
index b4d6af81..87c7487 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
@@ -16,6 +16,7 @@ static int mt7615_start(struct ieee80211_hw *hw)
{
struct mt7615_dev *dev = hw->priv;
+ dev->mt76.survey_time = ktime_get_boottime();
set_bit(MT76_STATE_RUNNING, &dev->mt76.state);
ieee80211_queue_delayed_work(mt76_hw(dev), &dev->mt76.mac_work,
MT7615_WATCHDOG_TIME);
@@ -85,9 +86,9 @@ static int mt7615_add_interface(struct ieee80211_hw *hw,
}
mvif->omac_idx = idx;
- /* TODO: DBDC support. Use band 0 and wmm 0 for now */
+ /* TODO: DBDC support. Use band 0 for now */
mvif->band_idx = 0;
- mvif->wmm_idx = 0;
+ mvif->wmm_idx = mvif->idx % MT7615_MAX_WMM_SETS;
ret = mt7615_mcu_set_dev_info(dev, vif, 1);
if (ret)
@@ -135,20 +136,32 @@ static int mt7615_set_channel(struct mt7615_dev *dev)
int ret;
cancel_delayed_work_sync(&dev->mt76.mac_work);
+
+ mutex_lock(&dev->mt76.mutex);
set_bit(MT76_RESET, &dev->mt76.state);
+ mt7615_dfs_check_channel(dev);
+
mt76_set_channel(&dev->mt76);
ret = mt7615_mcu_set_channel(dev);
if (ret)
- return ret;
+ goto out;
+ ret = mt7615_dfs_init_radar_detector(dev);
+ mt7615_mac_cca_stats_reset(dev);
+ dev->mt76.survey_time = ktime_get_boottime();
+ /* TODO: add DBDC support */
+ mt76_rr(dev, MT_MIB_SDR16(0));
+
+out:
clear_bit(MT76_RESET, &dev->mt76.state);
+ mutex_unlock(&dev->mt76.mutex);
mt76_txq_schedule_all(&dev->mt76);
ieee80211_queue_delayed_work(mt76_hw(dev), &dev->mt76.mac_work,
MT7615_WATCHDOG_TIME);
- return 0;
+ return ret;
}
static int mt7615_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
@@ -172,18 +185,34 @@ static int mt7615_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
!(key->flags & IEEE80211_KEY_FLAG_PAIRWISE))
return -EOPNOTSUPP;
+ /* fall back to sw encryption for unsupported ciphers */
+ switch (key->cipher) {
+ case WLAN_CIPHER_SUITE_AES_CMAC:
+ key->flags |= IEEE80211_KEY_FLAG_GENERATE_MMIE;
+ break;
+ case WLAN_CIPHER_SUITE_WEP40:
+ case WLAN_CIPHER_SUITE_WEP104:
+ case WLAN_CIPHER_SUITE_TKIP:
+ case WLAN_CIPHER_SUITE_CCMP:
+ case WLAN_CIPHER_SUITE_CCMP_256:
+ case WLAN_CIPHER_SUITE_GCMP:
+ case WLAN_CIPHER_SUITE_GCMP_256:
+ case WLAN_CIPHER_SUITE_SMS4:
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
if (cmd == SET_KEY) {
key->hw_key_idx = wcid->idx;
wcid->hw_key_idx = idx;
- } else {
- if (idx == wcid->hw_key_idx)
- wcid->hw_key_idx = -1;
-
- key = NULL;
+ } else if (idx == wcid->hw_key_idx) {
+ wcid->hw_key_idx = -1;
}
- mt76_wcid_key_setup(&dev->mt76, wcid, key);
+ mt76_wcid_key_setup(&dev->mt76, wcid,
+ cmd == SET_KEY ? key : NULL);
- return mt7615_mcu_set_wtbl_key(dev, wcid->idx, key, cmd);
+ return mt7615_mac_wtbl_set_key(dev, wcid, key, cmd);
}
static int mt7615_config(struct ieee80211_hw *hw, u32 changed)
@@ -191,14 +220,14 @@ static int mt7615_config(struct ieee80211_hw *hw, u32 changed)
struct mt7615_dev *dev = hw->priv;
int ret = 0;
- mutex_lock(&dev->mt76.mutex);
-
if (changed & IEEE80211_CONF_CHANGE_CHANNEL) {
ieee80211_stop_queues(hw);
ret = mt7615_set_channel(dev);
ieee80211_wake_queues(hw);
}
+ mutex_lock(&dev->mt76.mutex);
+
if (changed & IEEE80211_CONF_CHANGE_POWER)
ret = mt7615_mcu_set_tx_power(dev);
@@ -220,16 +249,12 @@ static int
mt7615_conf_tx(struct ieee80211_hw *hw, struct ieee80211_vif *vif, u16 queue,
const struct ieee80211_tx_queue_params *params)
{
+ struct mt7615_vif *mvif = (struct mt7615_vif *)vif->drv_priv;
struct mt7615_dev *dev = hw->priv;
- static const u8 wmm_queue_map[] = {
- [IEEE80211_AC_BK] = 0,
- [IEEE80211_AC_BE] = 1,
- [IEEE80211_AC_VI] = 2,
- [IEEE80211_AC_VO] = 3,
- };
- /* TODO: hw wmm_set 1~3 */
- return mt7615_mcu_set_wmm(dev, wmm_queue_map[queue], params);
+ queue += mvif->wmm_idx * MT7615_MAX_WMM_SETS;
+
+ return mt7615_mcu_set_wmm(dev, queue, params);
}
static void mt7615_configure_filter(struct ieee80211_hw *hw,
@@ -299,6 +324,18 @@ static void mt7615_bss_info_changed(struct ieee80211_hw *hw,
mutex_unlock(&dev->mt76.mutex);
}
+static void
+mt7615_channel_switch_beacon(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif,
+ struct cfg80211_chan_def *chandef)
+{
+ struct mt7615_dev *dev = hw->priv;
+
+ mutex_lock(&dev->mt76.mutex);
+ mt7615_mcu_set_bcn(dev, vif, true);
+ mutex_unlock(&dev->mt76.mutex);
+}
+
int mt7615_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta)
{
@@ -358,7 +395,7 @@ static void mt7615_sta_rate_tbl_update(struct ieee80211_hw *hw,
break;
}
msta->n_rates = i;
- mt7615_mcu_set_rates(dev, msta, NULL, msta->rates);
+ mt7615_mac_set_rates(dev, msta, NULL, msta->rates);
msta->rate_probe = false;
spin_unlock_bh(&dev->mt76.lock);
}
@@ -410,7 +447,7 @@ mt7615_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
struct ieee80211_txq *txq = sta->txq[params->tid];
struct mt7615_sta *msta = (struct mt7615_sta *)sta->drv_priv;
u16 tid = params->tid;
- u16 *ssn = ¶ms->ssn;
+ u16 ssn = params->ssn;
struct mt76_txq *mtxq;
if (!txq)
@@ -420,7 +457,7 @@ mt7615_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
switch (action) {
case IEEE80211_AMPDU_RX_START:
- mt76_rx_aggr_start(&dev->mt76, &msta->wcid, tid, *ssn,
+ mt76_rx_aggr_start(&dev->mt76, &msta->wcid, tid, ssn,
params->buf_size);
mt7615_mcu_set_rx_ba(dev, params, 1);
break;
@@ -436,11 +473,10 @@ mt7615_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
case IEEE80211_AMPDU_TX_STOP_FLUSH:
case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
mtxq->aggr = false;
- ieee80211_send_bar(vif, sta->addr, tid, mtxq->agg_ssn);
mt7615_mcu_set_tx_ba(dev, params, 0);
break;
case IEEE80211_AMPDU_TX_START:
- mtxq->agg_ssn = IEEE80211_SN_TO_SEQ(*ssn);
+ mtxq->agg_ssn = IEEE80211_SN_TO_SEQ(ssn);
ieee80211_start_tx_ba_cb_irqsafe(vif, sta->addr, tid);
break;
case IEEE80211_AMPDU_TX_STOP_CONT:
@@ -453,23 +489,6 @@ mt7615_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
return 0;
}
-static void
-mt7615_sw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- const u8 *mac)
-{
- struct mt7615_dev *dev = hw->priv;
-
- set_bit(MT76_SCANNING, &dev->mt76.state);
-}
-
-static void
-mt7615_sw_scan_complete(struct ieee80211_hw *hw, struct ieee80211_vif *vif)
-{
- struct mt7615_dev *dev = hw->priv;
-
- clear_bit(MT76_SCANNING, &dev->mt76.state);
-}
-
const struct ieee80211_ops mt7615_ops = {
.tx = mt7615_tx,
.start = mt7615_start,
@@ -486,8 +505,10 @@ const struct ieee80211_ops mt7615_ops = {
.set_rts_threshold = mt7615_set_rts_threshold,
.wake_tx_queue = mt76_wake_tx_queue,
.sta_rate_tbl_update = mt7615_sta_rate_tbl_update,
- .sw_scan_start = mt7615_sw_scan,
- .sw_scan_complete = mt7615_sw_scan_complete,
+ .sw_scan_start = mt76_sw_scan,
+ .sw_scan_complete = mt76_sw_scan_complete,
.release_buffered_frames = mt76_release_buffered_frames,
.get_txpower = mt76_get_txpower,
+ .channel_switch_beacon = mt7615_channel_switch_beacon,
+ .get_survey = mt76_get_survey,
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
index cdad2c8..275d5ea 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.c
@@ -75,7 +75,7 @@ static int __mt7615_mcu_msg_send(struct mt7615_dev *dev, struct sk_buff *skb,
txd = mcu_txd->txd;
- val = FIELD_PREP(MT_TXD0_TX_BYTES, cpu_to_le16(skb->len)) |
+ val = FIELD_PREP(MT_TXD0_TX_BYTES, skb->len) |
FIELD_PREP(MT_TXD0_P_IDX, MT_TX_PORT_IDX_MCU) |
FIELD_PREP(MT_TXD0_Q_IDX, q_idx);
txd[0] = cpu_to_le32(val);
@@ -113,12 +113,38 @@ static int __mt7615_mcu_msg_send(struct mt7615_dev *dev, struct sk_buff *skb,
}
static int
+mt7615_mcu_parse_response(struct mt7615_dev *dev, int cmd,
+ struct sk_buff *skb, int seq)
+{
+ struct mt7615_mcu_rxd *rxd = (struct mt7615_mcu_rxd *)skb->data;
+ int ret = 0;
+
+ if (seq != rxd->seq)
+ return -EAGAIN;
+
+ switch (cmd) {
+ case -MCU_CMD_PATCH_SEM_CONTROL:
+ skb_pull(skb, sizeof(*rxd) - 4);
+ ret = *skb->data;
+ break;
+ case MCU_EXT_CMD_GET_TEMP:
+ skb_pull(skb, sizeof(*rxd));
+ ret = le32_to_cpu(*(__le32 *)skb->data);
+ break;
+ default:
+ break;
+ }
+ dev_kfree_skb(skb);
+
+ return ret;
+}
+
+static int
mt7615_mcu_msg_send(struct mt76_dev *mdev, int cmd, const void *data,
int len, bool wait_resp)
{
struct mt7615_dev *dev = container_of(mdev, struct mt7615_dev, mt76);
unsigned long expires = jiffies + 10 * HZ;
- struct mt7615_mcu_rxd *rxd;
struct sk_buff *skb;
int ret, seq;
@@ -141,16 +167,9 @@ mt7615_mcu_msg_send(struct mt76_dev *mdev, int cmd, const void *data,
break;
}
- rxd = (struct mt7615_mcu_rxd *)skb->data;
- if (seq != rxd->seq)
- continue;
-
- if (cmd == -MCU_CMD_PATCH_SEM_CONTROL) {
- skb_pull(skb, sizeof(*rxd) - 4);
- ret = *skb->data;
- }
- dev_kfree_skb(skb);
- break;
+ ret = mt7615_mcu_parse_response(dev, cmd, skb, seq);
+ if (ret != -EAGAIN)
+ break;
}
out:
@@ -159,6 +178,62 @@ mt7615_mcu_msg_send(struct mt76_dev *mdev, int cmd, const void *data,
return ret;
}
+static void
+mt7615_mcu_csa_finish(void *priv, u8 *mac, struct ieee80211_vif *vif)
+{
+ if (vif->csa_active)
+ ieee80211_csa_finish(vif);
+}
+
+static void
+mt7615_mcu_rx_ext_event(struct mt7615_dev *dev, struct sk_buff *skb)
+{
+ struct mt7615_mcu_rxd *rxd = (struct mt7615_mcu_rxd *)skb->data;
+
+ switch (rxd->ext_eid) {
+ case MCU_EXT_EVENT_RDD_REPORT:
+ ieee80211_radar_detected(dev->mt76.hw);
+ dev->hw_pattern++;
+ break;
+ case MCU_EXT_EVENT_CSA_NOTIFY:
+ ieee80211_iterate_active_interfaces_atomic(dev->mt76.hw,
+ IEEE80211_IFACE_ITER_RESUME_ALL,
+ mt7615_mcu_csa_finish, dev);
+ break;
+ default:
+ break;
+ }
+}
+
+static void
+mt7615_mcu_rx_unsolicited_event(struct mt7615_dev *dev, struct sk_buff *skb)
+{
+ struct mt7615_mcu_rxd *rxd = (struct mt7615_mcu_rxd *)skb->data;
+
+ switch (rxd->eid) {
+ case MCU_EVENT_EXT:
+ mt7615_mcu_rx_ext_event(dev, skb);
+ break;
+ default:
+ break;
+ }
+ dev_kfree_skb(skb);
+}
+
+void mt7615_mcu_rx_event(struct mt7615_dev *dev, struct sk_buff *skb)
+{
+ struct mt7615_mcu_rxd *rxd = (struct mt7615_mcu_rxd *)skb->data;
+
+ if (rxd->ext_eid == MCU_EXT_EVENT_THERMAL_PROTECT ||
+ rxd->ext_eid == MCU_EXT_EVENT_FW_LOG_2_HOST ||
+ rxd->ext_eid == MCU_EXT_EVENT_ASSERT_DUMP ||
+ rxd->ext_eid == MCU_EXT_EVENT_PS_SYNC ||
+ !rxd->seq)
+ mt7615_mcu_rx_unsolicited_event(dev, skb);
+ else
+ mt76_mcu_rx_event(&dev->mt76, skb);
+}
+
static int mt7615_mcu_init_download(struct mt7615_dev *dev, u32 addr,
u32 len, u32 mode)
{
@@ -192,6 +267,7 @@ static int mt7615_mcu_send_firmware(struct mt7615_dev *dev, const void *data,
data += cur_len;
len -= cur_len;
+ mt76_queue_tx_cleanup(dev, MT_TXQ_FWDL, false);
}
return ret;
@@ -257,9 +333,9 @@ static int mt7615_driver_own(struct mt7615_dev *dev)
static int mt7615_load_patch(struct mt7615_dev *dev)
{
- const struct firmware *fw;
- const struct mt7615_patch_hdr *hdr;
const char *firmware = MT7615_ROM_PATCH;
+ const struct mt7615_patch_hdr *hdr;
+ const struct firmware *fw = NULL;
int len, ret, sem;
sem = mt7615_mcu_patch_sem_ctrl(dev, 1);
@@ -275,7 +351,7 @@ static int mt7615_load_patch(struct mt7615_dev *dev)
ret = request_firmware(&fw, firmware, dev->mt76.dev);
if (ret)
- return ret;
+ goto out;
if (!fw || !fw->data || fw->size < sizeof(*hdr)) {
dev_err(dev->mt76.dev, "Invalid firmware\n");
@@ -323,7 +399,7 @@ static int mt7615_load_patch(struct mt7615_dev *dev)
return ret;
}
-static u32 gen_dl_mode(u8 feature_set, bool is_cr4)
+static u32 mt7615_mcu_gen_dl_mode(u8 feature_set, bool is_cr4)
{
u32 ret = 0;
@@ -337,14 +413,45 @@ static u32 gen_dl_mode(u8 feature_set, bool is_cr4)
return ret;
}
+static int
+mt7615_mcu_send_ram_firmware(struct mt7615_dev *dev,
+ const struct mt7615_fw_trailer *hdr,
+ const u8 *data, bool is_cr4)
+{
+ int n_region = is_cr4 ? CR4_REGION_NUM : N9_REGION_NUM;
+ int err, i, offset = 0;
+ u32 len, addr, mode;
+
+ for (i = 0; i < n_region; i++) {
+ mode = mt7615_mcu_gen_dl_mode(hdr[i].feature_set, is_cr4);
+ len = le32_to_cpu(hdr[i].len) + IMG_CRC_LEN;
+ addr = le32_to_cpu(hdr[i].addr);
+
+ err = mt7615_mcu_init_download(dev, addr, len, mode);
+ if (err) {
+ dev_err(dev->mt76.dev, "Download request failed\n");
+ return err;
+ }
+
+ err = mt7615_mcu_send_firmware(dev, data + offset, len);
+ if (err) {
+ dev_err(dev->mt76.dev, "Failed to send firmware to device\n");
+ return err;
+ }
+
+ offset += len;
+ }
+
+ return 0;
+}
+
static int mt7615_load_ram(struct mt7615_dev *dev)
{
const struct firmware *fw;
const struct mt7615_fw_trailer *hdr;
const char *n9_firmware = MT7615_FIRMWARE_N9;
const char *cr4_firmware = MT7615_FIRMWARE_CR4;
- u32 n9_ilm_addr, offset;
- int i, ret;
+ int ret;
ret = request_firmware(&fw, n9_firmware, dev->mt76.dev);
if (ret)
@@ -362,31 +469,12 @@ static int mt7615_load_ram(struct mt7615_dev *dev)
dev_info(dev->mt76.dev, "N9 Firmware Version: %.10s, Build Time: %.15s\n",
hdr->fw_ver, hdr->build_date);
- n9_ilm_addr = le32_to_cpu(hdr->addr);
+ ret = mt7615_mcu_send_ram_firmware(dev, hdr, fw->data, false);
+ if (ret)
+ goto out;
- for (offset = 0, i = 0; i < N9_REGION_NUM; i++) {
- u32 len, addr, mode;
-
- len = le32_to_cpu(hdr[i].len) + IMG_CRC_LEN;
- addr = le32_to_cpu(hdr[i].addr);
- mode = gen_dl_mode(hdr[i].feature_set, false);
-
- ret = mt7615_mcu_init_download(dev, addr, len, mode);
- if (ret) {
- dev_err(dev->mt76.dev, "Download request failed\n");
- goto out;
- }
-
- ret = mt7615_mcu_send_firmware(dev, fw->data + offset, len);
- if (ret) {
- dev_err(dev->mt76.dev, "Failed to send firmware to device\n");
- goto out;
- }
-
- offset += len;
- }
-
- ret = mt7615_mcu_start_firmware(dev, n9_ilm_addr, FW_START_OVERRIDE);
+ ret = mt7615_mcu_start_firmware(dev, le32_to_cpu(hdr->addr),
+ FW_START_OVERRIDE);
if (ret) {
dev_err(dev->mt76.dev, "Failed to start N9 firmware\n");
goto out;
@@ -410,27 +498,9 @@ static int mt7615_load_ram(struct mt7615_dev *dev)
dev_info(dev->mt76.dev, "CR4 Firmware Version: %.10s, Build Time: %.15s\n",
hdr->fw_ver, hdr->build_date);
- for (offset = 0, i = 0; i < CR4_REGION_NUM; i++) {
- u32 len, addr, mode;
-
- len = le32_to_cpu(hdr[i].len) + IMG_CRC_LEN;
- addr = le32_to_cpu(hdr[i].addr);
- mode = gen_dl_mode(hdr[i].feature_set, true);
-
- ret = mt7615_mcu_init_download(dev, addr, len, mode);
- if (ret) {
- dev_err(dev->mt76.dev, "Download request failed\n");
- goto out;
- }
-
- ret = mt7615_mcu_send_firmware(dev, fw->data + offset, len);
- if (ret) {
- dev_err(dev->mt76.dev, "Failed to send firmware to device\n");
- goto out;
- }
-
- offset += len;
- }
+ ret = mt7615_mcu_send_ram_firmware(dev, hdr, fw->data, true);
+ if (ret)
+ goto out;
ret = mt7615_mcu_start_firmware(dev, 0, FW_START_WORKING_PDA_CR4);
if (ret)
@@ -469,6 +539,8 @@ static int mt7615_load_firmware(struct mt7615_dev *dev)
return -EIO;
}
+ mt76_queue_tx_cleanup(dev, MT_TXQ_FWDL, false);
+
dev_dbg(dev->mt76.dev, "Firmware init done\n");
return 0;
@@ -573,6 +645,8 @@ int mt7615_mcu_set_wmm(struct mt7615_dev *dev, u8 queue,
#define WMM_CW_MIN_SET BIT(1)
#define WMM_CW_MAX_SET BIT(2)
#define WMM_TXOP_SET BIT(3)
+#define WMM_PARAM_SET (WMM_AIFS_SET | WMM_CW_MIN_SET | \
+ WMM_CW_MAX_SET | WMM_TXOP_SET)
struct req_data {
u8 number;
u8 rsv[3];
@@ -585,19 +659,17 @@ int mt7615_mcu_set_wmm(struct mt7615_dev *dev, u8 queue,
} __packed req = {
.number = 1,
.queue = queue,
- .valid = WMM_AIFS_SET | WMM_TXOP_SET,
+ .valid = WMM_PARAM_SET,
.aifs = params->aifs,
+ .cw_min = 5,
+ .cw_max = cpu_to_le16(10),
.txop = cpu_to_le16(params->txop),
};
- if (params->cw_min) {
- req.valid |= WMM_CW_MIN_SET;
- req.cw_min = params->cw_min;
- }
- if (params->cw_max) {
- req.valid |= WMM_CW_MAX_SET;
- req.cw_max = cpu_to_le16(params->cw_max);
- }
+ if (params->cw_min)
+ req.cw_min = fls(params->cw_min);
+ if (params->cw_max)
+ req.cw_max = cpu_to_le16(fls(params->cw_max));
return __mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_EDCA_UPDATE,
&req, sizeof(req), true);
@@ -824,78 +896,6 @@ int mt7615_mcu_set_bss_info(struct mt7615_dev *dev,
return ret;
}
-static enum mt7615_cipher_type
-mt7615_get_key_info(struct ieee80211_key_conf *key, u8 *key_data)
-{
- if (!key || key->keylen > 32)
- return MT_CIPHER_NONE;
-
- memcpy(key_data, key->key, key->keylen);
-
- switch (key->cipher) {
- case WLAN_CIPHER_SUITE_WEP40:
- return MT_CIPHER_WEP40;
- case WLAN_CIPHER_SUITE_WEP104:
- return MT_CIPHER_WEP104;
- case WLAN_CIPHER_SUITE_TKIP:
- /* Rx/Tx MIC keys are swapped */
- memcpy(key_data + 16, key->key + 24, 8);
- memcpy(key_data + 24, key->key + 16, 8);
- return MT_CIPHER_TKIP;
- case WLAN_CIPHER_SUITE_CCMP:
- return MT_CIPHER_AES_CCMP;
- case WLAN_CIPHER_SUITE_CCMP_256:
- return MT_CIPHER_CCMP_256;
- case WLAN_CIPHER_SUITE_GCMP:
- return MT_CIPHER_GCMP;
- case WLAN_CIPHER_SUITE_GCMP_256:
- return MT_CIPHER_GCMP_256;
- case WLAN_CIPHER_SUITE_SMS4:
- return MT_CIPHER_WAPI;
- default:
- return MT_CIPHER_NONE;
- }
-}
-
-int mt7615_mcu_set_wtbl_key(struct mt7615_dev *dev, int wcid,
- struct ieee80211_key_conf *key,
- enum set_key_cmd cmd)
-{
- struct {
- struct wtbl_req_hdr hdr;
- struct wtbl_sec_key key;
- } req = {
- .hdr = {
- .wlan_idx = wcid,
- .operation = WTBL_SET,
- .tlv_num = cpu_to_le16(1),
- },
- .key = {
- .tag = cpu_to_le16(WTBL_SEC_KEY),
- .len = cpu_to_le16(sizeof(struct wtbl_sec_key)),
- .add = cmd,
- },
- };
-
- if (cmd == SET_KEY) {
- u8 cipher;
-
- cipher = mt7615_get_key_info(key, req.key.key_material);
- if (cipher == MT_CIPHER_NONE)
- return -EOPNOTSUPP;
-
- req.key.rkv = 1;
- req.key.cipher_id = cipher;
- req.key.key_id = key->keyidx;
- req.key.key_len = key->keylen;
- } else {
- req.key.key_len = sizeof(req.key.key_material);
- }
-
- return __mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_WTBL_UPDATE,
- &req, sizeof(req), true);
-}
-
static int
mt7615_mcu_add_wtbl_bmc(struct mt7615_dev *dev,
struct mt7615_vif *mvif)
@@ -1099,6 +1099,7 @@ int mt7615_mcu_set_bcn(struct mt7615_dev *dev, struct ieee80211_vif *vif,
{
struct mt7615_vif *mvif = (struct mt7615_vif *)vif->drv_priv;
struct mt76_wcid *wcid = &dev->mt76.global_wcid;
+ struct ieee80211_mutable_offsets offs;
struct req {
u8 omac_idx;
u8 enable;
@@ -1119,13 +1120,10 @@ int mt7615_mcu_set_bcn(struct mt7615_dev *dev, struct ieee80211_vif *vif,
.enable = en,
.wlan_idx = wcid->idx,
.band_idx = mvif->band_idx,
- /* pky_type: 0 for bcn, 1 for tim */
- .pkt_type = 0,
};
struct sk_buff *skb;
- u16 tim_off;
- skb = ieee80211_beacon_get_tim(mt76_hw(dev), vif, &tim_off, NULL);
+ skb = ieee80211_beacon_get_template(mt76_hw(dev), vif, &offs);
if (!skb)
return -EINVAL;
@@ -1139,8 +1137,14 @@ int mt7615_mcu_set_bcn(struct mt7615_dev *dev, struct ieee80211_vif *vif,
0, NULL);
memcpy(req.pkt + MT_TXD_SIZE, skb->data, skb->len);
req.pkt_len = cpu_to_le16(MT_TXD_SIZE + skb->len);
- req.tim_ie_pos = cpu_to_le16(MT_TXD_SIZE + tim_off);
+ req.tim_ie_pos = cpu_to_le16(MT_TXD_SIZE + offs.tim_offset);
+ if (offs.csa_counter_offs[0]) {
+ u16 csa_offs;
+ csa_offs = MT_TXD_SIZE + offs.csa_counter_offs[0] - 4;
+ req.csa_ie_pos = cpu_to_le16(csa_offs);
+ req.csa_cnt = skb->data[offs.csa_counter_offs[0]];
+ }
dev_kfree_skb(skb);
return __mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_BCN_OFFLOAD,
@@ -1213,9 +1217,62 @@ int mt7615_mcu_set_tx_power(struct mt7615_dev *dev)
return ret;
}
+int mt7615_mcu_rdd_cmd(struct mt7615_dev *dev,
+ enum mt7615_rdd_cmd cmd, u8 index,
+ u8 rx_sel, u8 val)
+{
+ struct {
+ u8 ctrl;
+ u8 rdd_idx;
+ u8 rdd_rx_sel;
+ u8 val;
+ u8 rsv[4];
+ } req = {
+ .ctrl = cmd,
+ .rdd_idx = index,
+ .rdd_rx_sel = rx_sel,
+ .val = val,
+ };
+
+ return __mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_CTRL,
+ &req, sizeof(req), true);
+}
+
+int mt7615_mcu_rdd_send_pattern(struct mt7615_dev *dev)
+{
+ struct {
+ u8 pulse_num;
+ u8 rsv[3];
+ struct {
+ u32 start_time;
+ u16 width;
+ s16 power;
+ } pattern[32];
+ } req = {
+ .pulse_num = dev->radar_pattern.n_pulses,
+ };
+ u32 start_time = ktime_to_ms(ktime_get_boottime());
+ int i;
+
+ if (dev->radar_pattern.n_pulses > ARRAY_SIZE(req.pattern))
+ return -EINVAL;
+
+ /* TODO: add some noise here */
+ for (i = 0; i < dev->radar_pattern.n_pulses; i++) {
+ req.pattern[i].width = dev->radar_pattern.width;
+ req.pattern[i].power = dev->radar_pattern.power;
+ req.pattern[i].start_time = start_time +
+ i * dev->radar_pattern.period;
+ }
+
+ return __mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_SET_RDD_PATTERN,
+ &req, sizeof(req), false);
+}
+
int mt7615_mcu_set_channel(struct mt7615_dev *dev)
{
- struct cfg80211_chan_def *chdef = &dev->mt76.chandef;
+ struct cfg80211_chan_def *chandef = &dev->mt76.chandef;
+ int freq1 = chandef->center_freq1, freq2 = chandef->center_freq2;
struct {
u8 control_chan;
u8 center_chan;
@@ -1234,17 +1291,20 @@ int mt7615_mcu_set_channel(struct mt7615_dev *dev)
u8 rsv1[3];
u8 txpower_sku[53];
u8 rsv2[3];
- } req = {0};
+ } req = {
+ .control_chan = chandef->chan->hw_value,
+ .center_chan = ieee80211_frequency_to_channel(freq1),
+ .tx_streams = (dev->mt76.chainmask >> 8) & 0xf,
+ .rx_streams_mask = dev->mt76.antenna_mask,
+ .center_chan2 = ieee80211_frequency_to_channel(freq2),
+ };
int ret;
- req.control_chan = chdef->chan->hw_value;
- req.center_chan = ieee80211_frequency_to_channel(chdef->center_freq1);
- req.tx_streams = (dev->mt76.chainmask >> 8) & 0xf;
- req.rx_streams_mask = dev->mt76.antenna_mask;
- req.switch_reason = CH_SWITCH_NORMAL;
- req.band_idx = 0;
- req.center_chan2 = ieee80211_frequency_to_channel(chdef->center_freq2);
- req.txpower_drop = 0;
+ if ((chandef->chan->flags & IEEE80211_CHAN_RADAR) &&
+ chandef->chan->dfs_state != NL80211_DFS_AVAILABLE)
+ req.switch_reason = CH_SWITCH_DFS;
+ else
+ req.switch_reason = CH_SWITCH_NORMAL;
switch (dev->mt76.chandef.width) {
case NL80211_CHAN_WIDTH_40:
@@ -1269,6 +1329,7 @@ int mt7615_mcu_set_channel(struct mt7615_dev *dev)
case NL80211_CHAN_WIDTH_20:
default:
req.bw = CMD_CBW_20MHZ;
+ break;
}
memset(req.txpower_sku, 0x3f, 49);
@@ -1533,92 +1594,15 @@ int mt7615_mcu_set_rx_ba(struct mt7615_dev *dev,
&wtbl_req, sizeof(wtbl_req), true);
}
-void mt7615_mcu_set_rates(struct mt7615_dev *dev, struct mt7615_sta *sta,
- struct ieee80211_tx_rate *probe_rate,
- struct ieee80211_tx_rate *rates)
+int mt7615_mcu_get_temperature(struct mt7615_dev *dev, int index)
{
- int wcid = sta->wcid.idx;
- u32 addr = MT_WTBL_BASE + wcid * MT_WTBL_ENTRY_SIZE;
- bool stbc = false;
- int n_rates = sta->n_rates;
- u8 bw, bw_prev, bw_idx = 0;
- u16 val[4];
- u16 probe_val;
- u32 w5, w27;
- int i;
+ struct {
+ u8 action;
+ u8 rsv[3];
+ } req = {
+ .action = index,
+ };
- if (!mt76_poll(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_BUSY, 0, 5000))
- return;
-
- for (i = n_rates; i < 4; i++)
- rates[i] = rates[n_rates - 1];
-
- val[0] = mt7615_mac_tx_rate_val(dev, &rates[0], stbc, &bw);
- bw_prev = bw;
-
- if (probe_rate) {
- probe_val = mt7615_mac_tx_rate_val(dev, probe_rate, stbc, &bw);
- if (bw)
- bw_idx = 1;
- else
- bw_prev = 0;
- } else {
- probe_val = val[0];
- }
-
- val[1] = mt7615_mac_tx_rate_val(dev, &rates[1], stbc, &bw);
- if (bw_prev) {
- bw_idx = 3;
- bw_prev = bw;
- }
-
- val[2] = mt7615_mac_tx_rate_val(dev, &rates[2], stbc, &bw);
- if (bw_prev) {
- bw_idx = 5;
- bw_prev = bw;
- }
-
- val[3] = mt7615_mac_tx_rate_val(dev, &rates[3], stbc, &bw);
- if (bw_prev)
- bw_idx = 7;
-
- w27 = mt76_rr(dev, addr + 27 * 4);
- w27 &= ~MT_WTBL_W27_CC_BW_SEL;
- w27 |= FIELD_PREP(MT_WTBL_W27_CC_BW_SEL, bw);
-
- w5 = mt76_rr(dev, addr + 5 * 4);
- w5 &= ~(MT_WTBL_W5_BW_CAP | MT_WTBL_W5_CHANGE_BW_RATE);
- w5 |= FIELD_PREP(MT_WTBL_W5_BW_CAP, bw) |
- FIELD_PREP(MT_WTBL_W5_CHANGE_BW_RATE, bw_idx ? bw_idx - 1 : 7);
-
- mt76_wr(dev, MT_WTBL_RIUCR0, w5);
-
- mt76_wr(dev, MT_WTBL_RIUCR1,
- FIELD_PREP(MT_WTBL_RIUCR1_RATE0, probe_val) |
- FIELD_PREP(MT_WTBL_RIUCR1_RATE1, val[0]) |
- FIELD_PREP(MT_WTBL_RIUCR1_RATE2_LO, val[0]));
-
- mt76_wr(dev, MT_WTBL_RIUCR2,
- FIELD_PREP(MT_WTBL_RIUCR2_RATE2_HI, val[0] >> 8) |
- FIELD_PREP(MT_WTBL_RIUCR2_RATE3, val[1]) |
- FIELD_PREP(MT_WTBL_RIUCR2_RATE4, val[1]) |
- FIELD_PREP(MT_WTBL_RIUCR2_RATE5_LO, val[2]));
-
- mt76_wr(dev, MT_WTBL_RIUCR3,
- FIELD_PREP(MT_WTBL_RIUCR3_RATE5_HI, val[2] >> 4) |
- FIELD_PREP(MT_WTBL_RIUCR3_RATE6, val[2]) |
- FIELD_PREP(MT_WTBL_RIUCR3_RATE7, val[3]));
-
- mt76_wr(dev, MT_WTBL_UPDATE,
- FIELD_PREP(MT_WTBL_UPDATE_WLAN_IDX, wcid) |
- MT_WTBL_UPDATE_RATE_UPDATE |
- MT_WTBL_UPDATE_TX_COUNT_CLEAR);
-
- mt76_wr(dev, addr + 27 * 4, w27);
-
- if (!(sta->wcid.tx_info & MT_WCID_TX_INFO_SET))
- mt76_poll(dev, MT_WTBL_UPDATE, MT_WTBL_UPDATE_BUSY, 0, 5000);
-
- sta->rate_count = 2 * MT7615_RATE_RETRY * n_rates;
- sta->wcid.tx_info |= MT_WCID_TX_INFO_SET;
+ return __mt76_mcu_send_msg(&dev->mt76, MCU_EXT_CMD_GET_TEMP, &req,
+ sizeof(req), true);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
index f8b51ad..1fd7dffa6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mcu.h
@@ -23,6 +23,28 @@ struct mt7615_mcu_txd {
u32 reserved[5];
} __packed __aligned(4);
+/* event table */
+enum {
+ MCU_EVENT_TARGET_ADDRESS_LEN = 0x01,
+ MCU_EVENT_FW_START = 0x01,
+ MCU_EVENT_GENERIC = 0x01,
+ MCU_EVENT_ACCESS_REG = 0x02,
+ MCU_EVENT_MT_PATCH_SEM = 0x04,
+ MCU_EVENT_CH_PRIVILEGE = 0x18,
+ MCU_EVENT_EXT = 0xed,
+ MCU_EVENT_RESTART_DL = 0xef,
+};
+
+/* ext event table */
+enum {
+ MCU_EXT_EVENT_PS_SYNC = 0x5,
+ MCU_EXT_EVENT_FW_LOG_2_HOST = 0x13,
+ MCU_EXT_EVENT_THERMAL_PROTECT = 0x22,
+ MCU_EXT_EVENT_ASSERT_DUMP = 0x23,
+ MCU_EXT_EVENT_RDD_REPORT = 0x3a,
+ MCU_EXT_EVENT_CSA_NOTIFY = 0x4f,
+};
+
struct mt7615_mcu_rxd {
__le32 rxd[4];
@@ -76,11 +98,14 @@ enum {
MCU_EXT_CMD_BSS_INFO_UPDATE = 0x26,
MCU_EXT_CMD_EDCA_UPDATE = 0x27,
MCU_EXT_CMD_DEV_INFO_UPDATE = 0x2A,
+ MCU_EXT_CMD_GET_TEMP = 0x2c,
MCU_EXT_CMD_WTBL_UPDATE = 0x32,
+ MCU_EXT_CMD_SET_RDD_CTRL = 0x3a,
MCU_EXT_CMD_PROTECT_CTRL = 0x3e,
MCU_EXT_CMD_MAC_INIT_CTRL = 0x46,
MCU_EXT_CMD_BCN_OFFLOAD = 0x49,
MCU_EXT_CMD_SET_RX_PATH = 0x4e,
+ MCU_EXT_CMD_SET_RDD_PATTERN = 0x7d,
};
enum {
@@ -264,34 +289,6 @@ struct wtbl_hdr_trans {
u8 rsv;
} __packed;
-enum mt7615_cipher_type {
- MT_CIPHER_NONE,
- MT_CIPHER_WEP40,
- MT_CIPHER_TKIP,
- MT_CIPHER_TKIP_NO_MIC,
- MT_CIPHER_AES_CCMP,
- MT_CIPHER_WEP104,
- MT_CIPHER_BIP_CMAC_128,
- MT_CIPHER_WEP128,
- MT_CIPHER_WAPI,
- MT_CIPHER_CCMP_256 = 10,
- MT_CIPHER_GCMP,
- MT_CIPHER_GCMP_256,
-};
-
-struct wtbl_sec_key {
- __le16 tag;
- __le16 len;
- u8 add; /* 0: add, 1: remove */
- u8 rkv;
- u8 ikv;
- u8 cipher_id;
- u8 key_id;
- u8 key_len;
- u8 rsv[2];
- u8 key_material[32];
-} __packed;
-
enum {
MT_BA_TYPE_INVALID,
MT_BA_TYPE_ORIGINATOR,
@@ -375,7 +372,6 @@ struct wtbl_raw {
sizeof(struct wtbl_vht) + \
sizeof(struct wtbl_tx_ps) + \
sizeof(struct wtbl_hdr_trans) + \
- sizeof(struct wtbl_sec_key) + \
sizeof(struct wtbl_ba) + \
sizeof(struct wtbl_bf) + \
sizeof(struct wtbl_smps) + \
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h b/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
index f02ffcf..cef3fd43 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h
@@ -10,12 +10,13 @@
#include "regs.h"
#define MT7615_MAX_INTERFACES 4
+#define MT7615_MAX_WMM_SETS 4
#define MT7615_WTBL_SIZE 128
#define MT7615_WTBL_RESERVED (MT7615_WTBL_SIZE - 1)
#define MT7615_WTBL_STA (MT7615_WTBL_RESERVED - \
MT7615_MAX_INTERFACES)
-#define MT7615_WATCHDOG_TIME 100 /* ms */
+#define MT7615_WATCHDOG_TIME (HZ / 10)
#define MT7615_RATE_RETRY 2
#define MT7615_TX_RING_SIZE 1024
@@ -32,6 +33,9 @@
#define MT7615_EEPROM_SIZE 1024
#define MT7615_TOKEN_SIZE 4096
+#define MT_FRAC_SCALE 12
+#define MT_FRAC(val, div) (((val) << MT_FRAC_SCALE) / (div))
+
struct mt7615_vif;
struct mt7615_sta;
@@ -42,12 +46,21 @@ enum mt7615_hw_txq_id {
MT7615_TXQ_FWDL,
};
+struct mt7615_rate_set {
+ struct ieee80211_tx_rate probe_rate;
+ struct ieee80211_tx_rate rates[4];
+};
+
struct mt7615_sta {
struct mt76_wcid wcid; /* must be first */
struct mt7615_vif *vif;
- struct ieee80211_tx_rate rates[8];
+ struct ieee80211_tx_rate rates[4];
+
+ struct mt7615_rate_set rateset[2];
+ u32 rate_set_tsf;
+
u8 rate_count;
u8 n_rates;
@@ -68,6 +81,22 @@ struct mt7615_dev {
u32 vif_mask;
u32 omac_mask;
+ struct {
+ u8 n_pulses;
+ u32 period;
+ u16 width;
+ s16 power;
+ } radar_pattern;
+ u32 hw_pattern;
+ int dfs_state;
+
+ int false_cca_ofdm, false_cca_cck;
+ unsigned long last_cca_adj;
+ u8 mac_work_count;
+ s8 ofdm_sensitivity;
+ s8 cck_sensitivity;
+ bool scs_en;
+
spinlock_t token_lock;
struct idr token;
};
@@ -97,6 +126,30 @@ enum {
EXT_BSSID_END
};
+enum {
+ MT_HW_RDD0,
+ MT_HW_RDD1,
+};
+
+enum {
+ MT_RX_SEL0,
+ MT_RX_SEL1,
+};
+
+enum mt7615_rdd_cmd {
+ RDD_STOP,
+ RDD_START,
+ RDD_DET_MODE,
+ RDD_DET_STOP,
+ RDD_CAC_START,
+ RDD_CAC_END,
+ RDD_NORMAL_START,
+ RDD_DISABLE_DFS_CAL,
+ RDD_PULSE_DBG,
+ RDD_READ_PULSE,
+ RDD_RESUME_BF,
+};
+
extern const struct ieee80211_ops mt7615_ops;
extern struct pci_driver mt7615_pci_driver;
@@ -115,10 +168,7 @@ int mt7615_mcu_set_dev_info(struct mt7615_dev *dev,
struct ieee80211_vif *vif, bool enable);
int mt7615_mcu_set_bss_info(struct mt7615_dev *dev, struct ieee80211_vif *vif,
int en);
-int mt7615_mcu_set_wtbl_key(struct mt7615_dev *dev, int wcid,
- struct ieee80211_key_conf *key,
- enum set_key_cmd cmd);
-void mt7615_mcu_set_rates(struct mt7615_dev *dev, struct mt7615_sta *sta,
+void mt7615_mac_set_rates(struct mt7615_dev *dev, struct mt7615_sta *sta,
struct ieee80211_tx_rate *probe_rate,
struct ieee80211_tx_rate *rates);
int mt7615_mcu_wtbl_bmc(struct mt7615_dev *dev, struct ieee80211_vif *vif,
@@ -144,6 +194,29 @@ int mt7615_mcu_set_rx_ba(struct mt7615_dev *dev,
bool add);
int mt7615_mcu_set_ht_cap(struct mt7615_dev *dev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
+void mt7615_mcu_rx_event(struct mt7615_dev *dev, struct sk_buff *skb);
+int mt7615_mcu_rdd_cmd(struct mt7615_dev *dev,
+ enum mt7615_rdd_cmd cmd, u8 index,
+ u8 rx_sel, u8 val);
+int mt7615_dfs_start_radar_detector(struct mt7615_dev *dev);
+int mt7615_dfs_stop_radar_detector(struct mt7615_dev *dev);
+int mt7615_mcu_rdd_send_pattern(struct mt7615_dev *dev);
+
+static inline bool is_mt7622(struct mt76_dev *dev)
+{
+ return mt76_chip(dev) == 0x7622;
+}
+
+static inline void mt7615_dfs_check_channel(struct mt7615_dev *dev)
+{
+ enum nl80211_chan_width width = dev->mt76.chandef.width;
+ u32 freq = dev->mt76.chandef.chan->center_freq;
+ struct ieee80211_hw *hw = mt76_hw(dev);
+
+ if (hw->conf.chandef.chan->center_freq != freq ||
+ hw->conf.chandef.width != width)
+ dev->dfs_state = -1;
+}
static inline void mt7615_irq_enable(struct mt7615_dev *dev, u32 mask)
{
@@ -155,9 +228,9 @@ static inline void mt7615_irq_disable(struct mt7615_dev *dev, u32 mask)
mt76_set_irq_mask(&dev->mt76, MT_INT_MASK_CSR, mask, 0);
}
-u16 mt7615_mac_tx_rate_val(struct mt7615_dev *dev,
- const struct ieee80211_tx_rate *rate,
- bool stbc, u8 *bw);
+void mt7615_update_channel(struct mt76_dev *mdev);
+void mt7615_mac_cca_stats_reset(struct mt7615_dev *dev);
+void mt7615_mac_set_scs(struct mt7615_dev *dev, bool enable);
int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
struct sk_buff *skb, struct mt76_wcid *wcid,
struct ieee80211_sta *sta, int pid,
@@ -165,11 +238,15 @@ int mt7615_mac_write_txwi(struct mt7615_dev *dev, __le32 *txwi,
int mt7615_mac_fill_rx(struct mt7615_dev *dev, struct sk_buff *skb);
void mt7615_mac_add_txs(struct mt7615_dev *dev, void *data);
void mt7615_mac_tx_free(struct mt7615_dev *dev, struct sk_buff *skb);
+int mt7615_mac_wtbl_set_key(struct mt7615_dev *dev, struct mt76_wcid *wcid,
+ struct ieee80211_key_conf *key,
+ enum set_key_cmd cmd);
int mt7615_mcu_set_eeprom(struct mt7615_dev *dev);
int mt7615_mcu_init_mac(struct mt7615_dev *dev);
int mt7615_mcu_set_rts_thresh(struct mt7615_dev *dev, u32 val);
int mt7615_mcu_ctrl_pm_state(struct mt7615_dev *dev, int enter);
+int mt7615_mcu_get_temperature(struct mt7615_dev *dev, int index);
int mt7615_mcu_set_tx_power(struct mt7615_dev *dev);
void mt7615_mcu_exit(struct mt7615_dev *dev);
@@ -193,5 +270,9 @@ void mt7615_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
void mt7615_mac_work(struct work_struct *work);
void mt7615_txp_skb_unmap(struct mt76_dev *dev,
struct mt76_txwi_cache *txwi);
+int mt76_dfs_start_rdd(struct mt7615_dev *dev, bool force);
+int mt7615_dfs_init_radar_detector(struct mt7615_dev *dev);
+
+int mt7615_init_debugfs(struct mt7615_dev *dev);
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/pci.c b/drivers/net/wireless/mediatek/mt76/mt7615/pci.c
index 9e82cb5..e250607 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/pci.c
@@ -81,6 +81,7 @@ static int mt7615_pci_probe(struct pci_dev *pdev,
.sta_add = mt7615_sta_add,
.sta_assoc = mt7615_sta_assoc,
.sta_remove = mt7615_sta_remove,
+ .update_survey = mt7615_update_channel,
};
struct mt7615_dev *dev;
struct mt76_dev *mdev;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/regs.h b/drivers/net/wireless/mediatek/mt76/mt7615/regs.h
index 70e5ace..b193814 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/regs.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/regs.h
@@ -71,6 +71,34 @@
#define MT_WF_PHY_WF2_RFCTRL0 MT_WF_PHY(0x1900)
#define MT_WF_PHY_WF2_RFCTRL0_LPBCN_EN BIT(9)
+#define MT_WF_PHY_R0_B0_PHYMUX_5 MT_WF_PHY(0x0614)
+
+#define MT_WF_PHY_R0_B0_PHYCTRL_STS0 MT_WF_PHY(0x020c)
+#define MT_WF_PHYCTRL_STAT_PD_OFDM GENMASK(31, 16)
+#define MT_WF_PHYCTRL_STAT_PD_CCK GENMASK(15, 0)
+
+#define MT_WF_PHY_R0_B0_PHYCTRL_STS5 MT_WF_PHY(0x0220)
+#define MT_WF_PHYCTRL_STAT_MDRDY_OFDM GENMASK(31, 16)
+#define MT_WF_PHYCTRL_STAT_MDRDY_CCK GENMASK(15, 0)
+
+#define MT_WF_PHY_B0_MIN_PRI_PWR MT_WF_PHY(0x229c)
+#define MT_WF_PHY_B0_PD_OFDM_MASK GENMASK(28, 20)
+#define MT_WF_PHY_B0_PD_OFDM(v) ((v) << 20)
+#define MT_WF_PHY_B0_PD_BLK BIT(19)
+
+#define MT_WF_PHY_B1_MIN_PRI_PWR MT_WF_PHY(0x084)
+#define MT_WF_PHY_B1_PD_OFDM_MASK GENMASK(24, 16)
+#define MT_WF_PHY_B1_PD_OFDM(v) ((v) << 16)
+#define MT_WF_PHY_B1_PD_BLK BIT(25)
+
+#define MT_WF_PHY_B0_RXTD_CCK_PD MT_WF_PHY(0x2310)
+#define MT_WF_PHY_B0_PD_CCK_MASK GENMASK(8, 1)
+#define MT_WF_PHY_B0_PD_CCK(v) ((v) << 1)
+
+#define MT_WF_PHY_B1_RXTD_CCK_PD MT_WF_PHY(0x2314)
+#define MT_WF_PHY_B1_PD_CCK_MASK GENMASK(31, 24)
+#define MT_WF_PHY_B1_PD_CCK(v) ((v) << 24)
+
#define MT_WF_CFG_BASE 0x20200
#define MT_WF_CFG(ofs) (MT_WF_CFG_BASE + (ofs))
@@ -97,12 +125,25 @@
MT_AGG_ARxCR_LIMIT_SHIFT(_n), \
MT_AGG_ARxCR_LIMIT_SHIFT(_n))
+#define MT_AGG_ACR0 MT_WF_AGG(0x070)
+#define MT_AGG_ACR1 MT_WF_AGG(0x170)
+#define MT_AGG_ACR_NO_BA_RULE BIT(0)
+#define MT_AGG_ACR_NO_BA_AR_RULE BIT(1)
+#define MT_AGG_ACR_PKT_TIME_EN BIT(2)
+#define MT_AGG_ACR_CFEND_RATE GENMASK(15, 4)
+#define MT_AGG_ACR_BAR_RATE GENMASK(31, 20)
+
#define MT_AGG_SCR MT_WF_AGG(0x0fc)
#define MT_AGG_SCR_NLNAV_MID_PTEC_DIS BIT(3)
#define MT_WF_TMAC_BASE 0x21000
#define MT_WF_TMAC(ofs) (MT_WF_TMAC_BASE + (ofs))
+#define MT_TMAC_TRCR0 MT_WF_TMAC(0x09c)
+#define MT_TMAC_TRCR1 MT_WF_TMAC(0x070)
+#define MT_TMAC_TRCR_CCA_SEL GENMASK(31, 30)
+#define MT_TMAC_TRCR_SEC_CCA_SEL GENMASK(29, 28)
+
#define MT_TMAC_CTCR0 MT_WF_TMAC(0x0f4)
#define MT_TMAC_CTCR0_INS_DDLMT_REFTIME GENMASK(5, 0)
#define MT_TMAC_CTCR0_INS_DDLMT_DENSITY GENMASK(15, 12)
@@ -148,8 +189,15 @@
#define MT_WTBL_OFF_BASE 0x23400
#define MT_WTBL_OFF(n) (MT_WTBL_OFF_BASE + (n))
+#define MT_WTBL_W0_KEY_IDX GENMASK(24, 23)
+#define MT_WTBL_W0_RX_KEY_VALID BIT(26)
+#define MT_WTBL_W0_RX_IK_VALID BIT(27)
+
+#define MT_WTBL_W2_KEY_TYPE GENMASK(7, 4)
+
#define MT_WTBL_UPDATE MT_WTBL_OFF(0x030)
#define MT_WTBL_UPDATE_WLAN_IDX GENMASK(7, 0)
+#define MT_WTBL_UPDATE_RXINFO_UPDATE BIT(11)
#define MT_WTBL_UPDATE_RATE_UPDATE BIT(13)
#define MT_WTBL_UPDATE_TX_COUNT_CLEAR BIT(14)
#define MT_WTBL_UPDATE_BUSY BIT(31)
@@ -157,6 +205,9 @@
#define MT_WTBL_ON_BASE 0x23000
#define MT_WTBL_ON(_n) (MT_WTBL_ON_BASE + (_n))
+#define MT_WTBL_RICR0 MT_WTBL_ON(0x010)
+#define MT_WTBL_RICR1 MT_WTBL_ON(0x014)
+
#define MT_WTBL_RIUCR0 MT_WTBL_ON(0x020)
#define MT_WTBL_RIUCR1 MT_WTBL_ON(0x024)
@@ -181,8 +232,32 @@
#define MT_WTBL_W5_SHORT_GI_80 BIT(10)
#define MT_WTBL_W5_SHORT_GI_160 BIT(11)
#define MT_WTBL_W5_BW_CAP GENMASK(13, 12)
+#define MT_WTBL_W5_MPDU_FAIL_COUNT GENMASK(25, 23)
+#define MT_WTBL_W5_MPDU_OK_COUNT GENMASK(28, 26)
+#define MT_WTBL_W5_RATE_IDX GENMASK(31, 29)
+
#define MT_WTBL_W27_CC_BW_SEL GENMASK(6, 5)
+#define MT_LPON_BASE 0x24200
+#define MT_LPON(_n) (MT_LPON_BASE + (_n))
+
+#define MT_LPON_T0CR MT_LPON(0x010)
+#define MT_LPON_T0CR_MODE GENMASK(1, 0)
+
+#define MT_LPON_UTTR0 MT_LPON(0x018)
+#define MT_LPON_UTTR1 MT_LPON(0x01c)
+
+#define MT_WF_MIB_BASE 0x24800
+#define MT_WF_MIB(ofs) (MT_WF_MIB_BASE + (ofs))
+
+#define MT_MIB_M0_MISC_CR MT_WF_MIB(0x00c)
+#define MT_MIB_MB_SDR0(n) MT_WF_MIB(0x100 + ((n) << 4))
+#define MT_MIB_RTS_RETRIES_COUNT_MASK GENMASK(31, 16)
+#define MT_MIB_RTS_COUNT_MASK GENMASK(15, 0)
+
+#define MT_MIB_SDR16(n) MT_WF_MIB(0x48 + ((n) << 9))
+#define MT_MIB_BUSY_MASK GENMASK(23, 0)
+
#define MT_EFUSE_BASE 0x81070000
#define MT_EFUSE_BASE_CTRL 0x000
#define MT_EFUSE_BASE_CTRL_EMPTY BIT(30)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/Kconfig b/drivers/net/wireless/mediatek/mt76/mt76x0/Kconfig
index 209d8ab..7c88ed8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/Kconfig
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/Kconfig
@@ -10,7 +10,11 @@
depends on MAC80211
depends on USB
help
- This adds support for MT7610U-based wireless USB dongles.
+ This adds support for MT7610U-based wireless USB 2.0 dongles,
+ which comply with IEEE 802.11ac standards and support 1x1
+ 433Mbps PHY rate.
+
+ To compile this driver as a module, choose M here.
config MT76x0E
tristate "MediaTek MT76x0E (PCIe) support"
@@ -18,4 +22,8 @@
depends on MAC80211
depends on PCI
help
- This adds support for MT7610/MT7630-based wireless PCIe devices.
+ This adds support for MT7610/MT7630-based wireless PCIe devices,
+ which comply with IEEE 802.11ac standards and support 1x1
+ 433Mbps PHY rate.
+
+ To compile this driver as a module, choose M here.
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/main.c b/drivers/net/wireless/mediatek/mt76/mt76x0/main.c
index d7bf7bc..efb7ca9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/main.c
@@ -8,18 +8,16 @@
#include <linux/etherdevice.h>
#include "mt76x0.h"
-static int
+static void
mt76x0_set_channel(struct mt76x02_dev *dev, struct cfg80211_chan_def *chandef)
{
- int ret;
-
cancel_delayed_work_sync(&dev->cal_work);
- dev->beacon_ops->pre_tbtt_enable(dev, false);
+ mt76x02_pre_tbtt_enable(dev, false);
if (mt76_is_mmio(dev))
tasklet_disable(&dev->dfs_pd.dfs_tasklet);
mt76_set_channel(&dev->mt76);
- ret = mt76x0_phy_set_channel(dev, chandef);
+ mt76x0_phy_set_channel(dev, chandef);
/* channel cycle counters read-and-clear */
mt76_rr(dev, MT_CH_IDLE);
@@ -31,23 +29,20 @@ mt76x0_set_channel(struct mt76x02_dev *dev, struct cfg80211_chan_def *chandef)
mt76x02_dfs_init_params(dev);
tasklet_enable(&dev->dfs_pd.dfs_tasklet);
}
- dev->beacon_ops->pre_tbtt_enable(dev, true);
+ mt76x02_pre_tbtt_enable(dev, true);
mt76_txq_schedule_all(&dev->mt76);
-
- return ret;
}
int mt76x0_config(struct ieee80211_hw *hw, u32 changed)
{
struct mt76x02_dev *dev = hw->priv;
- int ret = 0;
mutex_lock(&dev->mt76.mutex);
if (changed & IEEE80211_CONF_CHANGE_CHANNEL) {
ieee80211_stop_queues(hw);
- ret = mt76x0_set_channel(dev, &hw->conf.chandef);
+ mt76x0_set_channel(dev, &hw->conf.chandef);
ieee80211_wake_queues(hw);
}
@@ -69,6 +64,6 @@ int mt76x0_config(struct ieee80211_hw *hw, u32 changed)
mutex_unlock(&dev->mt76.mutex);
- return ret;
+ return 0;
}
EXPORT_SYMBOL_GPL(mt76x0_config);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h b/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h
index 97e47cd..26517e0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/mt76x0.h
@@ -54,7 +54,7 @@ int mt76x0_config(struct ieee80211_hw *hw, u32 changed);
/* PHY */
void mt76x0_phy_init(struct mt76x02_dev *dev);
int mt76x0_phy_wait_bbp_ready(struct mt76x02_dev *dev);
-int mt76x0_phy_set_channel(struct mt76x02_dev *dev,
+void mt76x0_phy_set_channel(struct mt76x02_dev *dev,
struct cfg80211_chan_def *chandef);
void mt76x0_phy_set_txpower(struct mt76x02_dev *dev);
void mt76x0_phy_calibrate(struct mt76x02_dev *dev, bool power_on);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c b/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
index 4585e1b..f84a7df 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
@@ -80,7 +69,7 @@ static const struct ieee80211_ops mt76x0e_ops = {
.sta_state = mt76_sta_state,
.set_key = mt76x02_set_key,
.conf_tx = mt76x02_conf_tx,
- .sw_scan_start = mt76x02_sw_scan,
+ .sw_scan_start = mt76_sw_scan,
.sw_scan_complete = mt76x02_sw_scan_complete,
.ampdu_action = mt76x02_ampdu_action,
.sta_rate_tbl_update = mt76x02_sta_rate_tbl_update,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/pci_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x0/pci_mcu.c
index 490c186..038187b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/pci_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/pci_mcu.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
#include <linux/firmware.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/phy.c b/drivers/net/wireless/mediatek/mt76/mt76x0/phy.c
index 1ecfc33..711a352 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/phy.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/phy.c
@@ -109,7 +109,7 @@ mt76x0_rf_wr(struct mt76x02_dev *dev, u32 offset, u8 val)
};
WARN_ON_ONCE(!test_bit(MT76_STATE_MCU_RUNNING,
- &dev->mt76.state));
+ &dev->mt76.state));
return mt76_wr_rp(dev, MT_MCU_MEMMAP_RF, &pair, 1);
} else {
return mt76x0_rf_csr_wr(dev, offset, val);
@@ -127,7 +127,7 @@ static int mt76x0_rf_rr(struct mt76x02_dev *dev, u32 offset)
};
WARN_ON_ONCE(!test_bit(MT76_STATE_MCU_RUNNING,
- &dev->mt76.state));
+ &dev->mt76.state));
ret = mt76_rd_rp(dev, MT_MCU_MEMMAP_RF, &pair, 1);
val = pair.value;
} else {
@@ -230,7 +230,8 @@ mt76x0_phy_set_band(struct mt76x02_dev *dev, enum nl80211_band band)
}
static void
-mt76x0_phy_set_chan_rf_params(struct mt76x02_dev *dev, u8 channel, u16 rf_bw_band)
+mt76x0_phy_set_chan_rf_params(struct mt76x02_dev *dev, u8 channel,
+ u16 rf_bw_band)
{
const struct mt76x0_freq_item *freq_item;
u16 rf_band = rf_bw_band & 0xff00;
@@ -252,9 +253,9 @@ mt76x0_phy_set_chan_rf_params(struct mt76x02_dev *dev, u8 channel, u16 rf_bw_ban
rf_band = mt76x0_frequency_plan[i].band;
if (b_sdm)
- freq_item = &(mt76x0_sdm_frequency_plan[i]);
+ freq_item = &mt76x0_sdm_frequency_plan[i];
else
- freq_item = &(mt76x0_frequency_plan[i]);
+ freq_item = &mt76x0_frequency_plan[i];
mt76x0_rf_wr(dev, MT_RF(0, 37), freq_item->pllR37);
mt76x0_rf_wr(dev, MT_RF(0, 36), freq_item->pllR36);
@@ -359,11 +360,12 @@ mt76x0_phy_set_chan_rf_params(struct mt76x02_dev *dev, u8 channel, u16 rf_bw_ban
band = (rf_band & RF_G_BAND) ? NL80211_BAND_2GHZ : NL80211_BAND_5GHZ;
if (mt76x02_ext_pa_enabled(dev, band)) {
- /*
- MT_RF_MISC (offset: 0x0518)
- [2]1'b1: enable external A band PA, 1'b0: disable external A band PA
- [3]1'b1: enable external G band PA, 1'b0: disable external G band PA
- */
+ /* MT_RF_MISC (offset: 0x0518)
+ * [2]1'b1: enable external A band PA
+ * 1'b0: disable external A band PA
+ * [3]1'b1: enable external G band PA
+ * 1'b0: disable external G band PA
+ */
if (rf_band & RF_A_BAND)
mt76_set(dev, MT_RF_MISC, BIT(2));
else
@@ -385,7 +387,9 @@ mt76x0_phy_set_chan_rf_params(struct mt76x02_dev *dev, u8 channel, u16 rf_bw_ban
mt76_wr(dev, MT_TX_ALC_CFG_1, mac_reg);
} else {
mt76_wr(dev, MT_TX0_RF_GAIN_ATTEN, 0x686A7800);
- /* Set Atten mode = 0 For Ext A band, Disable Tx Inc dcoc Cal. */
+ /* Set Atten mode = 0
+ * For Ext A band, Disable Tx Inc dcoc Cal.
+ */
mac_reg = mt76_rr(dev, MT_TX_ALC_CFG_1);
mac_reg &= 0x890400FF;
mt76_wr(dev, MT_TX_ALC_CFG_1, mac_reg);
@@ -490,7 +494,7 @@ mt76x0_phy_bbp_set_bw(struct mt76x02_dev *dev, enum nl80211_chan_width width)
case NL80211_CHAN_WIDTH_160:
case NL80211_CHAN_WIDTH_5:
/* TODO error */
- return ;
+ return;
}
mt76x02_mcu_function_select(dev, BW_SETTING, bw);
@@ -905,8 +909,8 @@ void mt76x0_phy_calibrate(struct mt76x02_dev *dev, bool power_on)
}
EXPORT_SYMBOL_GPL(mt76x0_phy_calibrate);
-int mt76x0_phy_set_channel(struct mt76x02_dev *dev,
- struct cfg80211_chan_def *chandef)
+void mt76x0_phy_set_channel(struct mt76x02_dev *dev,
+ struct cfg80211_chan_def *chandef)
{
u32 ext_cca_chan[4] = {
[0] = FIELD_PREP(MT_EXT_CCA_CFG_CCA0, 0) |
@@ -940,7 +944,6 @@ int mt76x0_phy_set_channel(struct mt76x02_dev *dev,
freq1 = chandef->center_freq1;
channel = chandef->chan->hw_value;
rf_bw_band = (channel <= 14) ? RF_G_BAND : RF_A_BAND;
- dev->mt76.chandef = *chandef;
switch (chandef->width) {
case NL80211_CHAN_WIDTH_40:
@@ -1001,7 +1004,7 @@ int mt76x0_phy_set_channel(struct mt76x02_dev *dev,
/* enable vco */
mt76x0_rf_set(dev, MT_RF(0, 4), BIT(7));
if (scan)
- return 0;
+ return;
mt76x02_init_agc_gain(dev);
mt76x0_phy_calibrate(dev, false);
@@ -1009,8 +1012,6 @@ int mt76x0_phy_set_channel(struct mt76x02_dev *dev,
ieee80211_queue_delayed_work(dev->mt76.hw, &dev->cal_work,
MT_CALIBRATE_INTERVAL);
-
- return 0;
}
static void mt76x0_phy_temp_sensor(struct mt76x02_dev *dev)
@@ -1074,7 +1075,7 @@ mt76x0_phy_update_channel_gain(struct mt76x02_dev *dev)
dev->cal.avg_rssi_all = -75;
low_gain = (dev->cal.avg_rssi_all > mt76x02_get_rssi_gain_thresh(dev)) +
- (dev->cal.avg_rssi_all > mt76x02_get_low_rssi_gain_thresh(dev));
+ (dev->cal.avg_rssi_all > mt76x02_get_low_rssi_gain_thresh(dev));
gain_change = dev->cal.low_gain < 0 ||
(dev->cal.low_gain & 2) ^ (low_gain & 2);
@@ -1169,7 +1170,8 @@ static void mt76x0_phy_rf_init(struct mt76x02_dev *dev)
if (item->bw_band == RF_BW_20)
mt76x0_rf_wr(dev, item->rf_bank_reg, item->value);
- else if (((RF_G_BAND | RF_BW_20) & item->bw_band) == (RF_G_BAND | RF_BW_20))
+ else if (((RF_G_BAND | RF_BW_20) & item->bw_band) ==
+ (RF_G_BAND | RF_BW_20))
mt76x0_rf_wr(dev, item->rf_bank_reg, item->value);
}
@@ -1181,10 +1183,9 @@ static void mt76x0_phy_rf_init(struct mt76x02_dev *dev)
}
}
- /*
- Frequency calibration
- E1: B0.R22<6:0>: xo_cxo<6:0>
- E2: B0.R21<0>: xo_cxo<0>, B0.R22<7:0>: xo_cxo<8:1>
+ /* Frequency calibration
+ * E1: B0.R22<6:0>: xo_cxo<6:0>
+ * E2: B0.R21<0>: xo_cxo<0>, B0.R22<7:0>: xo_cxo<8:1>
*/
mt76x0_rf_wr(dev, MT_RF(0, 22),
min_t(u8, dev->cal.rx.freq_offset, 0xbf));
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/phy.h b/drivers/net/wireless/mediatek/mt76/mt76x0/phy.h
index b4b2ca7..441d655 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/phy.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/phy.h
@@ -6,8 +6,8 @@
#ifndef _MT76X0_PHY_H_
#define _MT76X0_PHY_H_
-#define RF_G_BAND 0x0100
-#define RF_A_BAND 0x0200
+#define RF_G_BAND 0x0100
+#define RF_A_BAND 0x0200
#define RF_A_BAND_LB 0x0400
#define RF_A_BAND_MB 0x0800
#define RF_A_BAND_HB 0x1000
@@ -18,9 +18,9 @@
#define RF_BW_10 4
#define RF_BW_80 8
-#define MT_RF(bank, reg) ((bank) << 16 | (reg))
-#define MT_RF_BANK(offset) (offset >> 16)
-#define MT_RF_REG(offset) (offset & 0xff)
+#define MT_RF(bank, reg) ((bank) << 16 | (reg))
+#define MT_RF_BANK(offset) ((offset) >> 16)
+#define MT_RF_REG(offset) ((offset) & 0xff)
#define MT_RF_VCO_BP_CLOSE_LOOP BIT(3)
#define MT_RF_VCO_BP_CLOSE_LOOP_MASK GENMASK(3, 0)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c b/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c
index 627ed1f..00a445d 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/usb.c
@@ -32,10 +32,13 @@ static struct usb_device_id mt76x0_device_table[] = {
{ USB_DEVICE(0x20f4, 0x806b) }, /* TRENDnet TEW-806UBH */
{ USB_DEVICE(0x7392, 0xc711) }, /* Devolo Wifi ac Stick */
{ USB_DEVICE(0x0df6, 0x0079) }, /* Sitecom Europe B.V. ac Stick */
- { USB_DEVICE(0x2357, 0x0105),
- .driver_info = 1, }, /* TP-LINK Archer T1U */
- { USB_DEVICE_AND_INTERFACE_INFO(0x0E8D, 0x7630, 0xff, 0x2, 0xff)}, /* MT7630U */
- { USB_DEVICE_AND_INTERFACE_INFO(0x0E8D, 0x7650, 0xff, 0x2, 0xff)}, /* MT7650U */
+ { USB_DEVICE(0x2357, 0x0123) }, /* TP-LINK T2UHP */
+ /* TP-LINK Archer T1U */
+ { USB_DEVICE(0x2357, 0x0105), .driver_info = 1, },
+ /* MT7630U */
+ { USB_DEVICE_AND_INTERFACE_INFO(0x0E8D, 0x7630, 0xff, 0x2, 0xff)},
+ /* MT7650U */
+ { USB_DEVICE_AND_INTERFACE_INFO(0x0E8D, 0x7650, 0xff, 0x2, 0xff)},
{ 0, }
};
@@ -125,22 +128,23 @@ static const struct ieee80211_ops mt76x0u_ops = {
.sta_state = mt76_sta_state,
.set_key = mt76x02_set_key,
.conf_tx = mt76x02_conf_tx,
- .sw_scan_start = mt76x02_sw_scan,
+ .sw_scan_start = mt76_sw_scan,
.sw_scan_complete = mt76x02_sw_scan_complete,
.ampdu_action = mt76x02_ampdu_action,
.sta_rate_tbl_update = mt76x02_sta_rate_tbl_update,
.set_rts_threshold = mt76x02_set_rts_threshold,
.wake_tx_queue = mt76_wake_tx_queue,
.get_txpower = mt76_get_txpower,
+ .get_survey = mt76_get_survey,
.set_tim = mt76_set_tim,
.release_buffered_frames = mt76_release_buffered_frames,
};
-static int mt76x0u_init_hardware(struct mt76x02_dev *dev)
+static int mt76x0u_init_hardware(struct mt76x02_dev *dev, bool reset)
{
int err;
- mt76x0_chip_onoff(dev, true, true);
+ mt76x0_chip_onoff(dev, true, reset);
if (!mt76x02_wait_for_mac(&dev->mt76))
return -ETIMEDOUT;
@@ -161,6 +165,13 @@ static int mt76x0u_init_hardware(struct mt76x02_dev *dev)
FIELD_PREP(MT_TXOP_TRUN_EN, 0x3f) |
FIELD_PREP(MT_TXOP_EXT_CCA_DLY, 0x58));
+ mt76_wr(dev, MT_CH_TIME_CFG,
+ MT_CH_TIME_CFG_TIMER_EN |
+ MT_CH_TIME_CFG_TX_AS_BUSY |
+ MT_CH_TIME_CFG_RX_AS_BUSY |
+ MT_CH_TIME_CFG_NAV_AS_BUSY |
+ MT_CH_TIME_CFG_EIFS_AS_BUSY);
+
return 0;
}
@@ -173,7 +184,7 @@ static int mt76x0u_register_device(struct mt76x02_dev *dev)
if (err < 0)
goto out_err;
- err = mt76x0u_init_hardware(dev);
+ err = mt76x0u_init_hardware(dev, true);
if (err < 0)
goto out_err;
@@ -200,6 +211,7 @@ static int mt76x0u_probe(struct usb_interface *usb_intf,
const struct usb_device_id *id)
{
static const struct mt76_driver_ops drv_ops = {
+ .update_survey = mt76x02_update_channel,
.tx_prepare_skb = mt76x02u_tx_prepare_skb,
.tx_complete_skb = mt76x02u_tx_complete_skb,
.tx_status_data = mt76x02_tx_status_data,
@@ -236,7 +248,7 @@ static int mt76x0u_probe(struct usb_interface *usb_intf,
if (ret)
goto err;
- /* Disable the HW, otherwise MCU fail to initalize on hot reboot */
+ /* Disable the HW, otherwise MCU fail to initialize on hot reboot */
mt76x0_chip_onoff(dev, false, false);
if (!mt76x02_wait_for_mac(mdev)) {
@@ -274,9 +286,9 @@ static int mt76x0u_probe(struct usb_interface *usb_intf,
static void mt76x0_disconnect(struct usb_interface *usb_intf)
{
struct mt76x02_dev *dev = usb_get_intfdata(usb_intf);
- bool initalized = test_bit(MT76_STATE_INITIALIZED, &dev->mt76.state);
+ bool initialized = test_bit(MT76_STATE_INITIALIZED, &dev->mt76.state);
- if (!initalized)
+ if (!initialized)
return;
ieee80211_unregister_hw(dev->mt76.hw);
@@ -309,7 +321,7 @@ static int __maybe_unused mt76x0_resume(struct usb_interface *usb_intf)
if (ret < 0)
goto err;
- ret = mt76x0u_init_hardware(dev);
+ ret = mt76x0u_init_hardware(dev, false);
if (ret)
goto err;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/usb_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x0/usb_mcu.c
index 4a28276..888a930 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/usb_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/usb_mcu.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
#include <linux/firmware.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02.h b/drivers/net/wireless/mediatek/mt76/mt76x02.h
index f7fd53a..e858bba 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02.h
@@ -1,18 +1,7 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Stanislaw Gruszka <stf_xl@wp.pl>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_H
@@ -71,10 +60,15 @@ struct mt76x02_calibration {
struct mt76x02_beacon_ops {
unsigned int nslots;
unsigned int slot_size;
- void (*pre_tbtt_enable) (struct mt76x02_dev *, bool);
- void (*beacon_enable) (struct mt76x02_dev *, bool);
+ void (*pre_tbtt_enable)(struct mt76x02_dev *dev, bool en);
+ void (*beacon_enable)(struct mt76x02_dev *dev, bool en);
};
+#define mt76x02_beacon_enable(dev, enable) \
+ (dev)->beacon_ops->beacon_enable(dev, enable)
+#define mt76x02_pre_tbtt_enable(dev, enable) \
+ (dev)->beacon_ops->pre_tbtt_enable(dev, enable)
+
struct mt76x02_dev {
struct mt76_dev mt76; /* must be first */
@@ -137,8 +131,8 @@ extern struct ieee80211_rate mt76x02_rates[12];
void mt76x02_init_device(struct mt76x02_dev *dev);
void mt76x02_configure_filter(struct ieee80211_hw *hw,
- unsigned int changed_flags,
- unsigned int *total_flags, u64 multicast);
+ unsigned int changed_flags,
+ unsigned int *total_flags, u64 multicast);
int mt76x02_sta_add(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
void mt76x02_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
@@ -147,20 +141,20 @@ void mt76x02_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
void mt76x02_config_mac_addr_list(struct mt76x02_dev *dev);
int mt76x02_add_interface(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif);
+ struct ieee80211_vif *vif);
void mt76x02_remove_interface(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif);
+ struct ieee80211_vif *vif);
int mt76x02_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- struct ieee80211_ampdu_params *params);
+ struct ieee80211_ampdu_params *params);
int mt76x02_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
- struct ieee80211_vif *vif, struct ieee80211_sta *sta,
- struct ieee80211_key_conf *key);
+ struct ieee80211_vif *vif, struct ieee80211_sta *sta,
+ struct ieee80211_key_conf *key);
int mt76x02_conf_tx(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- u16 queue, const struct ieee80211_tx_queue_params *params);
+ u16 queue, const struct ieee80211_tx_queue_params *params);
void mt76x02_sta_rate_tbl_update(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif,
- struct ieee80211_sta *sta);
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta);
s8 mt76x02_tx_get_max_txpwr_adj(struct mt76x02_dev *dev,
const struct ieee80211_tx_rate *rate);
s8 mt76x02_tx_get_txpwr_adj(struct mt76x02_dev *dev, s8 txpwr,
@@ -183,8 +177,6 @@ int mt76x02_tx_prepare_skb(struct mt76_dev *mdev, void *txwi,
enum mt76_txq_id qid, struct mt76_wcid *wcid,
struct ieee80211_sta *sta,
struct mt76_tx_info *tx_info);
-void mt76x02_sw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- const u8 *mac);
void mt76x02_sw_scan_complete(struct ieee80211_hw *hw,
struct ieee80211_vif *vif);
void mt76x02_sta_ps(struct mt76_dev *dev, struct ieee80211_sta *sta, bool ps);
@@ -197,6 +189,7 @@ struct beacon_bc_data {
struct sk_buff_head q;
struct sk_buff *tail[8];
};
+
void mt76x02_init_beacon_config(struct mt76x02_dev *dev);
void mt76x02e_init_beacon_config(struct mt76x02_dev *dev);
void mt76x02_resync_beacon_timer(struct mt76x02_dev *dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_beacon.c b/drivers/net/wireless/mediatek/mt76/mt76x02_beacon.c
index d61c686..92305bd 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_beacon.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_beacon.c
@@ -1,19 +1,8 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
* Copyright (C) 2018 Stanislaw Gruszka <stf_xl@wp.pl>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x02.h"
@@ -115,53 +104,41 @@ int mt76x02_mac_set_beacon(struct mt76x02_dev *dev, u8 vif_idx,
}
EXPORT_SYMBOL_GPL(mt76x02_mac_set_beacon);
-static void
-__mt76x02_mac_set_beacon_enable(struct mt76x02_dev *dev, u8 vif_idx,
- bool val, struct sk_buff *skb)
-{
- u8 old_mask = dev->mt76.beacon_mask;
- bool en;
- u32 reg;
-
- if (val) {
- dev->mt76.beacon_mask |= BIT(vif_idx);
- if (skb)
- mt76x02_mac_set_beacon(dev, vif_idx, skb);
- } else {
- dev->mt76.beacon_mask &= ~BIT(vif_idx);
- mt76x02_mac_set_beacon(dev, vif_idx, NULL);
- }
-
- if (!!old_mask == !!dev->mt76.beacon_mask)
- return;
-
- en = dev->mt76.beacon_mask;
-
- reg = MT_BEACON_TIME_CFG_BEACON_TX |
- MT_BEACON_TIME_CFG_TBTT_EN |
- MT_BEACON_TIME_CFG_TIMER_EN;
- mt76_rmw(dev, MT_BEACON_TIME_CFG, reg, reg * en);
-
- dev->beacon_ops->beacon_enable(dev, en);
-}
-
void mt76x02_mac_set_beacon_enable(struct mt76x02_dev *dev,
- struct ieee80211_vif *vif, bool val)
+ struct ieee80211_vif *vif, bool enable)
{
- u8 vif_idx = ((struct mt76x02_vif *)vif->drv_priv)->idx;
- struct sk_buff *skb = NULL;
+ struct mt76x02_vif *mvif = (struct mt76x02_vif *)vif->drv_priv;
+ u8 old_mask = dev->mt76.beacon_mask;
- dev->beacon_ops->pre_tbtt_enable(dev, false);
-
- if (mt76_is_usb(dev))
- skb = ieee80211_beacon_get(mt76_hw(dev), vif);
+ mt76x02_pre_tbtt_enable(dev, false);
if (!dev->mt76.beacon_mask)
dev->tbtt_count = 0;
- __mt76x02_mac_set_beacon_enable(dev, vif_idx, val, skb);
+ if (enable) {
+ dev->mt76.beacon_mask |= BIT(mvif->idx);
+ } else {
+ dev->mt76.beacon_mask &= ~BIT(mvif->idx);
+ mt76x02_mac_set_beacon(dev, mvif->idx, NULL);
+ }
- dev->beacon_ops->pre_tbtt_enable(dev, true);
+ if (!!old_mask == !!dev->mt76.beacon_mask)
+ goto out;
+
+ if (dev->mt76.beacon_mask)
+ mt76_set(dev, MT_BEACON_TIME_CFG,
+ MT_BEACON_TIME_CFG_BEACON_TX |
+ MT_BEACON_TIME_CFG_TBTT_EN |
+ MT_BEACON_TIME_CFG_TIMER_EN);
+ else
+ mt76_clear(dev, MT_BEACON_TIME_CFG,
+ MT_BEACON_TIME_CFG_BEACON_TX |
+ MT_BEACON_TIME_CFG_TBTT_EN |
+ MT_BEACON_TIME_CFG_TIMER_EN);
+ mt76x02_beacon_enable(dev, !!dev->mt76.beacon_mask);
+
+out:
+ mt76x02_pre_tbtt_enable(dev, true);
}
void
@@ -237,7 +214,8 @@ mt76x02_add_buffered_bc(void *priv, u8 *mac, struct ieee80211_vif *vif)
}
void
-mt76x02_enqueue_buffered_bc(struct mt76x02_dev *dev, struct beacon_bc_data *data,
+mt76x02_enqueue_buffered_bc(struct mt76x02_dev *dev,
+ struct beacon_bc_data *data,
int max_nframes)
{
int i, nframes;
@@ -281,4 +259,3 @@ void mt76x02_init_beacon_config(struct mt76x02_dev *dev)
}
EXPORT_SYMBOL_GPL(mt76x02_init_beacon_config);
-
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_debugfs.c b/drivers/net/wireless/mediatek/mt76/mt76x02_debugfs.c
index 1b1e424..0cb2a7b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_debugfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_debugfs.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/debugfs.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.c b/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.c
index 50e9b31..5dec33e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x02.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.h b/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.h
index 0408613..491010a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_dfs.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_DFS_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_dma.h b/drivers/net/wireless/mediatek/mt76/mt76x02_dma.h
index 6394010..4aff4f8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_dma.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_dma.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_DMA_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c b/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c
index 07f0496..c54c50f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <asm/unaligned.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.h b/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.h
index 0ba536d..99941a4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_eeprom.h
@@ -1,18 +1,7 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_EEPROM_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
index 82bafb5..abacb4e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Stanislaw Gruszka <stf_xl@wp.pl>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x02.h"
@@ -92,7 +81,6 @@ void mt76x02_mac_wcid_sync_pn(struct mt76x02_dev *dev, u8 idx,
atomic64_set(&key->tx_pn, pn);
}
-
int mt76x02_mac_wcid_set_key(struct mt76x02_dev *dev, u8 idx,
struct ieee80211_key_conf *key)
{
@@ -267,7 +255,7 @@ bool mt76x02_mac_load_tx_status(struct mt76x02_dev *dev,
static int
mt76x02_mac_process_tx_rate(struct ieee80211_tx_rate *txrate, u16 rate,
- enum nl80211_band band)
+ enum nl80211_band band)
{
u8 idx = FIELD_GET(MT_RXWI_RATE_INDEX, rate);
@@ -343,7 +331,7 @@ void mt76x02_mac_write_txwi(struct mt76x02_dev *dev, struct mt76x02_txwi *txwi,
ieee80211_has_protected(hdr->frame_control)) {
wcid = NULL;
ieee80211_get_tx_rates(info->control.vif, sta, skb,
- info->control.rates, 1);
+ info->control.rates, 1);
}
if (wcid)
@@ -353,6 +341,7 @@ void mt76x02_mac_write_txwi(struct mt76x02_dev *dev, struct mt76x02_txwi *txwi,
if (wcid && wcid->sw_iv && key) {
u64 pn = atomic64_inc_return(&key->tx_pn);
+
ccmp_pn[0] = pn;
ccmp_pn[1] = pn >> 8;
ccmp_pn[2] = 0;
@@ -445,8 +434,8 @@ mt76x02_tx_rate_fallback(struct ieee80211_tx_rate *rates, int idx, int phy)
case MT_PHY_TYPE_HT:
/* MCS 8 falls back to MCS 0 */
if (rates[0].idx == 8) {
- rates[1].idx = 0;
- break;
+ rates[1].idx = 0;
+ break;
}
/* fall through */
default:
@@ -568,9 +557,9 @@ void mt76x02_send_tx_status(struct mt76x02_dev *dev,
u32 stat_val, stat_cache;
stat_val = stat->rate;
- stat_val |= ((u32) stat->retry) << 16;
+ stat_val |= ((u32)stat->retry) << 16;
stat_cache = msta->status.rate;
- stat_cache |= ((u32) msta->status.retry) << 16;
+ stat_cache |= ((u32)msta->status.retry) << 16;
if (*update == 0 && stat_val == stat_cache &&
stat->wcid == msta->status.wcid && msta->n_frames < 32) {
@@ -718,7 +707,7 @@ mt76x02_mac_get_rssi(struct mt76x02_dev *dev, s8 rssi, int chain)
int mt76x02_mac_process_rx(struct mt76x02_dev *dev, struct sk_buff *skb,
void *rxi)
{
- struct mt76_rx_status *status = (struct mt76_rx_status *) skb->cb;
+ struct mt76_rx_status *status = (struct mt76_rx_status *)skb->cb;
struct mt76x02_rxwi *rxwi = rxi;
struct mt76x02_sta *sta;
u32 rxinfo = le32_to_cpu(rxwi->rxinfo);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h
index cb39da7..efa4ef9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h
@@ -1,18 +1,7 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Stanislaw Gruszka <stf_xl@wp.pl>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76X02_MAC_H
@@ -207,7 +196,7 @@ void mt76x02_mac_set_bssid(struct mt76x02_dev *dev, u8 idx, const u8 *addr);
int mt76x02_mac_set_beacon(struct mt76x02_dev *dev, u8 vif_idx,
struct sk_buff *skb);
void mt76x02_mac_set_beacon_enable(struct mt76x02_dev *dev,
- struct ieee80211_vif *vif, bool val);
+ struct ieee80211_vif *vif, bool enable);
void mt76x02_edcca_init(struct mt76x02_dev *dev);
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.c
index 6501b85..4be7a24 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
@@ -65,7 +54,7 @@ int mt76x02_mcu_msg_send(struct mt76_dev *mdev, int cmd, const void *data,
break;
}
- rxfce = (u32 *) skb->cb;
+ rxfce = (u32 *)skb->cb;
if (seq == FIELD_GET(MT_RX_FCE_INFO_CMD_SEQ, *rxfce))
check_seq = true;
@@ -86,11 +75,11 @@ int mt76x02_mcu_function_select(struct mt76x02_dev *dev, enum mcu_function func,
u32 val)
{
struct {
- __le32 id;
- __le32 value;
+ __le32 id;
+ __le32 value;
} __packed __aligned(4) msg = {
- .id = cpu_to_le32(func),
- .value = cpu_to_le32(val),
+ .id = cpu_to_le32(func),
+ .value = cpu_to_le32(val),
};
bool wait = false;
@@ -111,7 +100,8 @@ int mt76x02_mcu_set_radio_state(struct mt76x02_dev *dev, bool on)
.level = cpu_to_le32(0),
};
- return mt76_mcu_send_msg(dev, CMD_POWER_SAVING_OP, &msg, sizeof(msg), false);
+ return mt76_mcu_send_msg(dev, CMD_POWER_SAVING_OP, &msg, sizeof(msg),
+ false);
}
EXPORT_SYMBOL_GPL(mt76x02_mcu_set_radio_state);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.h b/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.h
index a7b0d3e..c81a965 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mcu.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_MCU_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c
index 467b283..dc77307 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
@@ -97,7 +86,8 @@ void mt76x02e_init_beacon_config(struct mt76x02_dev *dev)
dev->beacon_ops = &beacon_ops;
/* Fire a pre-TBTT interrupt 8 ms before TBTT */
- mt76_rmw_field(dev, MT_INT_TIMER_CFG, MT_INT_TIMER_CFG_PRE_TBTT, 8 << 4);
+ mt76_rmw_field(dev, MT_INT_TIMER_CFG, MT_INT_TIMER_CFG_PRE_TBTT,
+ 8 << 4);
mt76_rmw_field(dev, MT_INT_TIMER_CFG, MT_INT_TIMER_CFG_GP_TIMER,
MT_DFS_GP_INTERVAL);
mt76_wr(dev, MT_INT_TIMER_EN, 0);
@@ -201,7 +191,7 @@ int mt76x02_dma_init(struct mt76x02_dev *dev)
return -ENOMEM;
tasklet_init(&dev->mt76.tx_tasklet, mt76x02_tx_tasklet,
- (unsigned long) dev);
+ (unsigned long)dev);
tasklet_init(&dev->mt76.pre_tbtt_tasklet, mt76x02_pre_tbtt_tasklet,
(unsigned long)dev);
@@ -395,12 +385,12 @@ static void mt76x02_key_sync(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
struct mt76_wcid *wcid;
if (!sta)
- return;
+ return;
- wcid = (struct mt76_wcid *) sta->drv_priv;
+ wcid = (struct mt76_wcid *)sta->drv_priv;
if (wcid->hw_key_idx != key->keyidx || wcid->sw_iv)
- return;
+ return;
mt76x02_mac_wcid_sync_pn(dev, wcid->idx, key);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_phy.c b/drivers/net/wireless/mediatek/mt76/mt76x02_phy.c
index a54b63a..d733426 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_phy.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_phy.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
@@ -183,7 +172,8 @@ bool mt76x02_phy_adjust_vga_gain(struct mt76x02_dev *dev)
bool ret = false;
u32 false_cca;
- false_cca = FIELD_GET(MT_RX_STAT_1_CCA_ERRORS, mt76_rr(dev, MT_RX_STAT_1));
+ false_cca = FIELD_GET(MT_RX_STAT_1_CCA_ERRORS,
+ mt76_rr(dev, MT_RX_STAT_1));
dev->cal.false_cca = false_cca;
if (false_cca > 800 && dev->cal.agc_gain_adjust < limit) {
dev->cal.agc_gain_adjust += 2;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_phy.h b/drivers/net/wireless/mediatek/mt76/mt76x02_phy.h
index d2971db..fc2e410 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_phy.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_phy.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_PHY_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_regs.h b/drivers/net/wireless/mediatek/mt76/mt76x02_regs.h
index ea78339..21c0f35 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_regs.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_regs.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76X02_REGS_H
@@ -19,8 +8,8 @@
#define MT_ASIC_VERSION 0x0000
-#define MT76XX_REV_E3 0x22
-#define MT76XX_REV_E4 0x33
+#define MT76XX_REV_E3 0x22
+#define MT76XX_REV_E4 0x33
#define MT_CMB_CTRL 0x0020
#define MT_CMB_CTRL_XTAL_RDY BIT(22)
@@ -120,7 +109,7 @@
#define MT_INT_RX_DONE(_n) BIT(_n)
#define MT_INT_RX_DONE_ALL GENMASK(1, 0)
#define MT_INT_TX_DONE_ALL GENMASK(13, 4)
-#define MT_INT_TX_DONE(_n) BIT(_n + 4)
+#define MT_INT_TX_DONE(_n) BIT((_n) + 4)
#define MT_INT_RX_COHERENT BIT(16)
#define MT_INT_TX_COHERENT BIT(17)
#define MT_INT_ANY_COHERENT BIT(18)
@@ -149,21 +138,21 @@
#define MT_WPDMA_DELAY_INT_CFG 0x0210
-#define MT_WMM_AIFSN 0x0214
+#define MT_WMM_AIFSN 0x0214
#define MT_WMM_AIFSN_MASK GENMASK(3, 0)
#define MT_WMM_AIFSN_SHIFT(_n) ((_n) * 4)
-#define MT_WMM_CWMIN 0x0218
+#define MT_WMM_CWMIN 0x0218
#define MT_WMM_CWMIN_MASK GENMASK(3, 0)
#define MT_WMM_CWMIN_SHIFT(_n) ((_n) * 4)
-#define MT_WMM_CWMAX 0x021c
+#define MT_WMM_CWMAX 0x021c
#define MT_WMM_CWMAX_MASK GENMASK(3, 0)
#define MT_WMM_CWMAX_SHIFT(_n) ((_n) * 4)
#define MT_WMM_TXOP_BASE 0x0220
#define MT_WMM_TXOP(_n) (MT_WMM_TXOP_BASE + (((_n) / 2) << 2))
-#define MT_WMM_TXOP_SHIFT(_n) ((_n & 1) * 16)
+#define MT_WMM_TXOP_SHIFT(_n) (((_n) & 1) * 16)
#define MT_WMM_TXOP_MASK GENMASK(15, 0)
#define MT_WMM_CTRL 0x0230 /* MT76x0 */
@@ -607,7 +596,7 @@
#define MT_TX_AGG_CNT(_id) ((_id) < 8 ? \
MT_TX_AGG_CNT_BASE0 + ((_id) << 2) : \
- MT_TX_AGG_CNT_BASE1 + ((_id - 8) << 2))
+ MT_TX_AGG_CNT_BASE1 + (((_id) - 8) << 2))
#define MT_TX_STAT_FIFO_EXT 0x1798
#define MT_TX_STAT_FIFO_EXT_RETRY GENMASK(7, 0)
@@ -680,17 +669,17 @@
#define MT_SKEY_BASE_0 0xac00
#define MT_SKEY_BASE_1 0xb400
-#define MT_SKEY_0(_bss, _idx) (MT_SKEY_BASE_0 + (4 * (_bss) + _idx) * 32)
-#define MT_SKEY_1(_bss, _idx) (MT_SKEY_BASE_1 + (4 * ((_bss) & 7) + _idx) * 32)
-#define MT_SKEY(_bss, _idx) ((_bss & 8) ? MT_SKEY_1(_bss, _idx) : MT_SKEY_0(_bss, _idx))
+#define MT_SKEY_0(_bss, _idx) (MT_SKEY_BASE_0 + (4 * (_bss) + (_idx)) * 32)
+#define MT_SKEY_1(_bss, _idx) (MT_SKEY_BASE_1 + (4 * ((_bss) & 7) + (_idx)) * 32)
+#define MT_SKEY(_bss, _idx) (((_bss) & 8) ? MT_SKEY_1(_bss, _idx) : MT_SKEY_0(_bss, _idx))
#define MT_SKEY_MODE_BASE_0 0xb000
#define MT_SKEY_MODE_BASE_1 0xb3f0
-#define MT_SKEY_MODE_0(_bss) (MT_SKEY_MODE_BASE_0 + ((_bss / 2) << 2))
+#define MT_SKEY_MODE_0(_bss) (MT_SKEY_MODE_BASE_0 + (((_bss) / 2) << 2))
#define MT_SKEY_MODE_1(_bss) (MT_SKEY_MODE_BASE_1 + ((((_bss) & 7) / 2) << 2))
-#define MT_SKEY_MODE(_bss) ((_bss & 8) ? MT_SKEY_MODE_1(_bss) : MT_SKEY_MODE_0(_bss))
+#define MT_SKEY_MODE(_bss) (((_bss) & 8) ? MT_SKEY_MODE_1(_bss) : MT_SKEY_MODE_0(_bss))
#define MT_SKEY_MODE_MASK GENMASK(3, 0)
-#define MT_SKEY_MODE_SHIFT(_bss, _idx) (4 * ((_idx) + 4 * (_bss & 1)))
+#define MT_SKEY_MODE_SHIFT(_bss, _idx) (4 * ((_idx) + 4 * ((_bss) & 1)))
#define MT_BEACON_BASE 0xc000
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_trace.c b/drivers/net/wireless/mediatek/mt76/mt76x02_trace.c
index 5b42d2c..a812c3a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_trace.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_trace.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_trace.h b/drivers/net/wireless/mediatek/mt76/mt76x02_trace.h
index 713f12d..61ecaf0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_trace.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_trace.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#if !defined(__MT76x02_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
@@ -25,7 +14,8 @@
#define MAXNAME 32
#define DEV_ENTRY __array(char, wiphy_name, 32)
-#define DEV_ASSIGN strlcpy(__entry->wiphy_name, wiphy_name(mt76_hw(dev)->wiphy), MAXNAME)
+#define DEV_ASSIGN strlcpy(__entry->wiphy_name, \
+ wiphy_name(mt76_hw(dev)->wiphy), MAXNAME)
#define DEV_PR_FMT "%s"
#define DEV_PR_ARG __entry->wiphy_name
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_txrx.c b/drivers/net/wireless/mediatek/mt76/mt76x02_txrx.c
index 04118f0..f27aade 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_txrx.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_txrx.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_usb.h b/drivers/net/wireless/mediatek/mt76/mt76x02_usb.h
index 7b53f9e..98329de 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_usb.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_usb.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x02_USB_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_core.c b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_core.c
index e4332d5..78dfc1e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_core.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_core.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x02_usb.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c
index 0cb8751..a993cd7 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
index ad53234..aec73a0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Stanislaw Gruszka <stf_xl@wp.pl>
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
@@ -21,14 +10,14 @@
#define CCK_RATE(_idx, _rate) { \
.bitrate = _rate, \
.flags = IEEE80211_RATE_SHORT_PREAMBLE, \
- .hw_value = (MT_PHY_TYPE_CCK << 8) | _idx, \
- .hw_value_short = (MT_PHY_TYPE_CCK << 8) | (8 + _idx), \
+ .hw_value = (MT_PHY_TYPE_CCK << 8) | (_idx), \
+ .hw_value_short = (MT_PHY_TYPE_CCK << 8) | (8 + (_idx)), \
}
#define OFDM_RATE(_idx, _rate) { \
.bitrate = _rate, \
- .hw_value = (MT_PHY_TYPE_OFDM << 8) | _idx, \
- .hw_value_short = (MT_PHY_TYPE_OFDM << 8) | _idx, \
+ .hw_value = (MT_PHY_TYPE_OFDM << 8) | (_idx), \
+ .hw_value_short = (MT_PHY_TYPE_OFDM << 8) | (_idx), \
}
struct ieee80211_rate mt76x02_rates[] = {
@@ -61,6 +50,20 @@ static const struct ieee80211_iface_limit mt76x02_if_limits[] = {
},
};
+static const struct ieee80211_iface_limit mt76x02u_if_limits[] = {
+ {
+ .max = 1,
+ .types = BIT(NL80211_IFTYPE_ADHOC)
+ }, {
+ .max = 2,
+ .types = BIT(NL80211_IFTYPE_STATION) |
+#ifdef CONFIG_MAC80211_MESH
+ BIT(NL80211_IFTYPE_MESH_POINT) |
+#endif
+ BIT(NL80211_IFTYPE_AP)
+ },
+};
+
static const struct ieee80211_iface_combination mt76x02_if_comb[] = {
{
.limits = mt76x02_if_limits,
@@ -75,6 +78,16 @@ static const struct ieee80211_iface_combination mt76x02_if_comb[] = {
}
};
+static const struct ieee80211_iface_combination mt76x02u_if_comb[] = {
+ {
+ .limits = mt76x02u_if_limits,
+ .n_limits = ARRAY_SIZE(mt76x02u_if_limits),
+ .max_interfaces = 2,
+ .num_different_channels = 1,
+ .beacon_int_infra_match = true,
+ }
+};
+
static void
mt76x02_led_set_config(struct mt76_dev *mdev, u8 delay_on,
u8 delay_off)
@@ -151,6 +164,8 @@ void mt76x02_init_device(struct mt76x02_dev *dev)
if (mt76_is_usb(dev)) {
hw->extra_tx_headroom += sizeof(struct mt76x02_txwi) +
MT_DMA_HDR_LEN;
+ wiphy->iface_combinations = mt76x02u_if_comb;
+ wiphy->n_iface_combinations = ARRAY_SIZE(mt76x02u_if_comb);
} else {
INIT_DELAYED_WORK(&dev->wdt_work, mt76x02_wdt_work);
@@ -281,7 +296,7 @@ mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
mvif->idx = idx;
mvif->group_wcid.idx = MT_VIF_WCID(idx);
mvif->group_wcid.hw_key_idx = -1;
- mtxq = (struct mt76_txq *) vif->txq->drv_priv;
+ mtxq = (struct mt76_txq *)vif->txq->drv_priv;
mtxq->wcid = &mvif->group_wcid;
mt76_txq_init(&dev->mt76, vif->txq);
@@ -345,10 +360,10 @@ int mt76x02_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
enum ieee80211_ampdu_mlme_action action = params->action;
struct ieee80211_sta *sta = params->sta;
struct mt76x02_dev *dev = hw->priv;
- struct mt76x02_sta *msta = (struct mt76x02_sta *) sta->drv_priv;
+ struct mt76x02_sta *msta = (struct mt76x02_sta *)sta->drv_priv;
struct ieee80211_txq *txq = sta->txq[params->tid];
u16 tid = params->tid;
- u16 *ssn = ¶ms->ssn;
+ u16 ssn = params->ssn;
struct mt76_txq *mtxq;
if (!txq)
@@ -359,7 +374,7 @@ int mt76x02_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
switch (action) {
case IEEE80211_AMPDU_RX_START:
mt76_rx_aggr_start(&dev->mt76, &msta->wcid, tid,
- *ssn, params->buf_size);
+ ssn, params->buf_size);
mt76_set(dev, MT_WCID_ADDR(msta->wcid.idx) + 4, BIT(16 + tid));
break;
case IEEE80211_AMPDU_RX_STOP:
@@ -375,10 +390,9 @@ int mt76x02_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
case IEEE80211_AMPDU_TX_STOP_FLUSH:
case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
mtxq->aggr = false;
- ieee80211_send_bar(vif, sta->addr, tid, mtxq->agg_ssn);
break;
case IEEE80211_AMPDU_TX_START:
- mtxq->agg_ssn = IEEE80211_SN_TO_SEQ(*ssn);
+ mtxq->agg_ssn = IEEE80211_SN_TO_SEQ(ssn);
ieee80211_start_tx_ba_cb_irqsafe(vif, sta->addr, tid);
break;
case IEEE80211_AMPDU_TX_STOP_CONT:
@@ -434,7 +448,7 @@ int mt76x02_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
!(key->flags & IEEE80211_KEY_FLAG_PAIRWISE))
return -EOPNOTSUPP;
- msta = sta ? (struct mt76x02_sta *) sta->drv_priv : NULL;
+ msta = sta ? (struct mt76x02_sta *)sta->drv_priv : NULL;
wcid = msta ? &msta->wcid : &mvif->group_wcid;
if (cmd == SET_KEY) {
@@ -558,11 +572,11 @@ int mt76x02_set_rts_threshold(struct ieee80211_hw *hw, u32 val)
EXPORT_SYMBOL_GPL(mt76x02_set_rts_threshold);
void mt76x02_sta_rate_tbl_update(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif,
- struct ieee80211_sta *sta)
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta)
{
struct mt76x02_dev *dev = hw->priv;
- struct mt76x02_sta *msta = (struct mt76x02_sta *) sta->drv_priv;
+ struct mt76x02_sta *msta = (struct mt76x02_sta *)sta->drv_priv;
struct ieee80211_sta_rates *rates = rcu_dereference(sta->rates);
struct ieee80211_tx_rate rate = {};
@@ -588,15 +602,6 @@ void mt76x02_remove_hdr_pad(struct sk_buff *skb, int len)
}
EXPORT_SYMBOL_GPL(mt76x02_remove_hdr_pad);
-void mt76x02_sw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- const u8 *mac)
-{
- struct mt76x02_dev *dev = hw->priv;
-
- set_bit(MT76_SCANNING, &dev->mt76.state);
-}
-EXPORT_SYMBOL_GPL(mt76x02_sw_scan);
-
void mt76x02_sw_scan_complete(struct ieee80211_hw *hw,
struct ieee80211_vif *vif)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/Kconfig b/drivers/net/wireless/mediatek/mt76/mt76x2/Kconfig
index 1f69908..5fd4973 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/Kconfig
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/Kconfig
@@ -8,8 +8,12 @@
select MT76x2_COMMON
depends on MAC80211
depends on PCI
- ---help---
- This adds support for MT7612/MT7602/MT7662-based wireless PCIe devices.
+ help
+ This adds support for MT7612/MT7602/MT7662-based wireless PCIe
+ devices, which comply with IEEE 802.11ac standards and support
+ 2SS to 866Mbit/s PHY rate.
+
+ To compile this driver as a module, choose M here.
config MT76x2U
tristate "MediaTek MT76x2U (USB) support"
@@ -18,4 +22,8 @@
depends on MAC80211
depends on USB
help
- This adds support for MT7612U-based wireless USB dongles.
+ This adds support for MT7612U-based wireless USB 3.0 dongles,
+ which comply with IEEE 802.11ac standards and support 2SS to
+ 866Mbit/s PHY rate.
+
+ To compile this driver as a module, choose M here.
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c
index 6f69985..9f91556 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
@@ -33,7 +22,7 @@ mt76x2_eeprom_get_macaddr(struct mt76x02_dev *dev)
static bool
mt76x2_has_cal_free_data(struct mt76x02_dev *dev, u8 *efuse)
{
- u16 *efuse_w = (u16 *) efuse;
+ u16 *efuse_w = (u16 *)efuse;
if (efuse_w[MT_EE_NIC_CONF_0] != 0)
return false;
@@ -372,7 +361,8 @@ mt76x2_get_power_info_2g(struct mt76x02_dev *dev,
t->chain[chain].tssi_slope = data[0];
t->chain[chain].tssi_offset = data[1];
t->chain[chain].target_power = data[2];
- t->chain[chain].delta = mt76x02_sign_extend_optional(data[delta_idx], 7);
+ t->chain[chain].delta =
+ mt76x02_sign_extend_optional(data[delta_idx], 7);
val = mt76x02_eeprom_get(dev, MT_EE_RF_2G_TSSI_OFF_TXPOWER);
t->target_power = val >> 8;
@@ -381,7 +371,7 @@ mt76x2_get_power_info_2g(struct mt76x02_dev *dev,
static void
mt76x2_get_power_info_5g(struct mt76x02_dev *dev,
struct mt76x2_tx_power_info *t,
- struct ieee80211_channel *chan,
+ struct ieee80211_channel *chan,
int chain, int offset)
{
int channel = chan->hw_value;
@@ -423,7 +413,8 @@ mt76x2_get_power_info_5g(struct mt76x02_dev *dev,
t->chain[chain].tssi_slope = data[0];
t->chain[chain].tssi_offset = data[1];
t->chain[chain].target_power = data[2];
- t->chain[chain].delta = mt76x02_sign_extend_optional(data[delta_idx], 7);
+ t->chain[chain].delta =
+ mt76x02_sign_extend_optional(data[delta_idx], 7);
val = mt76x02_eeprom_get(dev, MT_EE_RF_2G_RX_HIGH_GAIN);
t->target_power = val & 0xff;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.h b/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.h
index 9e73552..4dcf651 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/eeprom.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x2_EEPROM_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/init.c
index 97c3543..79e583e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/init.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mac.c b/drivers/net/wireless/mediatek/mt76/mt76x2/mac.c
index e99d4c9..e08740c 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mac.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mac.h b/drivers/net/wireless/mediatek/mt76/mt76x2/mac.h
index 42ff221..a158302 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mac.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x2_MAC_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.c
index cd3e082..76d8cd3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.h b/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.h
index 40ef439..41fd665 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mcu.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x2_MCU_H
@@ -71,7 +60,8 @@ struct mt76x2_tssi_comp {
u8 offset1;
} __packed __aligned(4);
-int mt76x2_mcu_tssi_comp(struct mt76x02_dev *dev, struct mt76x2_tssi_comp *tssi_data);
+int mt76x2_mcu_tssi_comp(struct mt76x02_dev *dev,
+ struct mt76x2_tssi_comp *tssi_data);
int mt76x2_mcu_init_gain(struct mt76x02_dev *dev, u8 channel, u32 gain,
bool force);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h b/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h
index d7abe3d..41680c4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x2_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2u.h b/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2u.h
index 76cb1f84..c876bac4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2u.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/mt76x2u.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#ifndef __MT76x2U_H
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci.c
index e84d5c5..73c3104 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
index 71aea28..343127f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/delay.h>
@@ -336,4 +325,3 @@ int mt76x2_register_device(struct mt76x02_dev *dev)
return ret;
}
-
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
index 3a14673..4971685aa 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2.h"
@@ -176,7 +165,7 @@ const struct ieee80211_ops mt76x2_ops = {
.sta_state = mt76_sta_state,
.set_key = mt76x02_set_key,
.conf_tx = mt76x02_conf_tx,
- .sw_scan_start = mt76x02_sw_scan,
+ .sw_scan_start = mt76_sw_scan,
.sw_scan_complete = mt76x02_sw_scan_complete,
.flush = mt76x2_flush,
.ampdu_action = mt76x02_ampdu_action,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_mcu.c
index 605dc66..ca6f968 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_mcu.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
@@ -66,7 +55,7 @@ mt76pci_load_rom_patch(struct mt76x02_dev *dev)
mt76_wr(dev, MT_MCU_PCIE_REMAP_BASE4, MT_MCU_ROM_PATCH_OFFSET);
- cur = (__le32 *) (fw->data + sizeof(*hdr));
+ cur = (__le32 *)(fw->data + sizeof(*hdr));
len = fw->size - sizeof(*hdr);
mt76_wr_copy(dev, MT_MCU_ROM_PATCH_ADDR, cur, len);
@@ -121,7 +110,7 @@ mt76pci_load_firmware(struct mt76x02_dev *dev)
dev_info(dev->mt76.dev, "Build: %x\n", val);
dev_info(dev->mt76.dev, "Build Time: %.16s\n", hdr->build_time);
- cur = (__le32 *) (fw->data + sizeof(*hdr));
+ cur = (__le32 *)(fw->data + sizeof(*hdr));
len = le32_to_cpu(hdr->ilm_len);
mt76_wr(dev, MT_MCU_PCIE_REMAP_BASE4, MT_MCU_ILM_OFFSET);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_phy.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_phy.c
index 2edf1bd..23f35bf 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_phy.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_phy.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/delay.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/phy.c b/drivers/net/wireless/mediatek/mt76/mt76x2/phy.c
index cdedf95..edbab4f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/phy.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/phy.c
@@ -1,18 +1,7 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2.h"
@@ -25,7 +14,8 @@ mt76x2_adjust_high_lna_gain(struct mt76x02_dev *dev, int reg, s8 offset)
{
s8 gain;
- gain = FIELD_GET(MT_BBP_AGC_LNA_HIGH_GAIN, mt76_rr(dev, MT_BBP(AGC, reg)));
+ gain = FIELD_GET(MT_BBP_AGC_LNA_HIGH_GAIN,
+ mt76_rr(dev, MT_BBP(AGC, reg)));
gain -= offset / 2;
mt76_rmw_field(dev, MT_BBP(AGC, reg), MT_BBP_AGC_LNA_HIGH_GAIN, gain);
}
@@ -295,7 +285,7 @@ void mt76x2_phy_update_channel_gain(struct mt76x02_dev *dev)
dev->cal.avg_rssi_all = -75;
low_gain = (dev->cal.avg_rssi_all > mt76x02_get_rssi_gain_thresh(dev)) +
- (dev->cal.avg_rssi_all > mt76x02_get_low_rssi_gain_thresh(dev));
+ (dev->cal.avg_rssi_all > mt76x02_get_low_rssi_gain_thresh(dev));
gain_change = dev->cal.low_gain < 0 ||
(dev->cal.low_gain & 2) ^ (low_gain & 2);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c
index 7a994a7..da5e0f9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/kernel.h>
@@ -36,6 +25,7 @@ static int mt76x2u_probe(struct usb_interface *intf,
const struct usb_device_id *id)
{
static const struct mt76_driver_ops drv_ops = {
+ .update_survey = mt76x02_update_channel,
.tx_prepare_skb = mt76x02u_tx_prepare_skb,
.tx_complete_skb = mt76x02u_tx_complete_skb,
.tx_status_data = mt76x02_tx_status_data,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c
index 94f52f9..e305b37 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_init.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/delay.h>
@@ -195,6 +184,13 @@ int mt76x2u_init_hardware(struct mt76x02_dev *dev)
mt76x02_phy_set_rxpath(dev);
mt76x02_phy_set_txdac(dev);
+ mt76_wr(dev, MT_CH_TIME_CFG,
+ MT_CH_TIME_CFG_TIMER_EN |
+ MT_CH_TIME_CFG_TX_AS_BUSY |
+ MT_CH_TIME_CFG_RX_AS_BUSY |
+ MT_CH_TIME_CFG_NAV_AS_BUSY |
+ MT_CH_TIME_CFG_EIFS_AS_BUSY);
+
return mt76x2u_mac_stop(dev);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mac.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mac.c
index 3b82345..e7fea3a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mac.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2u.h"
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
index e4dfc3b..eb73cb8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2u.h"
@@ -48,7 +37,7 @@ mt76x2u_set_channel(struct mt76x02_dev *dev,
int err;
cancel_delayed_work_sync(&dev->cal_work);
- dev->beacon_ops->pre_tbtt_enable(dev, false);
+ mt76x02_pre_tbtt_enable(dev, false);
mutex_lock(&dev->mt76.mutex);
set_bit(MT76_RESET, &dev->mt76.state);
@@ -59,12 +48,16 @@ mt76x2u_set_channel(struct mt76x02_dev *dev,
err = mt76x2u_phy_set_channel(dev, chandef);
+ /* channel cycle counters read-and-clear */
+ mt76_rr(dev, MT_CH_IDLE);
+ mt76_rr(dev, MT_CH_BUSY);
+
mt76x2_mac_resume(dev);
clear_bit(MT76_RESET, &dev->mt76.state);
mutex_unlock(&dev->mt76.mutex);
- dev->beacon_ops->pre_tbtt_enable(dev, true);
+ mt76x02_pre_tbtt_enable(dev, true);
mt76_txq_schedule_all(&dev->mt76);
return err;
@@ -121,10 +114,11 @@ const struct ieee80211_ops mt76x2u_ops = {
.bss_info_changed = mt76x02_bss_info_changed,
.configure_filter = mt76x02_configure_filter,
.conf_tx = mt76x02_conf_tx,
- .sw_scan_start = mt76x02_sw_scan,
+ .sw_scan_start = mt76_sw_scan,
.sw_scan_complete = mt76x02_sw_scan_complete,
.sta_rate_tbl_update = mt76x02_sta_rate_tbl_update,
.get_txpower = mt76_get_txpower,
+ .get_survey = mt76_get_survey,
.set_tim = mt76_set_tim,
.release_buffered_frames = mt76_release_buffered_frames,
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mcu.c
index 152d41f..dd22d8a 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_mcu.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/firmware.h>
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_phy.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_phy.c
index dfd54f9..b1381f9 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_phy.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_phy.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76x2u.h"
diff --git a/drivers/net/wireless/mediatek/mt76/trace.c b/drivers/net/wireless/mediatek/mt76/trace.c
index ea4ab87..ed3df3c 100644
--- a/drivers/net/wireless/mediatek/mt76/trace.c
+++ b/drivers/net/wireless/mediatek/mt76/trace.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
diff --git a/drivers/net/wireless/mediatek/mt76/trace.h b/drivers/net/wireless/mediatek/mt76/trace.h
index ea30895..0b3e635 100644
--- a/drivers/net/wireless/mediatek/mt76/trace.h
+++ b/drivers/net/wireless/mediatek/mt76/trace.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#if !defined(__MT76_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
@@ -24,10 +13,11 @@
#define TRACE_SYSTEM mt76
#define MAXNAME 32
-#define DEV_ENTRY __array(char, wiphy_name, 32)
-#define DEV_ASSIGN strlcpy(__entry->wiphy_name, wiphy_name(dev->hw->wiphy), MAXNAME)
-#define DEV_PR_FMT "%s"
-#define DEV_PR_ARG __entry->wiphy_name
+#define DEV_ENTRY __array(char, wiphy_name, 32)
+#define DEV_ASSIGN strlcpy(__entry->wiphy_name, \
+ wiphy_name(dev->hw->wiphy), MAXNAME)
+#define DEV_PR_FMT "%s"
+#define DEV_PR_ARG __entry->wiphy_name
#define REG_ENTRY __field(u32, reg) __field(u32, val)
#define REG_ASSIGN __entry->reg = reg; __entry->val = val
diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c
index 5397827..c22a05f 100644
--- a/drivers/net/wireless/mediatek/mt76/tx.c
+++ b/drivers/net/wireless/mediatek/mt76/tx.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "mt76.h"
@@ -97,7 +86,7 @@ mt76_txq_get_qid(struct ieee80211_txq *txq)
static void
mt76_check_agg_ssn(struct mt76_txq *mtxq, struct sk_buff *skb)
{
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
if (!ieee80211_is_data_qos(hdr->frame_control) ||
!ieee80211_is_data_present(hdr->frame_control))
@@ -217,8 +206,8 @@ mt76_tx_status_skb_get(struct mt76_dev *dev, struct mt76_wcid *wcid, int pktid,
if (cb->pktid == pktid)
return skb;
- if (pktid >= 0 &&
- !time_after(jiffies, cb->jiffies + MT_TX_STATUS_SKB_TIMEOUT))
+ if (pktid >= 0 && !time_after(jiffies, cb->jiffies +
+ MT_TX_STATUS_SKB_TIMEOUT))
continue;
__mt76_tx_status_skb_done(dev, skb, MT_TX_CB_TXS_FAILED |
@@ -260,7 +249,7 @@ mt76_tx(struct mt76_dev *dev, struct ieee80211_sta *sta,
struct mt76_wcid *wcid, struct sk_buff *skb)
{
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
struct mt76_queue *q;
int qid = skb_get_queue_mapping(skb);
@@ -280,7 +269,7 @@ mt76_tx(struct mt76_dev *dev, struct ieee80211_sta *sta,
tid = skb->priority & IEEE80211_QOS_CTL_TID_MASK;
txq = sta->txq[tid];
- mtxq = (struct mt76_txq *) txq->drv_priv;
+ mtxq = (struct mt76_txq *)txq->drv_priv;
if (mtxq->aggr)
mt76_check_agg_ssn(mtxq, skb);
@@ -328,7 +317,7 @@ static void
mt76_queue_ps_skb(struct mt76_dev *dev, struct ieee80211_sta *sta,
struct sk_buff *skb, bool last)
{
- struct mt76_wcid *wcid = (struct mt76_wcid *) sta->drv_priv;
+ struct mt76_wcid *wcid = (struct mt76_wcid *)sta->drv_priv;
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
info->control.flags |= IEEE80211_TX_CTRL_PS_RESPONSE;
@@ -354,7 +343,7 @@ mt76_release_buffered_frames(struct ieee80211_hw *hw, struct ieee80211_sta *sta,
spin_lock_bh(&hwq->lock);
for (i = 0; tids && nframes; i++, tids >>= 1) {
struct ieee80211_txq *txq = sta->txq[i];
- struct mt76_txq *mtxq = (struct mt76_txq *) txq->drv_priv;
+ struct mt76_txq *mtxq = (struct mt76_txq *)txq->drv_priv;
struct sk_buff *skb;
if (!(tids & 1))
@@ -438,8 +427,7 @@ mt76_txq_send_burst(struct mt76_dev *dev, struct mt76_sw_queue *sq,
if (probe)
break;
- if (test_bit(MT76_OFFCHANNEL, &dev->state) ||
- test_bit(MT76_RESET, &dev->state))
+ if (test_bit(MT76_RESET, &dev->state))
return -EBUSY;
skb = mt76_txq_dequeue(dev, mtxq, false);
@@ -498,8 +486,7 @@ mt76_txq_schedule_list(struct mt76_dev *dev, enum mt76_txq_id qid)
if (sq->swq_queued >= 4)
break;
- if (test_bit(MT76_OFFCHANNEL, &dev->state) ||
- test_bit(MT76_RESET, &dev->state)) {
+ if (test_bit(MT76_RESET, &dev->state)) {
ret = -EBUSY;
break;
}
@@ -568,6 +555,13 @@ void mt76_txq_schedule_all(struct mt76_dev *dev)
}
EXPORT_SYMBOL_GPL(mt76_txq_schedule_all);
+void mt76_tx_tasklet(unsigned long data)
+{
+ struct mt76_dev *dev = (struct mt76_dev *)data;
+
+ mt76_txq_schedule_all(dev);
+}
+
void mt76_stop_tx_queues(struct mt76_dev *dev, struct ieee80211_sta *sta,
bool send_bar)
{
@@ -610,7 +604,7 @@ void mt76_txq_remove(struct mt76_dev *dev, struct ieee80211_txq *txq)
if (!txq)
return;
- mtxq = (struct mt76_txq *) txq->drv_priv;
+ mtxq = (struct mt76_txq *)txq->drv_priv;
while ((skb = skb_dequeue(&mtxq->retry_q)) != NULL)
ieee80211_free_txskb(dev->hw, skb);
@@ -619,7 +613,7 @@ EXPORT_SYMBOL_GPL(mt76_txq_remove);
void mt76_txq_init(struct mt76_dev *dev, struct ieee80211_txq *txq)
{
- struct mt76_txq *mtxq = (struct mt76_txq *) txq->drv_priv;
+ struct mt76_txq *mtxq = (struct mt76_txq *)txq->drv_priv;
skb_queue_head_init(&mtxq->retry_q);
diff --git a/drivers/net/wireless/mediatek/mt76/usb.c b/drivers/net/wireless/mediatek/mt76/usb.c
index fb87ce7..20c6fe5 100644
--- a/drivers/net/wireless/mediatek/mt76/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/usb.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
@@ -95,9 +84,9 @@ static u32 __mt76u_rr(struct mt76_dev *dev, u32 addr)
ret = __mt76u_vendor_request(dev, req,
USB_DIR_IN | USB_TYPE_VENDOR,
- 0, offset, usb->data, sizeof(__le32));
+ 0, offset, &usb->reg_val, sizeof(__le32));
if (ret == sizeof(__le32))
- data = get_unaligned_le32(usb->data);
+ data = le32_to_cpu(usb->reg_val);
trace_usb_reg_rr(dev, addr, data);
return data;
@@ -131,10 +120,10 @@ static void __mt76u_wr(struct mt76_dev *dev, u32 addr, u32 val)
}
offset = addr & ~MT_VEND_TYPE_MASK;
- put_unaligned_le32(val, usb->data);
+ usb->reg_val = cpu_to_le32(val);
__mt76u_vendor_request(dev, req,
USB_DIR_OUT | USB_TYPE_VENDOR, 0,
- offset, usb->data, sizeof(__le32));
+ offset, &usb->reg_val, sizeof(__le32));
trace_usb_reg_wr(dev, addr, val);
}
@@ -164,12 +153,12 @@ static void mt76u_copy(struct mt76_dev *dev, u32 offset,
int i, ret;
mutex_lock(&usb->usb_ctrl_mtx);
- for (i = 0; i < (len / 4); i++) {
- put_unaligned_le32(val[i], usb->data);
+ for (i = 0; i < DIV_ROUND_UP(len, 4); i++) {
+ put_unaligned(val[i], (u32 *)usb->data);
ret = __mt76u_vendor_request(dev, MT_VEND_MULTI_WRITE,
USB_DIR_OUT | USB_TYPE_VENDOR,
0, offset + i * 4, usb->data,
- sizeof(__le32));
+ sizeof(u32));
if (ret < 0)
break;
}
@@ -309,7 +298,7 @@ mt76u_fill_rx_sg(struct mt76_dev *dev, struct mt76_queue *q, struct urb *urb,
}
urb->num_sgs = max_t(int, i, urb->num_sgs);
- urb->transfer_buffer_length = urb->num_sgs * q->buf_size,
+ urb->transfer_buffer_length = urb->num_sgs * q->buf_size;
sg_init_marker(urb->sg, urb->num_sgs);
return i ? : -ENOMEM;
@@ -320,14 +309,13 @@ mt76u_refill_rx(struct mt76_dev *dev, struct urb *urb, int nsgs, gfp_t gfp)
{
struct mt76_queue *q = &dev->q_rx[MT_RXQ_MAIN];
- if (dev->usb.sg_en) {
+ if (dev->usb.sg_en)
return mt76u_fill_rx_sg(dev, q, urb, nsgs, gfp);
- } else {
- urb->transfer_buffer_length = q->buf_size;
- urb->transfer_buffer = page_frag_alloc(&q->rx_page,
- q->buf_size, gfp);
- return urb->transfer_buffer ? 0 : -ENOMEM;
- }
+
+ urb->transfer_buffer_length = q->buf_size;
+ urb->transfer_buffer = page_frag_alloc(&q->rx_page, q->buf_size, gfp);
+
+ return urb->transfer_buffer ? 0 : -ENOMEM;
}
static int
@@ -763,13 +751,14 @@ mt76u_tx_setup_buffers(struct mt76_dev *dev, struct sk_buff *skb,
if (!dev->usb.sg_en) {
urb->transfer_buffer = skb->data;
return 0;
- } else {
- sg_init_table(urb->sg, MT_TX_SG_MAX_SIZE);
- urb->num_sgs = skb_to_sgvec(skb, urb->sg, 0, skb->len);
- if (urb->num_sgs == 0)
- return -ENOMEM;
- return urb->num_sgs;
}
+
+ sg_init_table(urb->sg, MT_TX_SG_MAX_SIZE);
+ urb->num_sgs = skb_to_sgvec(skb, urb->sg, 0, skb->len);
+ if (!urb->num_sgs)
+ return -ENOMEM;
+
+ return urb->num_sgs;
}
static int
@@ -885,7 +874,8 @@ void mt76u_stop_tx(struct mt76_dev *dev)
struct mt76_queue *q;
int i, j, ret;
- ret = wait_event_timeout(dev->tx_wait, !mt76_has_tx_pending(dev), HZ/5);
+ ret = wait_event_timeout(dev->tx_wait, !mt76_has_tx_pending(dev),
+ HZ / 5);
if (!ret) {
dev_err(dev->dev, "timed out waiting for pending tx\n");
@@ -957,7 +947,7 @@ int mt76u_init(struct mt76_dev *dev,
.rr = mt76u_rr,
.wr = mt76u_wr,
.rmw = mt76u_rmw,
- .copy = mt76u_copy,
+ .write_copy = mt76u_copy,
.wr_rp = mt76u_wr_rp,
.rd_rp = mt76u_rd_rp,
.type = MT76_BUS_USB,
diff --git a/drivers/net/wireless/mediatek/mt76/usb_trace.c b/drivers/net/wireless/mediatek/mt76/usb_trace.c
index 7e1f540..9942bdd 100644
--- a/drivers/net/wireless/mediatek/mt76/usb_trace.c
+++ b/drivers/net/wireless/mediatek/mt76/usb_trace.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
diff --git a/drivers/net/wireless/mediatek/mt76/usb_trace.h b/drivers/net/wireless/mediatek/mt76/usb_trace.h
index b56c323..f5ab321 100644
--- a/drivers/net/wireless/mediatek/mt76/usb_trace.h
+++ b/drivers/net/wireless/mediatek/mt76/usb_trace.h
@@ -1,17 +1,6 @@
+/* SPDX-License-Identifier: ISC */
/*
* Copyright (C) 2018 Lorenzo Bianconi <lorenzo.bianconi83@gmail.com>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#if !defined(__MT76_USB_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
@@ -24,10 +13,11 @@
#define TRACE_SYSTEM mt76_usb
#define MAXNAME 32
-#define DEV_ENTRY __array(char, wiphy_name, 32)
-#define DEV_ASSIGN strlcpy(__entry->wiphy_name, wiphy_name(dev->hw->wiphy), MAXNAME)
-#define DEV_PR_FMT "%s "
-#define DEV_PR_ARG __entry->wiphy_name
+#define DEV_ENTRY __array(char, wiphy_name, 32)
+#define DEV_ASSIGN strlcpy(__entry->wiphy_name, \
+ wiphy_name(dev->hw->wiphy), MAXNAME)
+#define DEV_PR_FMT "%s "
+#define DEV_PR_ARG __entry->wiphy_name
#define REG_ENTRY __field(u32, reg) __field(u32, val)
#define REG_ASSIGN __entry->reg = reg; __entry->val = val
@@ -65,7 +55,7 @@ DECLARE_EVENT_CLASS(urb_transfer,
TP_PROTO(struct mt76_dev *dev, struct urb *u),
TP_ARGS(dev, u),
TP_STRUCT__entry(
- DEV_ENTRY __field(unsigned, pipe) __field(u32, len)
+ DEV_ENTRY __field(unsigned int, pipe) __field(u32, len)
),
TP_fast_assign(
DEV_ASSIGN;
diff --git a/drivers/net/wireless/mediatek/mt76/util.c b/drivers/net/wireless/mediatek/mt76/util.c
index 69270c1..23d1e1d 100644
--- a/drivers/net/wireless/mediatek/mt76/util.c
+++ b/drivers/net/wireless/mediatek/mt76/util.c
@@ -1,17 +1,6 @@
+// SPDX-License-Identifier: ISC
/*
* Copyright (C) 2016 Felix Fietkau <nbd@nbd.name>
- *
- * Permission to use, copy, modify, and/or distribute this software for any
- * purpose with or without fee is hereby granted, provided that the above
- * copyright notice and this permission notice appear in all copies.
- *
- * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
- * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
- * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
- * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
- * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
- * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <linux/module.h>
diff --git a/drivers/net/wireless/mediatek/mt76/util.h b/drivers/net/wireless/mediatek/mt76/util.h
index 9f8d592..fe3479c 100644
--- a/drivers/net/wireless/mediatek/mt76/util.h
+++ b/drivers/net/wireless/mediatek/mt76/util.h
@@ -12,7 +12,7 @@
#include <linux/bitfield.h>
#define MT76_INCR(_var, _size) \
- _var = (((_var) + 1) % _size)
+ (_var = (((_var) + 1) % (_size)))
int mt76_wcid_alloc(unsigned long *mask, int size);
@@ -25,7 +25,7 @@ mt76_wcid_free(unsigned long *mask, int idx)
static inline void
mt76_skb_set_moredata(struct sk_buff *skb, bool enable)
{
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *) skb->data;
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
if (enable)
hdr->frame_control |= cpu_to_le16(IEEE80211_FCTL_MOREDATA);
diff --git a/drivers/net/wireless/ralink/rt2x00/rt2800lib.c b/drivers/net/wireless/ralink/rt2x00/rt2800lib.c
index c9b957a..ecbe78b 100644
--- a/drivers/net/wireless/ralink/rt2x00/rt2800lib.c
+++ b/drivers/net/wireless/ralink/rt2x00/rt2800lib.c
@@ -6095,6 +6095,15 @@ static int rt2800_init_registers(struct rt2x00_dev *rt2x00dev)
}
/*
+ * Clear encryption initialization vectors on start, but keep them
+ * for watchdog reset. Otherwise we will have wrong IVs and not be
+ * able to keep connections after reset.
+ */
+ if (!test_bit(DEVICE_STATE_RESET, &rt2x00dev->flags))
+ for (i = 0; i < 256; i++)
+ rt2800_register_write(rt2x00dev, MAC_IVEIV_ENTRY(i), 0);
+
+ /*
* Clear all beacons
*/
for (i = 0; i < 8; i++)
diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00.h b/drivers/net/wireless/ralink/rt2x00/rt2x00.h
index 7e43690a..2b216ed 100644
--- a/drivers/net/wireless/ralink/rt2x00/rt2x00.h
+++ b/drivers/net/wireless/ralink/rt2x00/rt2x00.h
@@ -658,6 +658,7 @@ enum rt2x00_state_flags {
DEVICE_STATE_ENABLED_RADIO,
DEVICE_STATE_SCANNING,
DEVICE_STATE_FLUSHING,
+ DEVICE_STATE_RESET,
/*
* Driver configuration
diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c b/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c
index 35414f9..c3eab76 100644
--- a/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c
+++ b/drivers/net/wireless/ralink/rt2x00/rt2x00dev.c
@@ -371,9 +371,6 @@ static void rt2x00lib_fill_tx_status(struct rt2x00_dev *rt2x00dev,
IEEE80211_TX_CTL_AMPDU;
tx_info->status.ampdu_len = 1;
tx_info->status.ampdu_ack_len = success ? 1 : 0;
-
- if (!success)
- tx_info->flags |= IEEE80211_TX_STAT_AMPDU_NO_BACK;
}
if (rate_flags & IEEE80211_TX_RC_USE_RTS_CTS) {
@@ -1256,13 +1253,14 @@ static int rt2x00lib_initialize(struct rt2x00_dev *rt2x00dev)
int rt2x00lib_start(struct rt2x00_dev *rt2x00dev)
{
- int retval;
+ int retval = 0;
if (test_bit(DEVICE_STATE_STARTED, &rt2x00dev->flags)) {
/*
* This is special case for ieee80211_restart_hw(), otherwise
* mac80211 never call start() two times in row without stop();
*/
+ set_bit(DEVICE_STATE_RESET, &rt2x00dev->flags);
rt2x00dev->ops->lib->pre_reset_hw(rt2x00dev);
rt2x00lib_stop(rt2x00dev);
}
@@ -1273,14 +1271,14 @@ int rt2x00lib_start(struct rt2x00_dev *rt2x00dev)
*/
retval = rt2x00lib_load_firmware(rt2x00dev);
if (retval)
- return retval;
+ goto out;
/*
* Initialize the device.
*/
retval = rt2x00lib_initialize(rt2x00dev);
if (retval)
- return retval;
+ goto out;
rt2x00dev->intf_ap_count = 0;
rt2x00dev->intf_sta_count = 0;
@@ -1289,11 +1287,13 @@ int rt2x00lib_start(struct rt2x00_dev *rt2x00dev)
/* Enable the radio */
retval = rt2x00lib_enable_radio(rt2x00dev);
if (retval)
- return retval;
+ goto out;
set_bit(DEVICE_STATE_STARTED, &rt2x00dev->flags);
- return 0;
+out:
+ clear_bit(DEVICE_STATE_RESET, &rt2x00dev->flags);
+ return retval;
}
void rt2x00lib_stop(struct rt2x00_dev *rt2x00dev)
diff --git a/drivers/net/wireless/realtek/rtlwifi/debug.c b/drivers/net/wireless/realtek/rtlwifi/debug.c
index a051de1..55db71c 100644
--- a/drivers/net/wireless/realtek/rtlwifi/debug.c
+++ b/drivers/net/wireless/realtek/rtlwifi/debug.c
@@ -88,7 +88,7 @@ static const struct file_operations file_ops_common = {
.open = dl_debug_open_common,
.read = seq_read,
.llseek = seq_lseek,
- .release = seq_release,
+ .release = single_release,
};
static int rtl_debug_get_mac_page(struct seq_file *m, void *v)
diff --git a/drivers/net/wireless/realtek/rtlwifi/efuse.c b/drivers/net/wireless/realtek/rtlwifi/efuse.c
index ea4fc53..2646672 100644
--- a/drivers/net/wireless/realtek/rtlwifi/efuse.c
+++ b/drivers/net/wireless/realtek/rtlwifi/efuse.c
@@ -6,29 +6,12 @@
#include "pci.h"
#include <linux/export.h>
-static const u8 MAX_PGPKT_SIZE = 9;
static const u8 PGPKT_DATA_SIZE = 8;
static const int EFUSE_MAX_SIZE = 512;
#define START_ADDRESS 0x1000
#define REG_MCUFWDL 0x0080
-static const struct efuse_map RTL8712_SDIO_EFUSE_TABLE[] = {
- {0, 0, 0, 2},
- {0, 1, 0, 2},
- {0, 2, 0, 2},
- {1, 0, 0, 1},
- {1, 0, 1, 1},
- {1, 1, 0, 1},
- {1, 1, 1, 3},
- {1, 3, 0, 17},
- {3, 3, 1, 48},
- {10, 0, 0, 6},
- {10, 3, 0, 1},
- {10, 3, 1, 1},
- {11, 0, 0, 28}
-};
-
static const struct rtl_efuse_ops efuse_ops = {
.efuse_onebyte_read = efuse_one_byte_read,
.efuse_logical_map_read = efuse_shadow_read,
diff --git a/drivers/net/wireless/realtek/rtlwifi/pci.c b/drivers/net/wireless/realtek/rtlwifi/pci.c
index 7d96fe5..6087ec7 100644
--- a/drivers/net/wireless/realtek/rtlwifi/pci.c
+++ b/drivers/net/wireless/realtek/rtlwifi/pci.c
@@ -1793,6 +1793,8 @@ static int rtl_pci_start(struct ieee80211_hw *hw)
if (err) {
RT_TRACE(rtlpriv, COMP_INIT, DBG_DMESG,
"Failed to config hardware!\n");
+ kfree(rtlpriv->btcoexist.btc_context);
+ kfree(rtlpriv->btcoexist.wifi_only_context);
return err;
}
rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_RETRY_LIMIT,
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/trx.h
index bd86273..9177298 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/trx.h
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8188ee/trx.h
@@ -244,7 +244,7 @@ static inline void set_tx_desc_tx_buffer_address(__le32 *__pdesc, u32 __val)
*(__pdesc + 8) = cpu_to_le32(__val);
}
-static inline int get_tx_desc_tx_buffer_address(__le32 *__pdesc)
+static inline u32 get_tx_desc_tx_buffer_address(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 8));
}
@@ -464,22 +464,22 @@ static inline int get_rx_status_desc_magic_match(__le32 *__pdesc)
return le32_get_bits(*(__pdesc + 3), BIT(31));
}
-static inline int get_rx_desc_iv1(__le32 *__pdesc)
+static inline u32 get_rx_desc_iv1(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 4));
}
-static inline int get_rx_desc_tsfl(__le32 *__pdesc)
+static inline u32 get_rx_desc_tsfl(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 5));
}
-static inline int get_rx_desc_buff_addr(__le32 *__pdesc)
+static inline u32 get_rx_desc_buff_addr(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 6));
}
-static inline int get_rx_desc_buff_addr64(__le32 *__pdesc)
+static inline u32 get_rx_desc_buff_addr64(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 7));
}
@@ -501,12 +501,12 @@ static inline int get_rx_rpt2_desc_pkt_len(__le32 *__status)
return le32_get_bits(*(__status), GENMASK(8, 0));
}
-static inline int get_rx_rpt2_desc_macid_valid_1(__le32 *__status)
+static inline u32 get_rx_rpt2_desc_macid_valid_1(__le32 *__status)
{
return le32_to_cpu(*(__status + 4));
}
-static inline int get_rx_rpt2_desc_macid_valid_2(__le32 *__status)
+static inline u32 get_rx_rpt2_desc_macid_valid_2(__le32 *__status)
{
return le32_to_cpu(*(__status + 5));
}
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/trx.h
index 709dcac..b45b05a 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/trx.h
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192ce/trx.h
@@ -216,7 +216,7 @@ static inline void set_tx_desc_tx_buffer_address(__le32 *__pdesc, u32 __val)
*(__pdesc + 8) = cpu_to_le32(__val);
}
-static inline int get_tx_desc_tx_buffer_address(__le32 *__pdesc)
+static inline u32 get_tx_desc_tx_buffer_address(__le32 *__pdesc)
{
return le32_to_cpu(*((__pdesc + 8)));
}
@@ -311,12 +311,12 @@ static inline int get_rx_desc_bw(__le32 *__pdesc)
return le32_get_bits(*((__pdesc + 3)), BIT(9));
}
-static inline int get_rx_desc_tsfl(__le32 *__pdesc)
+static inline u32 get_rx_desc_tsfl(__le32 *__pdesc)
{
return le32_to_cpu(*((__pdesc + 5)));
}
-static inline int get_rx_desc_buff_addr(__le32 *__pdesc)
+static inline u32 get_rx_desc_buff_addr(__le32 *__pdesc)
{
return le32_to_cpu(*((__pdesc + 6)));
}
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/mac.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/mac.c
index b3ce800..c8daad1 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/mac.c
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/mac.c
@@ -638,7 +638,7 @@ static void _rtl92c_query_rxphystatus(struct ieee80211_hw *hw,
u32 rssi, total_rssi = 0;
bool in_powersavemode = false;
bool is_cck_rate;
- u8 *pdesc = (u8 *)p_desc;
+ __le32 *pdesc = (__le32 *)p_desc;
is_cck_rate = RX_HAL_IS_CCK_RATE(p_desc->rxmcs);
pstats->packet_matchbssid = packet_match_bssid;
@@ -736,9 +736,9 @@ static void _rtl92c_query_rxphystatus(struct ieee80211_hw *hw,
pstats->rx_pwdb_all = pwdb_all;
pstats->rxpower = rx_pwr_all;
pstats->recvsignalpower = rx_pwr_all;
- if (GET_RX_DESC_RX_MCS(pdesc) &&
- GET_RX_DESC_RX_MCS(pdesc) >= DESC_RATEMCS8 &&
- GET_RX_DESC_RX_MCS(pdesc) <= DESC_RATEMCS15)
+ if (get_rx_desc_rx_mcs(pdesc) &&
+ get_rx_desc_rx_mcs(pdesc) >= DESC_RATEMCS8 &&
+ get_rx_desc_rx_mcs(pdesc) <= DESC_RATEMCS15)
max_spatial_stream = 2;
else
max_spatial_stream = 1;
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.c
index 0020adc..fc52647 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.c
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.c
@@ -282,44 +282,45 @@ static enum rtl_desc_qsel _rtl8192cu_mq_to_descq(struct ieee80211_hw *hw,
bool rtl92cu_rx_query_desc(struct ieee80211_hw *hw,
struct rtl_stats *stats,
struct ieee80211_rx_status *rx_status,
- u8 *pdesc, struct sk_buff *skb)
+ u8 *pdesc8, struct sk_buff *skb)
{
struct rx_fwinfo_92c *p_drvinfo;
- struct rx_desc_92c *p_desc = (struct rx_desc_92c *)pdesc;
- u32 phystatus = GET_RX_DESC_PHY_STATUS(pdesc);
+ struct rx_desc_92c *p_desc = (struct rx_desc_92c *)pdesc8;
+ __le32 *pdesc = (__le32 *)pdesc8;
+ u32 phystatus = get_rx_desc_phy_status(pdesc);
- stats->length = (u16) GET_RX_DESC_PKT_LEN(pdesc);
- stats->rx_drvinfo_size = (u8)GET_RX_DESC_DRVINFO_SIZE(pdesc) *
+ stats->length = (u16)get_rx_desc_pkt_len(pdesc);
+ stats->rx_drvinfo_size = (u8)get_rx_desc_drvinfo_size(pdesc) *
RX_DRV_INFO_SIZE_UNIT;
- stats->rx_bufshift = (u8) (GET_RX_DESC_SHIFT(pdesc) & 0x03);
- stats->icv = (u16) GET_RX_DESC_ICV(pdesc);
- stats->crc = (u16) GET_RX_DESC_CRC32(pdesc);
+ stats->rx_bufshift = (u8)(get_rx_desc_shift(pdesc) & 0x03);
+ stats->icv = (u16)get_rx_desc_icv(pdesc);
+ stats->crc = (u16)get_rx_desc_crc32(pdesc);
stats->hwerror = (stats->crc | stats->icv);
- stats->decrypted = !GET_RX_DESC_SWDEC(pdesc);
- stats->rate = (u8) GET_RX_DESC_RX_MCS(pdesc);
- stats->shortpreamble = (u16) GET_RX_DESC_SPLCP(pdesc);
- stats->isampdu = (bool) (GET_RX_DESC_PAGGR(pdesc) == 1);
- stats->isfirst_ampdu = (bool)((GET_RX_DESC_PAGGR(pdesc) == 1)
- && (GET_RX_DESC_FAGGR(pdesc) == 1));
- stats->timestamp_low = GET_RX_DESC_TSFL(pdesc);
- stats->rx_is40mhzpacket = (bool)GET_RX_DESC_BW(pdesc);
- stats->is_ht = (bool)GET_RX_DESC_RX_HT(pdesc);
+ stats->decrypted = !get_rx_desc_swdec(pdesc);
+ stats->rate = (u8)get_rx_desc_rx_mcs(pdesc);
+ stats->shortpreamble = (u16)get_rx_desc_splcp(pdesc);
+ stats->isampdu = (bool)(get_rx_desc_paggr(pdesc) == 1);
+ stats->isfirst_ampdu = (bool)((get_rx_desc_paggr(pdesc) == 1) &&
+ (get_rx_desc_faggr(pdesc) == 1));
+ stats->timestamp_low = get_rx_desc_tsfl(pdesc);
+ stats->rx_is40mhzpacket = (bool)get_rx_desc_bw(pdesc);
+ stats->is_ht = (bool)get_rx_desc_rx_ht(pdesc);
rx_status->freq = hw->conf.chandef.chan->center_freq;
rx_status->band = hw->conf.chandef.chan->band;
- if (GET_RX_DESC_CRC32(pdesc))
+ if (get_rx_desc_crc32(pdesc))
rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
- if (!GET_RX_DESC_SWDEC(pdesc))
+ if (!get_rx_desc_swdec(pdesc))
rx_status->flag |= RX_FLAG_DECRYPTED;
- if (GET_RX_DESC_BW(pdesc))
+ if (get_rx_desc_bw(pdesc))
rx_status->bw = RATE_INFO_BW_40;
- if (GET_RX_DESC_RX_HT(pdesc))
+ if (get_rx_desc_rx_ht(pdesc))
rx_status->encoding = RX_ENC_HT;
rx_status->flag |= RX_FLAG_MACTIME_START;
if (stats->decrypted)
rx_status->flag |= RX_FLAG_DECRYPTED;
rx_status->rate_idx = rtlwifi_rate_mapping(hw, stats->is_ht,
false, stats->rate);
- rx_status->mactime = GET_RX_DESC_TSFL(pdesc);
+ rx_status->mactime = get_rx_desc_tsfl(pdesc);
if (phystatus) {
p_drvinfo = (struct rx_fwinfo_92c *)(skb->data +
stats->rx_bufshift);
@@ -339,7 +340,7 @@ static void _rtl_rx_process(struct ieee80211_hw *hw, struct sk_buff *skb)
(struct ieee80211_rx_status *)IEEE80211_SKB_RXCB(skb);
u32 skb_len, pkt_len, drvinfo_len;
struct rtl_priv *rtlpriv = rtl_priv(hw);
- u8 *rxdesc;
+ __le32 *rxdesc;
struct rtl_stats stats = {
.signal = 0,
.rate = 0,
@@ -350,44 +351,44 @@ static void _rtl_rx_process(struct ieee80211_hw *hw, struct sk_buff *skb)
struct ieee80211_hdr *hdr;
memset(rx_status, 0, sizeof(*rx_status));
- rxdesc = skb->data;
+ rxdesc = (__le32 *)skb->data;
skb_len = skb->len;
- drvinfo_len = (GET_RX_DESC_DRVINFO_SIZE(rxdesc) * RTL_RX_DRV_INFO_UNIT);
- pkt_len = GET_RX_DESC_PKT_LEN(rxdesc);
+ drvinfo_len = (get_rx_desc_drvinfo_size(rxdesc) * RTL_RX_DRV_INFO_UNIT);
+ pkt_len = get_rx_desc_pkt_len(rxdesc);
/* TODO: Error recovery. drop this skb or something. */
WARN_ON(skb_len < (pkt_len + RTL_RX_DESC_SIZE + drvinfo_len));
- stats.length = (u16) GET_RX_DESC_PKT_LEN(rxdesc);
- stats.rx_drvinfo_size = (u8)GET_RX_DESC_DRVINFO_SIZE(rxdesc) *
+ stats.length = (u16)get_rx_desc_pkt_len(rxdesc);
+ stats.rx_drvinfo_size = (u8)get_rx_desc_drvinfo_size(rxdesc) *
RX_DRV_INFO_SIZE_UNIT;
- stats.rx_bufshift = (u8) (GET_RX_DESC_SHIFT(rxdesc) & 0x03);
- stats.icv = (u16) GET_RX_DESC_ICV(rxdesc);
- stats.crc = (u16) GET_RX_DESC_CRC32(rxdesc);
+ stats.rx_bufshift = (u8)(get_rx_desc_shift(rxdesc) & 0x03);
+ stats.icv = (u16)get_rx_desc_icv(rxdesc);
+ stats.crc = (u16)get_rx_desc_crc32(rxdesc);
stats.hwerror = (stats.crc | stats.icv);
- stats.decrypted = !GET_RX_DESC_SWDEC(rxdesc);
- stats.rate = (u8) GET_RX_DESC_RX_MCS(rxdesc);
- stats.shortpreamble = (u16) GET_RX_DESC_SPLCP(rxdesc);
- stats.isampdu = (bool) ((GET_RX_DESC_PAGGR(rxdesc) == 1)
- && (GET_RX_DESC_FAGGR(rxdesc) == 1));
- stats.timestamp_low = GET_RX_DESC_TSFL(rxdesc);
- stats.rx_is40mhzpacket = (bool)GET_RX_DESC_BW(rxdesc);
- stats.is_ht = (bool)GET_RX_DESC_RX_HT(rxdesc);
+ stats.decrypted = !get_rx_desc_swdec(rxdesc);
+ stats.rate = (u8)get_rx_desc_rx_mcs(rxdesc);
+ stats.shortpreamble = (u16)get_rx_desc_splcp(rxdesc);
+ stats.isampdu = (bool)((get_rx_desc_paggr(rxdesc) == 1) &&
+ (get_rx_desc_faggr(rxdesc) == 1));
+ stats.timestamp_low = get_rx_desc_tsfl(rxdesc);
+ stats.rx_is40mhzpacket = (bool)get_rx_desc_bw(rxdesc);
+ stats.is_ht = (bool)get_rx_desc_rx_ht(rxdesc);
/* TODO: is center_freq changed when doing scan? */
/* TODO: Shall we add protection or just skip those two step? */
rx_status->freq = hw->conf.chandef.chan->center_freq;
rx_status->band = hw->conf.chandef.chan->band;
- if (GET_RX_DESC_CRC32(rxdesc))
+ if (get_rx_desc_crc32(rxdesc))
rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
- if (!GET_RX_DESC_SWDEC(rxdesc))
+ if (!get_rx_desc_swdec(rxdesc))
rx_status->flag |= RX_FLAG_DECRYPTED;
- if (GET_RX_DESC_BW(rxdesc))
+ if (get_rx_desc_bw(rxdesc))
rx_status->bw = RATE_INFO_BW_40;
- if (GET_RX_DESC_RX_HT(rxdesc))
+ if (get_rx_desc_rx_ht(rxdesc))
rx_status->encoding = RX_ENC_HT;
/* Data rate */
rx_status->rate_idx = rtlwifi_rate_mapping(hw, stats.is_ht,
false, stats.rate);
/* There is a phy status after this rx descriptor. */
- if (GET_RX_DESC_PHY_STATUS(rxdesc)) {
+ if (get_rx_desc_phy_status(rxdesc)) {
p_drvinfo = (struct rx_fwinfo_92c *)(rxdesc + RTL_RX_DESC_SIZE);
rtl92c_translate_rx_signal_stuff(hw, skb, &stats,
(struct rx_desc_92c *)rxdesc, p_drvinfo);
@@ -440,27 +441,27 @@ struct sk_buff *rtl8192c_tx_aggregate_hdl(struct ieee80211_hw *hw,
/*======================================== trx ===============================*/
-static void _rtl_fill_usb_tx_desc(u8 *txdesc)
+static void _rtl_fill_usb_tx_desc(__le32 *txdesc)
{
- SET_TX_DESC_OWN(txdesc, 1);
- SET_TX_DESC_LAST_SEG(txdesc, 1);
- SET_TX_DESC_FIRST_SEG(txdesc, 1);
+ set_tx_desc_own(txdesc, 1);
+ set_tx_desc_last_seg(txdesc, 1);
+ set_tx_desc_first_seg(txdesc, 1);
}
/**
* For HW recovery information
*/
-static void _rtl_tx_desc_checksum(u8 *txdesc)
+static void _rtl_tx_desc_checksum(__le32 *txdesc)
{
__le16 *ptr = (__le16 *)txdesc;
u16 checksum = 0;
u32 index;
/* Clear first */
- SET_TX_DESC_TX_DESC_CHECKSUM(txdesc, 0);
+ set_tx_desc_tx_desc_checksum(txdesc, 0);
for (index = 0; index < 16; index++)
checksum = checksum ^ le16_to_cpu(*(ptr + index));
- SET_TX_DESC_TX_DESC_CHECKSUM(txdesc, checksum);
+ set_tx_desc_tx_desc_checksum(txdesc, checksum);
}
void rtl92cu_tx_fill_desc(struct ieee80211_hw *hw,
@@ -483,61 +484,65 @@ void rtl92cu_tx_fill_desc(struct ieee80211_hw *hw,
u16 pktlen = skb->len;
enum rtl_desc_qsel fw_qsel = _rtl8192cu_mq_to_descq(hw, fc,
skb_get_queue_mapping(skb));
- u8 *txdesc;
+ u8 *txdesc8;
+ __le32 *txdesc;
seq_number = (le16_to_cpu(hdr->seq_ctrl) & IEEE80211_SCTL_SEQ) >> 4;
rtl_get_tcb_desc(hw, info, sta, skb, tcb_desc);
- txdesc = skb_push(skb, RTL_TX_HEADER_SIZE);
+ txdesc8 = skb_push(skb, RTL_TX_HEADER_SIZE);
+ txdesc = (__le32 *)txdesc8;
memset(txdesc, 0, RTL_TX_HEADER_SIZE);
- SET_TX_DESC_PKT_SIZE(txdesc, pktlen);
- SET_TX_DESC_LINIP(txdesc, 0);
- SET_TX_DESC_PKT_OFFSET(txdesc, RTL_DUMMY_OFFSET);
- SET_TX_DESC_OFFSET(txdesc, RTL_TX_HEADER_SIZE);
- SET_TX_DESC_TX_RATE(txdesc, tcb_desc->hw_rate);
+ set_tx_desc_pkt_size(txdesc, pktlen);
+ set_tx_desc_linip(txdesc, 0);
+ set_tx_desc_pkt_offset(txdesc, RTL_DUMMY_OFFSET);
+ set_tx_desc_offset(txdesc, RTL_TX_HEADER_SIZE);
+ set_tx_desc_tx_rate(txdesc, tcb_desc->hw_rate);
if (tcb_desc->use_shortgi || tcb_desc->use_shortpreamble)
- SET_TX_DESC_DATA_SHORTGI(txdesc, 1);
+ set_tx_desc_data_shortgi(txdesc, 1);
if (mac->tids[tid].agg.agg_state == RTL_AGG_ON &&
info->flags & IEEE80211_TX_CTL_AMPDU) {
- SET_TX_DESC_AGG_ENABLE(txdesc, 1);
- SET_TX_DESC_MAX_AGG_NUM(txdesc, 0x14);
+ set_tx_desc_agg_enable(txdesc, 1);
+ set_tx_desc_max_agg_num(txdesc, 0x14);
} else {
- SET_TX_DESC_AGG_BREAK(txdesc, 1);
+ set_tx_desc_agg_break(txdesc, 1);
}
- SET_TX_DESC_SEQ(txdesc, seq_number);
- SET_TX_DESC_RTS_ENABLE(txdesc, ((tcb_desc->rts_enable &&
- !tcb_desc->cts_enable) ? 1 : 0));
- SET_TX_DESC_HW_RTS_ENABLE(txdesc, ((tcb_desc->rts_enable ||
- tcb_desc->cts_enable) ? 1 : 0));
- SET_TX_DESC_CTS2SELF(txdesc, ((tcb_desc->cts_enable) ? 1 : 0));
- SET_TX_DESC_RTS_STBC(txdesc, ((tcb_desc->rts_stbc) ? 1 : 0));
- SET_TX_DESC_RTS_RATE(txdesc, tcb_desc->rts_rate);
- SET_TX_DESC_RTS_BW(txdesc, 0);
- SET_TX_DESC_RTS_SC(txdesc, tcb_desc->rts_sc);
- SET_TX_DESC_RTS_SHORT(txdesc,
+ set_tx_desc_seq(txdesc, seq_number);
+ set_tx_desc_rts_enable(txdesc,
+ ((tcb_desc->rts_enable &&
+ !tcb_desc->cts_enable) ? 1 : 0));
+ set_tx_desc_hw_rts_enable(txdesc,
+ ((tcb_desc->rts_enable ||
+ tcb_desc->cts_enable) ? 1 : 0));
+ set_tx_desc_cts2self(txdesc, ((tcb_desc->cts_enable) ? 1 : 0));
+ set_tx_desc_rts_stbc(txdesc, ((tcb_desc->rts_stbc) ? 1 : 0));
+ set_tx_desc_rts_rate(txdesc, tcb_desc->rts_rate);
+ set_tx_desc_rts_bw(txdesc, 0);
+ set_tx_desc_rts_sc(txdesc, tcb_desc->rts_sc);
+ set_tx_desc_rts_short(txdesc,
((tcb_desc->rts_rate <= DESC_RATE54M) ?
(tcb_desc->rts_use_shortpreamble ? 1 : 0)
: (tcb_desc->rts_use_shortgi ? 1 : 0)));
if (mac->bw_40) {
if (rate_flag & IEEE80211_TX_RC_DUP_DATA) {
- SET_TX_DESC_DATA_BW(txdesc, 1);
- SET_TX_DESC_DATA_SC(txdesc, 3);
+ set_tx_desc_data_bw(txdesc, 1);
+ set_tx_desc_data_sc(txdesc, 3);
} else if(rate_flag & IEEE80211_TX_RC_40_MHZ_WIDTH){
- SET_TX_DESC_DATA_BW(txdesc, 1);
- SET_TX_DESC_DATA_SC(txdesc, mac->cur_40_prime_sc);
+ set_tx_desc_data_bw(txdesc, 1);
+ set_tx_desc_data_sc(txdesc, mac->cur_40_prime_sc);
} else {
- SET_TX_DESC_DATA_BW(txdesc, 0);
- SET_TX_DESC_DATA_SC(txdesc, 0);
+ set_tx_desc_data_bw(txdesc, 0);
+ set_tx_desc_data_sc(txdesc, 0);
}
} else {
- SET_TX_DESC_DATA_BW(txdesc, 0);
- SET_TX_DESC_DATA_SC(txdesc, 0);
+ set_tx_desc_data_bw(txdesc, 0);
+ set_tx_desc_data_sc(txdesc, 0);
}
rcu_read_lock();
sta = ieee80211_find_sta(mac->vif, mac->bssid);
if (sta) {
u8 ampdu_density = sta->ht_cap.ampdu_density;
- SET_TX_DESC_AMPDU_DENSITY(txdesc, ampdu_density);
+ set_tx_desc_ampdu_density(txdesc, ampdu_density);
}
rcu_read_unlock();
if (info->control.hw_key) {
@@ -547,107 +552,110 @@ void rtl92cu_tx_fill_desc(struct ieee80211_hw *hw,
case WLAN_CIPHER_SUITE_WEP40:
case WLAN_CIPHER_SUITE_WEP104:
case WLAN_CIPHER_SUITE_TKIP:
- SET_TX_DESC_SEC_TYPE(txdesc, 0x1);
+ set_tx_desc_sec_type(txdesc, 0x1);
break;
case WLAN_CIPHER_SUITE_CCMP:
- SET_TX_DESC_SEC_TYPE(txdesc, 0x3);
+ set_tx_desc_sec_type(txdesc, 0x3);
break;
default:
- SET_TX_DESC_SEC_TYPE(txdesc, 0x0);
+ set_tx_desc_sec_type(txdesc, 0x0);
break;
}
}
- SET_TX_DESC_PKT_ID(txdesc, 0);
- SET_TX_DESC_QUEUE_SEL(txdesc, fw_qsel);
- SET_TX_DESC_DATA_RATE_FB_LIMIT(txdesc, 0x1F);
- SET_TX_DESC_RTS_RATE_FB_LIMIT(txdesc, 0xF);
- SET_TX_DESC_DISABLE_FB(txdesc, 0);
- SET_TX_DESC_USE_RATE(txdesc, tcb_desc->use_driver_rate ? 1 : 0);
+ set_tx_desc_pkt_id(txdesc, 0);
+ set_tx_desc_queue_sel(txdesc, fw_qsel);
+ set_tx_desc_data_rate_fb_limit(txdesc, 0x1F);
+ set_tx_desc_rts_rate_fb_limit(txdesc, 0xF);
+ set_tx_desc_disable_fb(txdesc, 0);
+ set_tx_desc_use_rate(txdesc, tcb_desc->use_driver_rate ? 1 : 0);
if (ieee80211_is_data_qos(fc)) {
if (mac->rdg_en) {
RT_TRACE(rtlpriv, COMP_SEND, DBG_TRACE,
"Enable RDG function\n");
- SET_TX_DESC_RDG_ENABLE(txdesc, 1);
- SET_TX_DESC_HTC(txdesc, 1);
+ set_tx_desc_rdg_enable(txdesc, 1);
+ set_tx_desc_htc(txdesc, 1);
}
}
if (rtlpriv->dm.useramask) {
- SET_TX_DESC_RATE_ID(txdesc, tcb_desc->ratr_index);
- SET_TX_DESC_MACID(txdesc, tcb_desc->mac_id);
+ set_tx_desc_rate_id(txdesc, tcb_desc->ratr_index);
+ set_tx_desc_macid(txdesc, tcb_desc->mac_id);
} else {
- SET_TX_DESC_RATE_ID(txdesc, 0xC + tcb_desc->ratr_index);
- SET_TX_DESC_MACID(txdesc, tcb_desc->ratr_index);
+ set_tx_desc_rate_id(txdesc, 0xC + tcb_desc->ratr_index);
+ set_tx_desc_macid(txdesc, tcb_desc->ratr_index);
}
if ((!ieee80211_is_data_qos(fc)) && ppsc->leisure_ps &&
ppsc->fwctrl_lps) {
- SET_TX_DESC_HWSEQ_EN(txdesc, 1);
- SET_TX_DESC_PKT_ID(txdesc, 8);
+ set_tx_desc_hwseq_en(txdesc, 1);
+ set_tx_desc_pkt_id(txdesc, 8);
if (!defaultadapter)
- SET_TX_DESC_QOS(txdesc, 1);
+ set_tx_desc_qos(txdesc, 1);
}
if (ieee80211_has_morefrags(fc))
- SET_TX_DESC_MORE_FRAG(txdesc, 1);
+ set_tx_desc_more_frag(txdesc, 1);
if (is_multicast_ether_addr(ieee80211_get_DA(hdr)) ||
is_broadcast_ether_addr(ieee80211_get_DA(hdr)))
- SET_TX_DESC_BMC(txdesc, 1);
+ set_tx_desc_bmc(txdesc, 1);
_rtl_fill_usb_tx_desc(txdesc);
_rtl_tx_desc_checksum(txdesc);
RT_TRACE(rtlpriv, COMP_SEND, DBG_TRACE, "==>\n");
}
-void rtl92cu_fill_fake_txdesc(struct ieee80211_hw *hw, u8 *pdesc,
+void rtl92cu_fill_fake_txdesc(struct ieee80211_hw *hw, u8 *pdesc8,
u32 buffer_len, bool is_pspoll)
{
+ __le32 *pdesc = (__le32 *)pdesc8;
+
/* Clear all status */
memset(pdesc, 0, RTL_TX_HEADER_SIZE);
- SET_TX_DESC_FIRST_SEG(pdesc, 1); /* bFirstSeg; */
- SET_TX_DESC_LAST_SEG(pdesc, 1); /* bLastSeg; */
- SET_TX_DESC_OFFSET(pdesc, RTL_TX_HEADER_SIZE); /* Offset = 32 */
- SET_TX_DESC_PKT_SIZE(pdesc, buffer_len); /* Buffer size + command hdr */
- SET_TX_DESC_QUEUE_SEL(pdesc, QSLT_MGNT); /* Fixed queue of Mgnt queue */
+ set_tx_desc_first_seg(pdesc, 1); /* bFirstSeg; */
+ set_tx_desc_last_seg(pdesc, 1); /* bLastSeg; */
+ set_tx_desc_offset(pdesc, RTL_TX_HEADER_SIZE); /* Offset = 32 */
+ set_tx_desc_pkt_size(pdesc, buffer_len); /* Buffer size + command hdr */
+ set_tx_desc_queue_sel(pdesc, QSLT_MGNT); /* Fixed queue of Mgnt queue */
/* Set NAVUSEHDR to prevent Ps-poll AId filed to be changed to error
* vlaue by Hw. */
if (is_pspoll) {
- SET_TX_DESC_NAV_USE_HDR(pdesc, 1);
+ set_tx_desc_nav_use_hdr(pdesc, 1);
} else {
- SET_TX_DESC_HWSEQ_EN(pdesc, 1); /* Hw set sequence number */
- SET_TX_DESC_PKT_ID(pdesc, 0x100); /* set bit3 to 1. */
+ set_tx_desc_hwseq_en(pdesc, 1); /* Hw set sequence number */
+ set_tx_desc_pkt_id(pdesc, BIT(3)); /* set bit3 to 1. */
}
- SET_TX_DESC_USE_RATE(pdesc, 1); /* use data rate which is set by Sw */
- SET_TX_DESC_OWN(pdesc, 1);
- SET_TX_DESC_TX_RATE(pdesc, DESC_RATE1M);
+ set_tx_desc_use_rate(pdesc, 1); /* use data rate which is set by Sw */
+ set_tx_desc_own(pdesc, 1);
+ set_tx_desc_tx_rate(pdesc, DESC_RATE1M);
_rtl_tx_desc_checksum(pdesc);
}
void rtl92cu_tx_fill_cmddesc(struct ieee80211_hw *hw,
- u8 *pdesc, bool firstseg,
+ u8 *pdesc8, bool firstseg,
bool lastseg, struct sk_buff *skb)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
u8 fw_queue = QSLT_BEACON;
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)(skb->data);
__le16 fc = hdr->frame_control;
+ __le32 *pdesc = (__le32 *)pdesc8;
memset((void *)pdesc, 0, RTL_TX_HEADER_SIZE);
if (firstseg)
- SET_TX_DESC_OFFSET(pdesc, RTL_TX_HEADER_SIZE);
- SET_TX_DESC_TX_RATE(pdesc, DESC_RATE1M);
- SET_TX_DESC_SEQ(pdesc, 0);
- SET_TX_DESC_LINIP(pdesc, 0);
- SET_TX_DESC_QUEUE_SEL(pdesc, fw_queue);
- SET_TX_DESC_FIRST_SEG(pdesc, 1);
- SET_TX_DESC_LAST_SEG(pdesc, 1);
- SET_TX_DESC_RATE_ID(pdesc, 7);
- SET_TX_DESC_MACID(pdesc, 0);
- SET_TX_DESC_OWN(pdesc, 1);
- SET_TX_DESC_PKT_SIZE(pdesc, (u16)skb->len);
- SET_TX_DESC_FIRST_SEG(pdesc, 1);
- SET_TX_DESC_LAST_SEG(pdesc, 1);
- SET_TX_DESC_OFFSET(pdesc, 0x20);
- SET_TX_DESC_USE_RATE(pdesc, 1);
+ set_tx_desc_offset(pdesc, RTL_TX_HEADER_SIZE);
+ set_tx_desc_tx_rate(pdesc, DESC_RATE1M);
+ set_tx_desc_seq(pdesc, 0);
+ set_tx_desc_linip(pdesc, 0);
+ set_tx_desc_queue_sel(pdesc, fw_queue);
+ set_tx_desc_first_seg(pdesc, 1);
+ set_tx_desc_last_seg(pdesc, 1);
+ set_tx_desc_rate_id(pdesc, 7);
+ set_tx_desc_macid(pdesc, 0);
+ set_tx_desc_own(pdesc, 1);
+ set_tx_desc_pkt_size(pdesc, (u16)skb->len);
+ set_tx_desc_first_seg(pdesc, 1);
+ set_tx_desc_last_seg(pdesc, 1);
+ set_tx_desc_offset(pdesc, 0x20);
+ set_tx_desc_use_rate(pdesc, 1);
if (!ieee80211_is_data_qos(fc)) {
- SET_TX_DESC_HWSEQ_EN(pdesc, 1);
- SET_TX_DESC_PKT_ID(pdesc, 8);
+ set_tx_desc_hwseq_en(pdesc, 1);
+ set_tx_desc_pkt_id(pdesc, 8);
}
RT_PRINT_DATA(rtlpriv, COMP_CMD, DBG_LOUD, "H2C Tx Cmd Content",
pdesc, RTL_TX_DESC_SIZE);
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.h
index ae2e8aa..171fe39 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.h
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/trx.h
@@ -73,286 +73,307 @@ struct rx_drv_info_92c {
/* macros to read various fields in RX descriptor */
/* DWORD 0 */
-#define GET_RX_DESC_PKT_LEN(__rxdesc) \
- LE_BITS_TO_4BYTE((__rxdesc), 0, 14)
-#define GET_RX_DESC_CRC32(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 14, 1)
-#define GET_RX_DESC_ICV(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 15, 1)
-#define GET_RX_DESC_DRVINFO_SIZE(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 16, 4)
-#define GET_RX_DESC_SECURITY(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 20, 3)
-#define GET_RX_DESC_QOS(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 23, 1)
-#define GET_RX_DESC_SHIFT(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 24, 2)
-#define GET_RX_DESC_PHY_STATUS(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 26, 1)
-#define GET_RX_DESC_SWDEC(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 27, 1)
-#define GET_RX_DESC_LAST_SEG(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 28, 1)
-#define GET_RX_DESC_FIRST_SEG(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 29, 1)
-#define GET_RX_DESC_EOR(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 30, 1)
-#define GET_RX_DESC_OWN(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc, 31, 1)
+static inline u32 get_rx_desc_pkt_len(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, GENMASK(13, 0));
+}
+
+static inline u32 get_rx_desc_crc32(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, BIT(14));
+}
+
+static inline u32 get_rx_desc_icv(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, BIT(15));
+}
+
+static inline u32 get_rx_desc_drvinfo_size(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, GENMASK(19, 16));
+}
+
+static inline u32 get_rx_desc_shift(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, GENMASK(25, 24));
+}
+
+static inline u32 get_rx_desc_phy_status(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, BIT(26));
+}
+
+static inline u32 get_rx_desc_swdec(__le32 *__rxdesc)
+{
+ return le32_get_bits(*__rxdesc, BIT(27));
+}
+
/* DWORD 1 */
-#define GET_RX_DESC_MACID(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 0, 5)
-#define GET_RX_DESC_TID(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 5, 4)
-#define GET_RX_DESC_PAGGR(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 14, 1)
-#define GET_RX_DESC_FAGGR(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 15, 1)
-#define GET_RX_DESC_A1_FIT(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 16, 4)
-#define GET_RX_DESC_A2_FIT(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 20, 4)
-#define GET_RX_DESC_PAM(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 24, 1)
-#define GET_RX_DESC_PWR(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 25, 1)
-#define GET_RX_DESC_MORE_DATA(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 26, 1)
-#define GET_RX_DESC_MORE_FRAG(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 27, 1)
-#define GET_RX_DESC_TYPE(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 28, 2)
-#define GET_RX_DESC_MC(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 30, 1)
-#define GET_RX_DESC_BC(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 4, 31, 1)
+static inline u32 get_rx_desc_paggr(__le32 *__rxdesc)
+{
+ return le32_get_bits(*(__rxdesc + 1), BIT(14));
+}
-/* DWORD 2 */
-#define GET_RX_DESC_SEQ(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 8, 0, 12)
-#define GET_RX_DESC_FRAG(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 8, 12, 4)
-#define GET_RX_DESC_USB_AGG_PKTNUM(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 8, 16, 8)
-#define GET_RX_DESC_NEXT_IND(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 8, 30, 1)
+static inline u32 get_rx_desc_faggr(__le32 *__rxdesc)
+{
+ return le32_get_bits(*(__rxdesc + 1), BIT(15));
+}
+
/* DWORD 3 */
-#define GET_RX_DESC_RX_MCS(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 0, 6)
-#define GET_RX_DESC_RX_HT(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 6, 1)
-#define GET_RX_DESC_AMSDU(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 7, 1)
-#define GET_RX_DESC_SPLCP(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 8, 1)
-#define GET_RX_DESC_BW(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 9, 1)
-#define GET_RX_DESC_HTC(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 10, 1)
-#define GET_RX_DESC_TCP_CHK_RPT(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 11, 1)
-#define GET_RX_DESC_IP_CHK_RPT(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 12, 1)
-#define GET_RX_DESC_TCP_CHK_VALID(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 13, 1)
-#define GET_RX_DESC_HWPC_ERR(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 14, 1)
-#define GET_RX_DESC_HWPC_IND(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 15, 1)
-#define GET_RX_DESC_IV0(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 12, 16, 16)
+static inline u32 get_rx_desc_rx_mcs(__le32 *__rxdesc)
+{
+ return le32_get_bits(*(__rxdesc + 3), GENMASK(5, 0));
+}
-/* DWORD 4 */
-#define GET_RX_DESC_IV1(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 16, 0, 32)
+static inline u32 get_rx_desc_rx_ht(__le32 *__rxdesc)
+{
+ return le32_get_bits(*(__rxdesc + 3), BIT(6));
+}
+
+static inline u32 get_rx_desc_splcp(__le32 *__rxdesc)
+{
+ return le32_get_bits(*(__rxdesc + 3), BIT(8));
+}
+
+static inline u32 get_rx_desc_bw(__le32 *__rxdesc)
+{
+ return le32_get_bits(*(__rxdesc + 3), BIT(9));
+}
+
/* DWORD 5 */
-#define GET_RX_DESC_TSFL(__rxdesc) \
- LE_BITS_TO_4BYTE(__rxdesc + 20, 0, 32)
+static inline u32 get_rx_desc_tsfl(__le32 *__rxdesc)
+{
+ return le32_to_cpu(*((__rxdesc + 5)));
+}
+
/*======================= tx desc ============================================*/
/* macros to set various fields in TX descriptor */
/* Dword 0 */
-#define SET_TX_DESC_PKT_SIZE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 0, 16, __value)
-#define SET_TX_DESC_OFFSET(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 16, 8, __value)
-#define SET_TX_DESC_BMC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 24, 1, __value)
-#define SET_TX_DESC_HTC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 25, 1, __value)
-#define SET_TX_DESC_LAST_SEG(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 26, 1, __value)
-#define SET_TX_DESC_FIRST_SEG(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 27, 1, __value)
-#define SET_TX_DESC_LINIP(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 28, 1, __value)
-#define SET_TX_DESC_NO_ACM(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 29, 1, __value)
-#define SET_TX_DESC_GF(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 30, 1, __value)
-#define SET_TX_DESC_OWN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc, 31, 1, __value)
+static inline void set_tx_desc_pkt_size(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, GENMASK(15, 0));
+}
+
+static inline void set_tx_desc_offset(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, GENMASK(23, 16));
+}
+
+static inline void set_tx_desc_bmc(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, BIT(24));
+}
+
+static inline void set_tx_desc_htc(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, BIT(25));
+}
+
+static inline void set_tx_desc_last_seg(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, BIT(26));
+}
+
+static inline void set_tx_desc_first_seg(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, BIT(27));
+}
+
+static inline void set_tx_desc_linip(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, BIT(28));
+}
+
+static inline void set_tx_desc_own(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits(__txdesc, __value, BIT(31));
+}
+
/* Dword 1 */
-#define SET_TX_DESC_MACID(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 0, 5, __value)
-#define SET_TX_DESC_AGG_ENABLE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 5, 1, __value)
-#define SET_TX_DESC_AGG_BREAK(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 6, 1, __value)
-#define SET_TX_DESC_RDG_ENABLE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 7, 1, __value)
-#define SET_TX_DESC_QUEUE_SEL(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 8, 5, __value)
-#define SET_TX_DESC_RDG_NAV_EXT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 13, 1, __value)
-#define SET_TX_DESC_LSIG_TXOP_EN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 14, 1, __value)
-#define SET_TX_DESC_PIFS(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 15, 1, __value)
-#define SET_TX_DESC_RATE_ID(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 16, 4, __value)
-#define SET_TX_DESC_RA_BRSR_ID(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 16, 4, __value)
-#define SET_TX_DESC_NAV_USE_HDR(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 20, 1, __value)
-#define SET_TX_DESC_EN_DESC_ID(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 21, 1, __value)
-#define SET_TX_DESC_SEC_TYPE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 22, 2, __value)
-#define SET_TX_DESC_PKT_OFFSET(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 4, 26, 5, __value)
+static inline void set_tx_desc_macid(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, GENMASK(4, 0));
+}
+
+static inline void set_tx_desc_agg_enable(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, BIT(5));
+}
+
+static inline void set_tx_desc_agg_break(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, BIT(6));
+}
+
+static inline void set_tx_desc_rdg_enable(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, BIT(7));
+}
+
+static inline void set_tx_desc_queue_sel(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, GENMASK(12, 8));
+}
+
+static inline void set_tx_desc_rate_id(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, GENMASK(19, 16));
+}
+
+static inline void set_tx_desc_nav_use_hdr(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, BIT(20));
+}
+
+static inline void set_tx_desc_sec_type(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, GENMASK(23, 22));
+}
+
+static inline void set_tx_desc_pkt_offset(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 1), __value, GENMASK(30, 26));
+}
+
/* Dword 2 */
-#define SET_TX_DESC_RTS_RC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 0, 6, __value)
-#define SET_TX_DESC_DATA_RC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 6, 6, __value)
-#define SET_TX_DESC_BAR_RTY_TH(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 14, 2, __value)
-#define SET_TX_DESC_MORE_FRAG(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 17, 1, __value)
-#define SET_TX_DESC_RAW(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 18, 1, __value)
-#define SET_TX_DESC_CCX(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 19, 1, __value)
-#define SET_TX_DESC_AMPDU_DENSITY(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 20, 3, __value)
-#define SET_TX_DESC_ANTSEL_A(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 24, 1, __value)
-#define SET_TX_DESC_ANTSEL_B(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 25, 1, __value)
-#define SET_TX_DESC_TX_ANT_CCK(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 26, 2, __value)
-#define SET_TX_DESC_TX_ANTL(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 28, 2, __value)
-#define SET_TX_DESC_TX_ANT_HT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 8, 30, 2, __value)
+static inline void set_tx_desc_more_frag(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 2), __value, BIT(17));
+}
+
+static inline void set_tx_desc_ampdu_density(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 2), __value, GENMASK(22, 20));
+}
+
/* Dword 3 */
-#define SET_TX_DESC_NEXT_HEAP_PAGE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 12, 0, 8, __value)
-#define SET_TX_DESC_TAIL_PAGE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 12, 8, 8, __value)
-#define SET_TX_DESC_SEQ(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 12, 16, 12, __value)
-#define SET_TX_DESC_PKT_ID(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 12, 28, 4, __value)
+static inline void set_tx_desc_seq(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 3), __value, GENMASK(27, 16));
+}
+
+static inline void set_tx_desc_pkt_id(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 3), __value, GENMASK(31, 28));
+}
+
/* Dword 4 */
-#define SET_TX_DESC_RTS_RATE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 0, 5, __value)
-#define SET_TX_DESC_AP_DCFE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 5, 1, __value)
-#define SET_TX_DESC_QOS(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 6, 1, __value)
-#define SET_TX_DESC_HWSEQ_EN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 7, 1, __value)
-#define SET_TX_DESC_USE_RATE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 8, 1, __value)
-#define SET_TX_DESC_DISABLE_RTS_FB(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 9, 1, __value)
-#define SET_TX_DESC_DISABLE_FB(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 10, 1, __value)
-#define SET_TX_DESC_CTS2SELF(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 11, 1, __value)
-#define SET_TX_DESC_RTS_ENABLE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 12, 1, __value)
-#define SET_TX_DESC_HW_RTS_ENABLE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 13, 1, __value)
-#define SET_TX_DESC_WAIT_DCTS(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 18, 1, __value)
-#define SET_TX_DESC_CTS2AP_EN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 19, 1, __value)
-#define SET_TX_DESC_DATA_SC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 20, 2, __value)
-#define SET_TX_DESC_DATA_STBC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 22, 2, __value)
-#define SET_TX_DESC_DATA_SHORT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 24, 1, __value)
-#define SET_TX_DESC_DATA_BW(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 25, 1, __value)
-#define SET_TX_DESC_RTS_SHORT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 26, 1, __value)
-#define SET_TX_DESC_RTS_BW(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 27, 1, __value)
-#define SET_TX_DESC_RTS_SC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 28, 2, __value)
-#define SET_TX_DESC_RTS_STBC(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 16, 30, 2, __value)
+static inline void set_tx_desc_rts_rate(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, GENMASK(4, 0));
+}
+
+static inline void set_tx_desc_qos(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(6));
+}
+
+static inline void set_tx_desc_hwseq_en(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(7));
+}
+
+static inline void set_tx_desc_use_rate(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(8));
+}
+
+static inline void set_tx_desc_disable_fb(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(10));
+}
+
+static inline void set_tx_desc_cts2self(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(11));
+}
+
+static inline void set_tx_desc_rts_enable(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(12));
+}
+
+static inline void set_tx_desc_hw_rts_enable(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(13));
+}
+
+static inline void set_tx_desc_data_sc(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, GENMASK(21, 20));
+}
+
+static inline void set_tx_desc_data_bw(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(25));
+}
+
+static inline void set_tx_desc_rts_short(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(26));
+}
+
+static inline void set_tx_desc_rts_bw(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, BIT(27));
+}
+
+static inline void set_tx_desc_rts_sc(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, GENMASK(29, 28));
+}
+
+static inline void set_tx_desc_rts_stbc(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 4), __value, GENMASK(31, 30));
+}
+
/* Dword 5 */
-#define SET_TX_DESC_TX_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc + 20, 0, 6, __val)
-#define SET_TX_DESC_DATA_SHORTGI(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc + 20, 6, 1, __val)
-#define SET_TX_DESC_CCX_TAG(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc + 20, 7, 1, __val)
-#define SET_TX_DESC_DATA_RATE_FB_LIMIT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 20, 8, 5, __value)
-#define SET_TX_DESC_RTS_RATE_FB_LIMIT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 20, 13, 4, __value)
-#define SET_TX_DESC_RETRY_LIMIT_ENABLE(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 20, 17, 1, __value)
-#define SET_TX_DESC_DATA_RETRY_LIMIT(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 20, 18, 6, __value)
-#define SET_TX_DESC_USB_TXAGG_NUM(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 20, 24, 8, __value)
+static inline void set_tx_desc_tx_rate(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 5), __val, GENMASK(5, 0));
+}
+
+static inline void set_tx_desc_data_shortgi(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 5), __val, BIT(6));
+}
+
+static inline void set_tx_desc_data_rate_fb_limit(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 5), __value, GENMASK(12, 8));
+}
+
+static inline void set_tx_desc_rts_rate_fb_limit(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 5), __value, GENMASK(16, 13));
+}
+
/* Dword 6 */
-#define SET_TX_DESC_TXAGC_A(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 0, 5, __value)
-#define SET_TX_DESC_TXAGC_B(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 5, 5, __value)
-#define SET_TX_DESC_USB_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 10, 1, __value)
-#define SET_TX_DESC_MAX_AGG_NUM(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 11, 5, __value)
-#define SET_TX_DESC_MCSG1_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 16, 4, __value)
-#define SET_TX_DESC_MCSG2_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 20, 4, __value)
-#define SET_TX_DESC_MCSG3_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 24, 4, __value)
-#define SET_TX_DESC_MCSG7_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 24, 28, 4, __value)
+static inline void set_tx_desc_max_agg_num(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 6), __value, GENMASK(15, 11));
+}
+
/* Dword 7 */
-#define SET_TX_DESC_TX_DESC_CHECKSUM(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 28, 0, 16, __value)
-#define SET_TX_DESC_MCSG4_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 28, 16, 4, __value)
-#define SET_TX_DESC_MCSG5_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 28, 20, 4, __value)
-#define SET_TX_DESC_MCSG6_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 28, 24, 4, __value)
-#define SET_TX_DESC_MCSG15_MAX_LEN(__txdesc, __value) \
- SET_BITS_TO_LE_4BYTE(__txdesc + 28, 28, 4, __value)
+static inline void set_tx_desc_tx_desc_checksum(__le32 *__txdesc, u32 __value)
+{
+ le32p_replace_bits((__txdesc + 7), __value, GENMASK(15, 0));
+}
+
int rtl8192cu_endpoint_mapping(struct ieee80211_hw *hw);
u16 rtl8192cu_mq_to_hwq(__le16 fc, u16 mac80211_queue_index);
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.c
index d297cfc..27f1a63 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.c
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.c
@@ -34,7 +34,7 @@ static void _rtl92ee_query_rxphystatus(struct ieee80211_hw *hw,
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
struct phy_status_rpt *p_phystrpt = (struct phy_status_rpt *)p_drvinfo;
- s8 rx_pwr_all = 0, rx_pwr[4];
+ s8 rx_pwr_all, rx_pwr[4];
u8 rf_rx_num = 0, evm, pwdb_all;
u8 i, max_spatial_stream;
u32 rssi, total_rssi = 0;
@@ -95,6 +95,7 @@ static void _rtl92ee_query_rxphystatus(struct ieee80211_hw *hw,
rx_pwr_all = 14 - 2 * vga_idx;
break;
default:
+ rx_pwr_all = 0;
break;
}
rx_pwr_all += 16;
@@ -271,13 +272,14 @@ static void _rtl92ee_translate_rx_signal_stuff(struct ieee80211_hw *hw,
}
static void _rtl92ee_insert_emcontent(struct rtl_tcb_desc *ptcb_desc,
- u8 *virtualaddress)
+ u8 *virtualaddress8)
{
- u32 dwtmp = 0;
+ u32 dwtmp;
+ __le32 *virtualaddress = (__le32 *)virtualaddress8;
memset(virtualaddress, 0, 8);
- SET_EARLYMODE_PKTNUM(virtualaddress, ptcb_desc->empkt_num);
+ set_earlymode_pktnum(virtualaddress, ptcb_desc->empkt_num);
if (ptcb_desc->empkt_num == 1) {
dwtmp = ptcb_desc->empkt_len[0];
} else {
@@ -285,7 +287,7 @@ static void _rtl92ee_insert_emcontent(struct rtl_tcb_desc *ptcb_desc,
dwtmp += ((dwtmp % 4) ? (4 - dwtmp % 4) : 0) + 4;
dwtmp += ptcb_desc->empkt_len[1];
}
- SET_EARLYMODE_LEN0(virtualaddress, dwtmp);
+ set_earlymode_len0(virtualaddress, dwtmp);
if (ptcb_desc->empkt_num <= 3) {
dwtmp = ptcb_desc->empkt_len[2];
@@ -294,7 +296,7 @@ static void _rtl92ee_insert_emcontent(struct rtl_tcb_desc *ptcb_desc,
dwtmp += ((dwtmp % 4) ? (4 - dwtmp % 4) : 0) + 4;
dwtmp += ptcb_desc->empkt_len[3];
}
- SET_EARLYMODE_LEN1(virtualaddress, dwtmp);
+ set_earlymode_len1(virtualaddress, dwtmp);
if (ptcb_desc->empkt_num <= 5) {
dwtmp = ptcb_desc->empkt_len[4];
} else {
@@ -302,8 +304,8 @@ static void _rtl92ee_insert_emcontent(struct rtl_tcb_desc *ptcb_desc,
dwtmp += ((dwtmp % 4) ? (4 - dwtmp % 4) : 0) + 4;
dwtmp += ptcb_desc->empkt_len[5];
}
- SET_EARLYMODE_LEN2_1(virtualaddress, dwtmp & 0xF);
- SET_EARLYMODE_LEN2_2(virtualaddress, dwtmp >> 4);
+ set_earlymode_len2_1(virtualaddress, dwtmp & 0xF);
+ set_earlymode_len2_2(virtualaddress, dwtmp >> 4);
if (ptcb_desc->empkt_num <= 7) {
dwtmp = ptcb_desc->empkt_len[6];
} else {
@@ -311,7 +313,7 @@ static void _rtl92ee_insert_emcontent(struct rtl_tcb_desc *ptcb_desc,
dwtmp += ((dwtmp % 4) ? (4 - dwtmp % 4) : 0) + 4;
dwtmp += ptcb_desc->empkt_len[7];
}
- SET_EARLYMODE_LEN3(virtualaddress, dwtmp);
+ set_earlymode_len3(virtualaddress, dwtmp);
if (ptcb_desc->empkt_num <= 9) {
dwtmp = ptcb_desc->empkt_len[8];
} else {
@@ -319,43 +321,44 @@ static void _rtl92ee_insert_emcontent(struct rtl_tcb_desc *ptcb_desc,
dwtmp += ((dwtmp % 4) ? (4 - dwtmp % 4) : 0) + 4;
dwtmp += ptcb_desc->empkt_len[9];
}
- SET_EARLYMODE_LEN4(virtualaddress, dwtmp);
+ set_earlymode_len4(virtualaddress, dwtmp);
}
bool rtl92ee_rx_query_desc(struct ieee80211_hw *hw,
struct rtl_stats *status,
struct ieee80211_rx_status *rx_status,
- u8 *pdesc, struct sk_buff *skb)
+ u8 *pdesc8, struct sk_buff *skb)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
struct rx_fwinfo *p_drvinfo;
struct ieee80211_hdr *hdr;
- u32 phystatus = GET_RX_DESC_PHYST(pdesc);
+ __le32 *pdesc = (__le32 *)pdesc8;
+ u32 phystatus = get_rx_desc_physt(pdesc);
u8 wake_match;
- if (GET_RX_STATUS_DESC_RPT_SEL(pdesc) == 0)
+ if (get_rx_status_desc_rpt_sel(pdesc) == 0)
status->packet_report_type = NORMAL_RX;
else
status->packet_report_type = C2H_PACKET;
- status->length = (u16)GET_RX_DESC_PKT_LEN(pdesc);
- status->rx_drvinfo_size = (u8)GET_RX_DESC_DRV_INFO_SIZE(pdesc) *
+ status->length = (u16)get_rx_desc_pkt_len(pdesc);
+ status->rx_drvinfo_size = (u8)get_rx_desc_drv_info_size(pdesc) *
RX_DRV_INFO_SIZE_UNIT;
- status->rx_bufshift = (u8)(GET_RX_DESC_SHIFT(pdesc) & 0x03);
- status->icv = (u16)GET_RX_DESC_ICV(pdesc);
- status->crc = (u16)GET_RX_DESC_CRC32(pdesc);
+ status->rx_bufshift = (u8)(get_rx_desc_shift(pdesc) & 0x03);
+ status->icv = (u16)get_rx_desc_icv(pdesc);
+ status->crc = (u16)get_rx_desc_crc32(pdesc);
status->hwerror = (status->crc | status->icv);
- status->decrypted = !GET_RX_DESC_SWDEC(pdesc);
- status->rate = (u8)GET_RX_DESC_RXMCS(pdesc);
- status->isampdu = (bool)(GET_RX_DESC_PAGGR(pdesc) == 1);
- status->timestamp_low = GET_RX_DESC_TSFL(pdesc);
+ status->decrypted = !get_rx_desc_swdec(pdesc);
+ status->rate = (u8)get_rx_desc_rxmcs(pdesc);
+ status->isampdu = (bool)(get_rx_desc_paggr(pdesc) == 1);
+ status->timestamp_low = get_rx_desc_tsfl(pdesc);
status->is_cck = RTL92EE_RX_HAL_IS_CCK_RATE(status->rate);
- status->macid = GET_RX_DESC_MACID(pdesc);
- if (GET_RX_STATUS_DESC_PATTERN_MATCH(pdesc))
+ status->macid = get_rx_desc_macid(pdesc);
+ if (get_rx_status_desc_pattern_match(pdesc))
wake_match = BIT(2);
- else if (GET_RX_STATUS_DESC_MAGIC_MATCH(pdesc))
+ else if (get_rx_status_desc_magic_match(pdesc))
wake_match = BIT(1);
- else if (GET_RX_STATUS_DESC_UNICAST_MATCH(pdesc))
+ else if (get_rx_status_desc_unicast_match(pdesc))
wake_match = BIT(0);
else
wake_match = 0;
@@ -409,42 +412,43 @@ bool rtl92ee_rx_query_desc(struct ieee80211_hw *hw,
p_drvinfo = (struct rx_fwinfo *)(skb->data +
status->rx_bufshift + 24);
- _rtl92ee_translate_rx_signal_stuff(hw, skb, status, pdesc,
+ _rtl92ee_translate_rx_signal_stuff(hw, skb, status, pdesc8,
p_drvinfo);
}
rx_status->signal = status->recvsignalpower + 10;
if (status->packet_report_type == TX_REPORT2) {
status->macid_valid_entry[0] =
- GET_RX_RPT2_DESC_MACID_VALID_1(pdesc);
+ get_rx_rpt2_desc_macid_valid_1(pdesc);
status->macid_valid_entry[1] =
- GET_RX_RPT2_DESC_MACID_VALID_2(pdesc);
+ get_rx_rpt2_desc_macid_valid_2(pdesc);
}
return true;
}
/*in Windows, this == Rx_92EE_Interrupt*/
-void rtl92ee_rx_check_dma_ok(struct ieee80211_hw *hw, u8 *header_desc,
+void rtl92ee_rx_check_dma_ok(struct ieee80211_hw *hw, u8 *header_desc8,
u8 queue_index)
{
u8 first_seg = 0;
u8 last_seg = 0;
u16 total_len = 0;
u16 read_cnt = 0;
+ __le32 *header_desc = (__le32 *)header_desc8;
if (header_desc == NULL)
return;
- total_len = (u16)GET_RX_BUFFER_DESC_TOTAL_LENGTH(header_desc);
+ total_len = (u16)get_rx_buffer_desc_total_length(header_desc);
- first_seg = (u8)GET_RX_BUFFER_DESC_FS(header_desc);
+ first_seg = (u8)get_rx_buffer_desc_fs(header_desc);
- last_seg = (u8)GET_RX_BUFFER_DESC_LS(header_desc);
+ last_seg = (u8)get_rx_buffer_desc_ls(header_desc);
while (total_len == 0 && first_seg == 0 && last_seg == 0) {
read_cnt++;
- total_len = (u16)GET_RX_BUFFER_DESC_TOTAL_LENGTH(header_desc);
- first_seg = (u8)GET_RX_BUFFER_DESC_FS(header_desc);
- last_seg = (u8)GET_RX_BUFFER_DESC_LS(header_desc);
+ total_len = (u16)get_rx_buffer_desc_total_length(header_desc);
+ first_seg = (u8)get_rx_buffer_desc_fs(header_desc);
+ last_seg = (u8)get_rx_buffer_desc_ls(header_desc);
if (read_cnt > 20)
break;
@@ -455,8 +459,8 @@ u16 rtl92ee_rx_desc_buff_remained_cnt(struct ieee80211_hw *hw, u8 queue_index)
{
struct rtl_pci *rtlpci = rtl_pcidev(rtl_pcipriv(hw));
struct rtl_priv *rtlpriv = rtl_priv(hw);
- u16 read_point = 0, write_point = 0, remind_cnt = 0;
- u32 tmp_4byte = 0;
+ u16 read_point, write_point, remind_cnt;
+ u32 tmp_4byte;
static bool start_rx;
tmp_4byte = rtl_read_dword(rtlpriv, REG_RXQ_TXBD_IDX);
@@ -490,7 +494,7 @@ u16 rtl92ee_rx_desc_buff_remained_cnt(struct ieee80211_hw *hw, u8 queue_index)
static u16 get_desc_addr_fr_q_idx(u16 queue_index)
{
- u16 desc_address = REG_BEQ_TXBD_IDX;
+ u16 desc_address;
switch (queue_index) {
case BK_QUEUE:
@@ -521,6 +525,7 @@ static u16 get_desc_addr_fr_q_idx(u16 queue_index)
desc_address = REG_BEQ_TXBD_IDX;
break;
default:
+ desc_address = REG_BEQ_TXBD_IDX;
break;
}
return desc_address;
@@ -530,7 +535,7 @@ u16 rtl92ee_get_available_desc(struct ieee80211_hw *hw, u8 q_idx)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
u16 point_diff = 0;
- u16 current_tx_read_point = 0, current_tx_write_point = 0;
+ u16 current_tx_read_point, current_tx_write_point;
u32 tmp_4byte;
tmp_4byte = rtl_read_dword(rtlpriv,
@@ -546,7 +551,7 @@ u16 rtl92ee_get_available_desc(struct ieee80211_hw *hw, u8 q_idx)
}
void rtl92ee_pre_fill_tx_bd_desc(struct ieee80211_hw *hw,
- u8 *tx_bd_desc, u8 *desc, u8 queue_index,
+ u8 *tx_bd_desc8, u8 *desc8, u8 queue_index,
struct sk_buff *skb, dma_addr_t addr)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
@@ -554,15 +559,17 @@ void rtl92ee_pre_fill_tx_bd_desc(struct ieee80211_hw *hw,
u32 pkt_len = skb->len;
u16 desc_size = 40; /*tx desc size*/
u32 psblen = 0;
- u16 tx_page_size = 0;
- u32 total_packet_size = 0;
+ u16 tx_page_size;
+ u32 total_packet_size;
u16 current_bd_desc;
- u8 i = 0;
+ u8 i;
u16 real_desc_size = 0x28;
u16 append_early_mode_size = 0;
u8 segmentnum = 1 << (RTL8192EE_SEG_NUM + 1);
dma_addr_t desc_dma_addr;
bool dma64 = rtlpriv->cfg->mod_params->dma64;
+ __le32 *desc = (__le32 *)desc8;
+ __le32 *tx_bd_desc = (__le32 *)tx_bd_desc8;
tx_page_size = 2;
current_bd_desc = rtlpci->tx_ring[queue_index].cur_tx_wp;
@@ -589,48 +596,48 @@ void rtl92ee_pre_fill_tx_bd_desc(struct ieee80211_hw *hw,
(current_bd_desc * TX_DESC_SIZE);
/* Reset */
- SET_TX_BUFF_DESC_LEN_0(tx_bd_desc, 0);
- SET_TX_BUFF_DESC_PSB(tx_bd_desc, 0);
- SET_TX_BUFF_DESC_OWN(tx_bd_desc, 0);
+ set_tx_buff_desc_len_0(tx_bd_desc, 0);
+ set_tx_buff_desc_psb(tx_bd_desc, 0);
+ set_tx_buff_desc_own(tx_bd_desc, 0);
for (i = 1; i < segmentnum; i++) {
- SET_TXBUFFER_DESC_LEN_WITH_OFFSET(tx_bd_desc, i, 0);
- SET_TXBUFFER_DESC_AMSDU_WITH_OFFSET(tx_bd_desc, i, 0);
- SET_TXBUFFER_DESC_ADD_LOW_WITH_OFFSET(tx_bd_desc, i, 0);
- SET_TXBUFFER_DESC_ADD_HIGH_WITH_OFFSET(tx_bd_desc, i, 0, dma64);
+ set_txbuffer_desc_len_with_offset(tx_bd_desc, i, 0);
+ set_txbuffer_desc_amsdu_with_offset(tx_bd_desc, i, 0);
+ set_txbuffer_desc_add_low_with_offset(tx_bd_desc, i, 0);
+ set_txbuffer_desc_add_high_with_offset(tx_bd_desc, i, 0, dma64);
}
/* Clear all status */
- CLEAR_PCI_TX_DESC_CONTENT(desc, TX_DESC_SIZE);
+ clear_pci_tx_desc_content(desc, TX_DESC_SIZE);
if (rtlpriv->rtlhal.earlymode_enable) {
if (queue_index < BEACON_QUEUE) {
/* This if needs braces */
- SET_TX_BUFF_DESC_LEN_0(tx_bd_desc, desc_size + 8);
+ set_tx_buff_desc_len_0(tx_bd_desc, desc_size + 8);
} else {
- SET_TX_BUFF_DESC_LEN_0(tx_bd_desc, desc_size);
+ set_tx_buff_desc_len_0(tx_bd_desc, desc_size);
}
} else {
- SET_TX_BUFF_DESC_LEN_0(tx_bd_desc, desc_size);
+ set_tx_buff_desc_len_0(tx_bd_desc, desc_size);
}
- SET_TX_BUFF_DESC_PSB(tx_bd_desc, psblen);
- SET_TX_BUFF_DESC_ADDR_LOW_0(tx_bd_desc, desc_dma_addr);
- SET_TX_BUFF_DESC_ADDR_HIGH_0(tx_bd_desc, ((u64)desc_dma_addr >> 32),
+ set_tx_buff_desc_psb(tx_bd_desc, psblen);
+ set_tx_buff_desc_addr_low_0(tx_bd_desc, desc_dma_addr);
+ set_tx_buff_desc_addr_high_0(tx_bd_desc, ((u64)desc_dma_addr >> 32),
dma64);
- SET_TXBUFFER_DESC_LEN_WITH_OFFSET(tx_bd_desc, 1, pkt_len);
+ set_txbuffer_desc_len_with_offset(tx_bd_desc, 1, pkt_len);
/* don't using extendsion mode. */
- SET_TXBUFFER_DESC_AMSDU_WITH_OFFSET(tx_bd_desc, 1, 0);
- SET_TXBUFFER_DESC_ADD_LOW_WITH_OFFSET(tx_bd_desc, 1, addr);
- SET_TXBUFFER_DESC_ADD_HIGH_WITH_OFFSET(tx_bd_desc, 1,
+ set_txbuffer_desc_amsdu_with_offset(tx_bd_desc, 1, 0);
+ set_txbuffer_desc_add_low_with_offset(tx_bd_desc, 1, addr);
+ set_txbuffer_desc_add_high_with_offset(tx_bd_desc, 1,
((u64)addr >> 32), dma64);
- SET_TX_DESC_PKT_SIZE(desc, (u16)(pkt_len));
- SET_TX_DESC_TX_BUFFER_SIZE(desc, (u16)(pkt_len));
+ set_tx_desc_pkt_size(desc, (u16)(pkt_len));
+ set_tx_desc_tx_buffer_size(desc, (u16)(pkt_len));
}
void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
- struct ieee80211_hdr *hdr, u8 *pdesc_tx,
+ struct ieee80211_hdr *hdr, u8 *pdesc8,
u8 *pbd_desc_tx,
struct ieee80211_tx_info *info,
struct ieee80211_sta *sta,
@@ -642,10 +649,9 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
struct rtl_pci *rtlpci = rtl_pcidev(rtl_pcipriv(hw));
struct rtl_hal *rtlhal = rtl_hal(rtlpriv);
struct rtlwifi_tx_info *tx_info = rtl_tx_skb_cb_info(skb);
- u8 *pdesc = (u8 *)pdesc_tx;
u16 seq_number;
__le16 fc = hdr->frame_control;
- unsigned int buf_len = 0;
+ unsigned int buf_len;
u8 fw_qsel = _rtl92ee_map_hwqueue_to_fwqueue(skb, hw_queue);
bool firstseg = ((hdr->seq_ctrl &
cpu_to_le16(IEEE80211_SCTL_FRAG)) == 0);
@@ -653,7 +659,8 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
cpu_to_le16(IEEE80211_FCTL_MOREFRAGS)) == 0);
dma_addr_t mapping;
u8 bw_40 = 0;
- u8 short_gi = 0;
+ u8 short_gi;
+ __le32 *pdesc = (__le32 *)pdesc8;
if (mac->opmode == NL80211_IFTYPE_STATION) {
bw_40 = mac->bw_40;
@@ -680,7 +687,7 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
}
if (pbd_desc_tx != NULL)
- rtl92ee_pre_fill_tx_bd_desc(hw, pbd_desc_tx, pdesc, hw_queue,
+ rtl92ee_pre_fill_tx_bd_desc(hw, pbd_desc_tx, pdesc8, hw_queue,
skb, mapping);
if (ieee80211_is_nullfunc(fc) || ieee80211_is_ctl(fc)) {
@@ -689,8 +696,8 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
}
if (firstseg) {
if (rtlhal->earlymode_enable) {
- SET_TX_DESC_PKT_OFFSET(pdesc, 1);
- SET_TX_DESC_OFFSET(pdesc,
+ set_tx_desc_pkt_offset(pdesc, 1);
+ set_tx_desc_offset(pdesc,
USB_HWDESC_HEADER_LEN + EM_HDR_LEN);
if (ptcb_desc->empkt_num) {
RT_TRACE(rtlpriv, COMP_SEND, DBG_TRACE,
@@ -700,18 +707,18 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
(u8 *)(skb->data));
}
} else {
- SET_TX_DESC_OFFSET(pdesc, USB_HWDESC_HEADER_LEN);
+ set_tx_desc_offset(pdesc, USB_HWDESC_HEADER_LEN);
}
- SET_TX_DESC_TX_RATE(pdesc, ptcb_desc->hw_rate);
+ set_tx_desc_tx_rate(pdesc, ptcb_desc->hw_rate);
if (ieee80211_is_mgmt(fc)) {
ptcb_desc->use_driver_rate = true;
} else {
if (rtlpriv->ra.is_special_data) {
ptcb_desc->use_driver_rate = true;
- SET_TX_DESC_TX_RATE(pdesc, DESC_RATE11M);
+ set_tx_desc_tx_rate(pdesc, DESC_RATE11M);
} else {
ptcb_desc->use_driver_rate = false;
}
@@ -723,46 +730,46 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
short_gi = (ptcb_desc->use_shortpreamble) ? 1 : 0;
if (info->flags & IEEE80211_TX_CTL_AMPDU) {
- SET_TX_DESC_AGG_ENABLE(pdesc, 1);
- SET_TX_DESC_MAX_AGG_NUM(pdesc, 0x14);
+ set_tx_desc_agg_enable(pdesc, 1);
+ set_tx_desc_max_agg_num(pdesc, 0x14);
}
- SET_TX_DESC_SEQ(pdesc, seq_number);
- SET_TX_DESC_RTS_ENABLE(pdesc,
+ set_tx_desc_seq(pdesc, seq_number);
+ set_tx_desc_rts_enable(pdesc,
((ptcb_desc->rts_enable &&
!ptcb_desc->cts_enable) ? 1 : 0));
- SET_TX_DESC_HW_RTS_ENABLE(pdesc, 0);
- SET_TX_DESC_CTS2SELF(pdesc,
+ set_tx_desc_hw_rts_enable(pdesc, 0);
+ set_tx_desc_cts2self(pdesc,
((ptcb_desc->cts_enable) ? 1 : 0));
- SET_TX_DESC_RTS_RATE(pdesc, ptcb_desc->rts_rate);
- SET_TX_DESC_RTS_SC(pdesc, ptcb_desc->rts_sc);
- SET_TX_DESC_RTS_SHORT(pdesc,
+ set_tx_desc_rts_rate(pdesc, ptcb_desc->rts_rate);
+ set_tx_desc_rts_sc(pdesc, ptcb_desc->rts_sc);
+ set_tx_desc_rts_short(pdesc,
((ptcb_desc->rts_rate <= DESC_RATE54M) ?
(ptcb_desc->rts_use_shortpreamble ? 1 : 0) :
(ptcb_desc->rts_use_shortgi ? 1 : 0)));
if (ptcb_desc->tx_enable_sw_calc_duration)
- SET_TX_DESC_NAV_USE_HDR(pdesc, 1);
+ set_tx_desc_nav_use_hdr(pdesc, 1);
if (bw_40) {
if (ptcb_desc->packet_bw == HT_CHANNEL_WIDTH_20_40) {
- SET_TX_DESC_DATA_BW(pdesc, 1);
- SET_TX_DESC_TX_SUB_CARRIER(pdesc, 3);
+ set_tx_desc_data_bw(pdesc, 1);
+ set_tx_desc_tx_sub_carrier(pdesc, 3);
} else {
- SET_TX_DESC_DATA_BW(pdesc, 0);
- SET_TX_DESC_TX_SUB_CARRIER(pdesc,
+ set_tx_desc_data_bw(pdesc, 0);
+ set_tx_desc_tx_sub_carrier(pdesc,
mac->cur_40_prime_sc);
}
} else {
- SET_TX_DESC_DATA_BW(pdesc, 0);
- SET_TX_DESC_TX_SUB_CARRIER(pdesc, 0);
+ set_tx_desc_data_bw(pdesc, 0);
+ set_tx_desc_tx_sub_carrier(pdesc, 0);
}
- SET_TX_DESC_LINIP(pdesc, 0);
+ set_tx_desc_linip(pdesc, 0);
if (sta) {
u8 ampdu_density = sta->ht_cap.ampdu_density;
- SET_TX_DESC_AMPDU_DENSITY(pdesc, ampdu_density);
+ set_tx_desc_ampdu_density(pdesc, ampdu_density);
}
if (info->control.hw_key) {
struct ieee80211_key_conf *key = info->control.hw_key;
@@ -771,65 +778,65 @@ void rtl92ee_tx_fill_desc(struct ieee80211_hw *hw,
case WLAN_CIPHER_SUITE_WEP40:
case WLAN_CIPHER_SUITE_WEP104:
case WLAN_CIPHER_SUITE_TKIP:
- SET_TX_DESC_SEC_TYPE(pdesc, 0x1);
+ set_tx_desc_sec_type(pdesc, 0x1);
break;
case WLAN_CIPHER_SUITE_CCMP:
- SET_TX_DESC_SEC_TYPE(pdesc, 0x3);
+ set_tx_desc_sec_type(pdesc, 0x3);
break;
default:
- SET_TX_DESC_SEC_TYPE(pdesc, 0x0);
+ set_tx_desc_sec_type(pdesc, 0x0);
break;
}
}
- SET_TX_DESC_QUEUE_SEL(pdesc, fw_qsel);
- SET_TX_DESC_DATA_RATE_FB_LIMIT(pdesc, 0x1F);
- SET_TX_DESC_RTS_RATE_FB_LIMIT(pdesc, 0xF);
- SET_TX_DESC_DISABLE_FB(pdesc,
+ set_tx_desc_queue_sel(pdesc, fw_qsel);
+ set_tx_desc_data_rate_fb_limit(pdesc, 0x1F);
+ set_tx_desc_rts_rate_fb_limit(pdesc, 0xF);
+ set_tx_desc_disable_fb(pdesc,
ptcb_desc->disable_ratefallback ? 1 : 0);
- SET_TX_DESC_USE_RATE(pdesc, ptcb_desc->use_driver_rate ? 1 : 0);
+ set_tx_desc_use_rate(pdesc, ptcb_desc->use_driver_rate ? 1 : 0);
- /*SET_TX_DESC_PWR_STATUS(pdesc, pwr_status);*/
+ /*set_tx_desc_pwr_status(pdesc, pwr_status);*/
/* Set TxRate and RTSRate in TxDesc */
/* This prevent Tx initial rate of new-coming packets */
/* from being overwritten by retried packet rate.*/
if (!ptcb_desc->use_driver_rate) {
- /*SET_TX_DESC_RTS_RATE(pdesc, 0x08); */
- /* SET_TX_DESC_TX_RATE(pdesc, 0x0b); */
+ /*set_tx_desc_rts_rate(pdesc, 0x08); */
+ /* set_tx_desc_tx_rate(pdesc, 0x0b); */
}
if (ieee80211_is_data_qos(fc)) {
if (mac->rdg_en) {
RT_TRACE(rtlpriv, COMP_SEND, DBG_TRACE,
"Enable RDG function.\n");
- SET_TX_DESC_RDG_ENABLE(pdesc, 1);
- SET_TX_DESC_HTC(pdesc, 1);
+ set_tx_desc_rdg_enable(pdesc, 1);
+ set_tx_desc_htc(pdesc, 1);
}
}
/* tx report */
- rtl_set_tx_report(ptcb_desc, pdesc, hw, tx_info);
+ rtl_set_tx_report(ptcb_desc, pdesc8, hw, tx_info);
}
- SET_TX_DESC_FIRST_SEG(pdesc, (firstseg ? 1 : 0));
- SET_TX_DESC_LAST_SEG(pdesc, (lastseg ? 1 : 0));
- SET_TX_DESC_TX_BUFFER_ADDRESS(pdesc, mapping);
+ set_tx_desc_first_seg(pdesc, (firstseg ? 1 : 0));
+ set_tx_desc_last_seg(pdesc, (lastseg ? 1 : 0));
+ set_tx_desc_tx_buffer_address(pdesc, mapping);
if (rtlpriv->dm.useramask) {
- SET_TX_DESC_RATE_ID(pdesc, ptcb_desc->ratr_index);
- SET_TX_DESC_MACID(pdesc, ptcb_desc->mac_id);
+ set_tx_desc_rate_id(pdesc, ptcb_desc->ratr_index);
+ set_tx_desc_macid(pdesc, ptcb_desc->mac_id);
} else {
- SET_TX_DESC_RATE_ID(pdesc, 0xC + ptcb_desc->ratr_index);
- SET_TX_DESC_MACID(pdesc, ptcb_desc->ratr_index);
+ set_tx_desc_rate_id(pdesc, 0xC + ptcb_desc->ratr_index);
+ set_tx_desc_macid(pdesc, ptcb_desc->ratr_index);
}
- SET_TX_DESC_MORE_FRAG(pdesc, (lastseg ? 0 : 1));
+ set_tx_desc_more_frag(pdesc, (lastseg ? 0 : 1));
if (is_multicast_ether_addr(ieee80211_get_DA(hdr)) ||
is_broadcast_ether_addr(ieee80211_get_DA(hdr))) {
- SET_TX_DESC_BMC(pdesc, 1);
+ set_tx_desc_bmc(pdesc, 1);
}
RT_TRACE(rtlpriv, COMP_SEND, DBG_TRACE, "\n");
}
void rtl92ee_tx_fill_cmddesc(struct ieee80211_hw *hw,
- u8 *pdesc, bool firstseg,
+ u8 *pdesc8, bool firstseg,
bool lastseg, struct sk_buff *skb)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
@@ -839,61 +846,63 @@ void rtl92ee_tx_fill_cmddesc(struct ieee80211_hw *hw,
skb->data, skb->len,
PCI_DMA_TODEVICE);
u8 txdesc_len = 40;
+ __le32 *pdesc = (__le32 *)pdesc8;
if (pci_dma_mapping_error(rtlpci->pdev, mapping)) {
RT_TRACE(rtlpriv, COMP_SEND, DBG_TRACE,
"DMA mapping error\n");
return;
}
- CLEAR_PCI_TX_DESC_CONTENT(pdesc, txdesc_len);
+ clear_pci_tx_desc_content(pdesc, txdesc_len);
if (firstseg)
- SET_TX_DESC_OFFSET(pdesc, txdesc_len);
+ set_tx_desc_offset(pdesc, txdesc_len);
- SET_TX_DESC_TX_RATE(pdesc, DESC_RATE1M);
+ set_tx_desc_tx_rate(pdesc, DESC_RATE1M);
- SET_TX_DESC_SEQ(pdesc, 0);
+ set_tx_desc_seq(pdesc, 0);
- SET_TX_DESC_LINIP(pdesc, 0);
+ set_tx_desc_linip(pdesc, 0);
- SET_TX_DESC_QUEUE_SEL(pdesc, fw_queue);
+ set_tx_desc_queue_sel(pdesc, fw_queue);
- SET_TX_DESC_FIRST_SEG(pdesc, 1);
- SET_TX_DESC_LAST_SEG(pdesc, 1);
+ set_tx_desc_first_seg(pdesc, 1);
+ set_tx_desc_last_seg(pdesc, 1);
- SET_TX_DESC_TX_BUFFER_SIZE(pdesc, (u16)(skb->len));
+ set_tx_desc_tx_buffer_size(pdesc, (u16)(skb->len));
- SET_TX_DESC_TX_BUFFER_ADDRESS(pdesc, mapping);
+ set_tx_desc_tx_buffer_address(pdesc, mapping);
- SET_TX_DESC_RATE_ID(pdesc, 7);
- SET_TX_DESC_MACID(pdesc, 0);
+ set_tx_desc_rate_id(pdesc, 7);
+ set_tx_desc_macid(pdesc, 0);
- SET_TX_DESC_OWN(pdesc, 1);
+ set_tx_desc_own(pdesc, 1);
- SET_TX_DESC_PKT_SIZE((u8 *)pdesc, (u16)(skb->len));
+ set_tx_desc_pkt_size(pdesc, (u16)(skb->len));
- SET_TX_DESC_FIRST_SEG(pdesc, 1);
- SET_TX_DESC_LAST_SEG(pdesc, 1);
+ set_tx_desc_first_seg(pdesc, 1);
+ set_tx_desc_last_seg(pdesc, 1);
- SET_TX_DESC_OFFSET(pdesc, 40);
+ set_tx_desc_offset(pdesc, 40);
- SET_TX_DESC_USE_RATE(pdesc, 1);
+ set_tx_desc_use_rate(pdesc, 1);
RT_PRINT_DATA(rtlpriv, COMP_CMD, DBG_LOUD,
"H2C Tx Cmd Content\n", pdesc, txdesc_len);
}
-void rtl92ee_set_desc(struct ieee80211_hw *hw, u8 *pdesc, bool istx,
+void rtl92ee_set_desc(struct ieee80211_hw *hw, u8 *pdesc8, bool istx,
u8 desc_name, u8 *val)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
u8 q_idx = *val;
bool dma64 = rtlpriv->cfg->mod_params->dma64;
+ __le32 *pdesc = (__le32 *)pdesc8;
if (istx) {
switch (desc_name) {
case HW_DESC_TX_NEXTDESC_ADDR:
- SET_TX_DESC_NEXT_DESC_ADDRESS(pdesc, *(u32 *)val);
+ set_tx_desc_next_desc_address(pdesc, *(u32 *)val);
break;
case HW_DESC_OWN:{
struct rtl_pci *rtlpci = rtl_pcidev(rtl_pcipriv(hw));
@@ -903,7 +912,7 @@ void rtl92ee_set_desc(struct ieee80211_hw *hw, u8 *pdesc, bool istx,
if (q_idx == BEACON_QUEUE) {
ring->cur_tx_wp = 0;
ring->cur_tx_rp = 0;
- SET_TX_BUFF_DESC_OWN(pdesc, 1);
+ set_tx_buff_desc_own(pdesc, 1);
return;
}
@@ -919,23 +928,23 @@ void rtl92ee_set_desc(struct ieee80211_hw *hw, u8 *pdesc, bool istx,
} else {
switch (desc_name) {
case HW_DESC_RX_PREPARE:
- SET_RX_BUFFER_DESC_LS(pdesc, 0);
- SET_RX_BUFFER_DESC_FS(pdesc, 0);
- SET_RX_BUFFER_DESC_TOTAL_LENGTH(pdesc, 0);
+ set_rx_buffer_desc_ls(pdesc, 0);
+ set_rx_buffer_desc_fs(pdesc, 0);
+ set_rx_buffer_desc_total_length(pdesc, 0);
- SET_RX_BUFFER_DESC_DATA_LENGTH(pdesc,
+ set_rx_buffer_desc_data_length(pdesc,
MAX_RECEIVE_BUFFER_SIZE +
RX_DESC_SIZE);
- SET_RX_BUFFER_PHYSICAL_LOW(pdesc, (*(dma_addr_t *)val) &
+ set_rx_buffer_physical_low(pdesc, (*(dma_addr_t *)val) &
DMA_BIT_MASK(32));
- SET_RX_BUFFER_PHYSICAL_HIGH(pdesc,
+ set_rx_buffer_physical_high(pdesc,
((u64)(*(dma_addr_t *)val)
>> 32),
dma64);
break;
case HW_DESC_RXERO:
- SET_RX_DESC_EOR(pdesc, 1);
+ set_rx_desc_eor(pdesc, 1);
break;
default:
WARN_ONCE(true,
@@ -947,20 +956,21 @@ void rtl92ee_set_desc(struct ieee80211_hw *hw, u8 *pdesc, bool istx,
}
u64 rtl92ee_get_desc(struct ieee80211_hw *hw,
- u8 *pdesc, bool istx, u8 desc_name)
+ u8 *pdesc8, bool istx, u8 desc_name)
{
struct rtl_priv *rtlpriv = rtl_priv(hw);
u64 ret = 0;
bool dma64 = rtlpriv->cfg->mod_params->dma64;
+ __le32 *pdesc = (__le32 *)pdesc8;
if (istx) {
switch (desc_name) {
case HW_DESC_OWN:
- ret = GET_TX_DESC_OWN(pdesc);
+ ret = get_tx_desc_own(pdesc);
break;
case HW_DESC_TXBUFF_ADDR:
- ret = GET_TXBUFFER_DESC_ADDR_LOW(pdesc, 1);
- ret |= (u64)GET_TXBUFFER_DESC_ADDR_HIGH(pdesc, 1,
+ ret = get_txbuffer_desc_addr_low(pdesc, 1);
+ ret |= (u64)get_txbuffer_desc_addr_high(pdesc, 1,
dma64) << 32;
break;
default:
@@ -972,13 +982,13 @@ u64 rtl92ee_get_desc(struct ieee80211_hw *hw,
} else {
switch (desc_name) {
case HW_DESC_OWN:
- ret = GET_RX_DESC_OWN(pdesc);
+ ret = get_rx_desc_own(pdesc);
break;
case HW_DESC_RXPKT_LEN:
- ret = GET_RX_DESC_PKT_LEN(pdesc);
+ ret = get_rx_desc_pkt_len(pdesc);
break;
case HW_DESC_RXBUFF_ADDR:
- ret = GET_RX_DESC_BUFF_ADDR(pdesc);
+ ret = get_rx_desc_buff_addr(pdesc);
break;
default:
WARN_ONCE(true,
@@ -1001,7 +1011,7 @@ bool rtl92ee_is_tx_desc_closed(struct ieee80211_hw *hw, u8 hw_queue, u16 index)
{
u16 cur_tx_rp, cur_tx_wp;
- u32 tmpu32 = 0;
+ u32 tmpu32;
tmpu32 =
rtl_read_dword(rtlpriv,
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.h
index a9e5e62..967cef3 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.h
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192ee/trx.h
@@ -14,466 +14,471 @@
#define RX_DESC_SIZE 24
#define MAX_RECEIVE_BUFFER_SIZE 8192
-#define SET_TX_DESC_PKT_SIZE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 0, 16, __val)
-#define SET_TX_DESC_OFFSET(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 16, 8, __val)
-#define SET_TX_DESC_BMC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 24, 1, __val)
-#define SET_TX_DESC_HTC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 25, 1, __val)
-#define SET_TX_DESC_LAST_SEG(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 26, 1, __val)
-#define SET_TX_DESC_FIRST_SEG(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 27, 1, __val)
-#define SET_TX_DESC_LINIP(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 28, 1, __val)
-#define SET_TX_DESC_NO_ACM(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 29, 1, __val)
-#define SET_TX_DESC_GF(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 30, 1, __val)
-#define SET_TX_DESC_OWN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 31, 1, __val)
+static inline void set_tx_desc_pkt_size(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, GENMASK(15, 0));
+}
-#define GET_TX_DESC_PKT_SIZE(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 0, 16)
-#define GET_TX_DESC_OFFSET(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 16, 8)
-#define GET_TX_DESC_BMC(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 24, 1)
-#define GET_TX_DESC_HTC(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 25, 1)
-#define GET_TX_DESC_LAST_SEG(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 26, 1)
-#define GET_TX_DESC_FIRST_SEG(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 27, 1)
-#define GET_TX_DESC_LINIP(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 28, 1)
-#define GET_TX_DESC_NO_ACM(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 29, 1)
-#define GET_TX_DESC_GF(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 30, 1)
-#define GET_TX_DESC_OWN(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 31, 1)
+static inline void set_tx_desc_offset(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, GENMASK(23, 16));
+}
-#define SET_TX_DESC_MACID(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 0, 7, __val)
-#define SET_TX_DESC_QUEUE_SEL(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 8, 5, __val)
-#define SET_TX_DESC_RDG_NAV_EXT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 13, 1, __val)
-#define SET_TX_DESC_LSIG_TXOP_EN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 14, 1, __val)
-#define SET_TX_DESC_PIFS(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 15, 1, __val)
-#define SET_TX_DESC_RATE_ID(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 16, 5, __val)
-#define SET_TX_DESC_EN_DESC_ID(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 21, 1, __val)
-#define SET_TX_DESC_SEC_TYPE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 22, 2, __val)
-#define SET_TX_DESC_PKT_OFFSET(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 24, 5, __val)
-#define SET_TX_DESC_MORE_DATA(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 29, 1, __val)
-#define SET_TX_DESC_TXOP_PS_CAP(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 30, 1, __val)
-#define SET_TX_DESC_TXOP_PS_MODE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+4, 31, 1, __val)
+static inline void set_tx_desc_bmc(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(24));
+}
-#define GET_TX_DESC_MACID(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 0, 5)
-#define GET_TX_DESC_AGG_ENABLE(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 5, 1)
-#define GET_TX_DESC_AGG_BREAK(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 6, 1)
-#define GET_TX_DESC_RDG_ENABLE(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 7, 1)
-#define GET_TX_DESC_QUEUE_SEL(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 8, 5)
-#define GET_TX_DESC_RDG_NAV_EXT(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 13, 1)
-#define GET_TX_DESC_LSIG_TXOP_EN(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 14, 1)
-#define GET_TX_DESC_PIFS(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 15, 1)
-#define GET_TX_DESC_RATE_ID(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 16, 4)
-#define GET_TX_DESC_NAV_USE_HDR(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 20, 1)
-#define GET_TX_DESC_EN_DESC_ID(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 21, 1)
-#define GET_TX_DESC_SEC_TYPE(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 22, 2)
-#define GET_TX_DESC_PKT_OFFSET(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 24, 5)
+static inline void set_tx_desc_htc(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(25));
+}
-#define SET_TX_DESC_PAID(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 0, 9, __val)
-#define SET_TX_DESC_CCA_RTS(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 10, 2, __val)
-#define SET_TX_DESC_AGG_ENABLE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 12, 1, __val)
-#define SET_TX_DESC_RDG_ENABLE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 13, 1, __val)
-#define SET_TX_DESC_NULL_0(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 14, 1, __val)
-#define SET_TX_DESC_NULL_1(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 8, 15, 1, __val)
-#define SET_TX_DESC_BK(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 16, 1, __val)
-#define SET_TX_DESC_MORE_FRAG(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 17, 1, __val)
-#define SET_TX_DESC_RAW(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 18, 1, __val)
-#define SET_TX_DESC_SPE_RPT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 8, 19, 1, __val)
-#define SET_TX_DESC_AMPDU_DENSITY(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 20, 3, __val)
-#define SET_TX_DESC_BT_NULL(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 23, 1, __val)
-#define SET_TX_DESC_GID(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+8, 24, 6, __val)
+static inline void set_tx_desc_last_seg(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(26));
+}
-#define SET_TX_DESC_WHEADER_LEN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 0, 4, __val)
-#define SET_TX_DESC_CHK_EN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 4, 1, __val)
-#define SET_TX_DESC_EARLY_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 5, 1, __val)
-#define SET_TX_DESC_HWSEQ_SEL(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 6, 2, __val)
-#define SET_TX_DESC_USE_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 8, 1, __val)
-#define SET_TX_DESC_DISABLE_RTS_FB(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 9, 1, __val)
-#define SET_TX_DESC_DISABLE_FB(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 10, 1, __val)
-#define SET_TX_DESC_CTS2SELF(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 11, 1, __val)
-#define SET_TX_DESC_RTS_ENABLE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 12, 1, __val)
-#define SET_TX_DESC_HW_RTS_ENABLE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 13, 1, __val)
-#define SET_TX_DESC_HW_PORT_ID(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 14, 1, __val)
-#define SET_TX_DESC_NAV_USE_HDR(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 15, 1, __val)
-#define SET_TX_DESC_USE_MAX_LEN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 16, 1, __val)
-#define SET_TX_DESC_MAX_AGG_NUM(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 17, 5, __val)
-#define SET_TX_DESC_NDPA(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 22, 2, __val)
-#define SET_TX_DESC_AMPDU_MAX_TIME(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+12, 24, 8, __val)
+static inline void set_tx_desc_first_seg(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(27));
+}
+
+static inline void set_tx_desc_linip(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(28));
+}
+
+static inline void set_tx_desc_own(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(31));
+}
+
+static inline int get_tx_desc_own(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc), BIT(31));
+}
+
+static inline void set_tx_desc_macid(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 1), __val, GENMASK(6, 0));
+}
+
+static inline void set_tx_desc_queue_sel(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 1), __val, GENMASK(12, 8));
+}
+
+static inline void set_tx_desc_rate_id(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 1), __val, GENMASK(20, 16));
+}
+
+static inline void set_tx_desc_sec_type(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 1), __val, GENMASK(23, 22));
+}
+
+static inline void set_tx_desc_pkt_offset(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 1), __val, GENMASK(28, 24));
+}
+
+static inline void set_tx_desc_agg_enable(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 2), __val, BIT(12));
+}
+
+static inline void set_tx_desc_rdg_enable(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 2), __val, BIT(13));
+}
+
+static inline void set_tx_desc_more_frag(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 2), __val, BIT(17));
+}
+
+static inline void set_tx_desc_ampdu_density(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 2), __val, GENMASK(22, 20));
+}
+
+static inline void set_tx_desc_use_rate(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, BIT(8));
+}
+
+static inline void set_tx_desc_disable_fb(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, BIT(10));
+}
+
+static inline void set_tx_desc_cts2self(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, BIT(11));
+}
+
+static inline void set_tx_desc_rts_enable(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, BIT(12));
+}
+
+static inline void set_tx_desc_hw_rts_enable(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, BIT(13));
+}
+
+static inline void set_tx_desc_nav_use_hdr(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, BIT(15));
+}
+
+static inline void set_tx_desc_max_agg_num(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 3), __val, GENMASK(21, 17));
+}
/* Dword 4 */
-#define SET_TX_DESC_TX_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 0, 7, __val)
-#define SET_TX_DESC_TRY_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 7, 1, __val)
-#define SET_TX_DESC_DATA_RATE_FB_LIMIT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 8, 5, __val)
-#define SET_TX_DESC_RTS_RATE_FB_LIMIT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 13, 4, __val)
-#define SET_TX_DESC_RETRY_LIMIT_ENABLE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 17, 1, __val)
-#define SET_TX_DESC_DATA_RETRY_LIMIT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 18, 6, __val)
-#define SET_TX_DESC_RTS_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 24, 5, __val)
-#define SET_TX_DESC_PCTS_ENABLE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 29, 1, __val)
-#define SET_TX_DESC_PCTS_MASK_IDX(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+16, 30, 2, __val)
+static inline void set_tx_desc_tx_rate(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4), __val, GENMASK(6, 0));
+}
+
+static inline void set_tx_desc_data_rate_fb_limit(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4), __val, GENMASK(12, 8));
+}
+
+static inline void set_tx_desc_rts_rate_fb_limit(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4), __val, GENMASK(16, 13));
+}
+
+static inline void set_tx_desc_rts_rate(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4), __val, GENMASK(28, 24));
+}
/* Dword 5 */
-#define SET_TX_DESC_TX_SUB_CARRIER(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 0, 4, __val)
-#define SET_TX_DESC_DATA_SHORT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 4, 1, __val)
-#define SET_TX_DESC_DATA_BW(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 5, 2, __val)
-#define SET_TX_DESC_DATA_LDPC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 7, 1, __val)
-#define SET_TX_DESC_DATA_STBC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 8, 2, __val)
-#define SET_TX_DESC_VCS_STBC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 10, 2, __val)
-#define SET_TX_DESC_RTS_SHORT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 12, 1, __val)
-#define SET_TX_DESC_RTS_SC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 13, 4, __val)
-#define SET_TX_DESC_TX_ANT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 24, 4, __val)
-#define SET_TX_DESC_TX_POWER_0_PSET(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+20, 28, 3, __val)
+static inline void set_tx_desc_tx_sub_carrier(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 5), __val, GENMASK(3, 0));
+}
-/* Dword 6 */
-#define SET_TX_DESC_SW_DEFINE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 24, 0, 12, __val)
-#define SET_TX_DESC_ANTSEL_A(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 24, 16, 3, __val)
-#define SET_TX_DESC_ANTSEL_B(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 24, 19, 3, __val)
-#define SET_TX_DESC_ANTSEL_C(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 24, 22, 3, __val)
-#define SET_TX_DESC_ANTSEL_D(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 24, 25, 3, __val)
+static inline void set_tx_desc_data_bw(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4), __val, GENMASK(6, 5));
+}
+
+static inline void set_tx_desc_rts_short(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 5), __val, BIT(12));
+}
+
+static inline void set_tx_desc_rts_sc(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 5), __val, GENMASK(16, 13));
+}
/* Dword 7 */
-#define SET_TX_DESC_TX_BUFFER_SIZE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+28, 0, 16, __val)
-#define SET_TX_DESC_USB_TXAGG_NUM(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+28, 24, 8, __val)
-
-/* Dword 8 */
-#define SET_TX_DESC_RTS_RC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+32, 0, 6, __val)
-#define SET_TX_DESC_BAR_RTY_TH(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+32, 6, 2, __val)
-#define SET_TX_DESC_DATA_RC(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+32, 8, 6, __val)
-#define SET_TX_DESC_ENABLE_HW_SELECT(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+32, 15, 1, __val)
-#define SET_TX_DESC_NEXT_HEAD_PAGE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+32, 16, 8, __val)
-#define SET_TX_DESC_TAIL_PAGE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+32, 24, 8, __val)
+static inline void set_tx_desc_tx_buffer_size(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 7), __val, GENMASK(15, 0));
+}
/* Dword 9 */
-#define SET_TX_DESC_PADDING_LENGTH(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+36, 0, 11, __val)
-#define SET_TX_DESC_TXBF_PATH(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+36, 11, 1, __val)
-#define SET_TX_DESC_SEQ(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+36, 12, 12, __val)
-#define SET_TX_DESC_FINAL_DATA_RATE(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+36, 24, 8, __val)
+static inline void set_tx_desc_seq(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 9), __val, GENMASK(23, 12));
+}
/* Dword 10 */
-#define SET_TX_DESC_TX_BUFFER_ADDRESS(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+40, 0, 32, __val)
+static inline void set_tx_desc_tx_buffer_address(__le32 *__pdesc, u32 __val)
+{
+ *(__pdesc + 10) = cpu_to_le32(__val);
+}
/* Dword 11*/
-#define SET_TX_DESC_NEXT_DESC_ADDRESS(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+48, 0, 32, __val)
+static inline void set_tx_desc_next_desc_address(__le32 *__pdesc, u32 __val)
+{
+ *(__pdesc + 12) = cpu_to_le32(__val);
+}
-#define SET_EARLYMODE_PKTNUM(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr, 0, 4, __val)
-#define SET_EARLYMODE_LEN0(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr, 4, 15, __val)
-#define SET_EARLYMODE_LEN1(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr, 16, 2, __val)
-#define SET_EARLYMODE_LEN1_1(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr, 19, 13, __val)
-#define SET_EARLYMODE_LEN1_2(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr+4, 0, 2, __val)
-#define SET_EARLYMODE_LEN2(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr+4, 2, 15, __val)
-#define SET_EARLYMODE_LEN2_1(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr, 2, 4, __val)
-#define SET_EARLYMODE_LEN2_2(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr+4, 0, 8, __val)
-#define SET_EARLYMODE_LEN3(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr+4, 17, 15, __val)
-#define SET_EARLYMODE_LEN4(__paddr, __val) \
- SET_BITS_TO_LE_4BYTE(__paddr+4, 20, 12, __val)
+static inline void set_earlymode_pktnum(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits(__paddr, __val, GENMASK(3, 0));
+}
+
+static inline void set_earlymode_len0(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits(__paddr, __val, GENMASK(18, 4));
+}
+
+static inline void set_earlymode_len1(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits(__paddr, __val, GENMASK(17, 16));
+}
+
+static inline void set_earlymode_len2_1(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits(__paddr, __val, GENMASK(5, 2));
+}
+
+static inline void set_earlymode_len2_2(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits((__paddr + 1), __val, GENMASK(7, 0));
+}
+
+static inline void set_earlymode_len3(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits((__paddr + 1), __val, GENMASK(31, 17));
+}
+
+static inline void set_earlymode_len4(__le32 *__paddr, u32 __val)
+{
+ le32p_replace_bits((__paddr + 1), __val, GENMASK(31, 20));
+}
/* TX/RX buffer descriptor */
-#define SET_TX_EXTBUFF_DESC_LEN(__pdesc, __val, __set) \
- SET_BITS_TO_LE_4BYTE(__pdesc+(__set*16), 0, 16, __val)
-#define SET_TX_EXTBUFF_DESC_ADDR_LOW(__pdesc, __val, __set)\
- SET_BITS_TO_LE_4BYTE(__pdesc+(__set*16)+4, 0, 32, __val)
-#define SET_TX_EXTBUFF_DESC_ADDR_HIGH(__pdesc, __val, __set)\
- SET_BITS_TO_LE_4BYTE(__pdesc+(__set*16)+8, 0, 32, __val)
-
/* for Txfilldescroptor92ee, fill the desc content. */
-#define SET_TXBUFFER_DESC_LEN_WITH_OFFSET(__pdesc, __offset, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + ((__offset) * 16), 0, 16, __val)
-#define SET_TXBUFFER_DESC_AMSDU_WITH_OFFSET(__pdesc, __offset, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + ((__offset) * 16), 31, 1, __val)
-#define SET_TXBUFFER_DESC_ADD_LOW_WITH_OFFSET(__pdesc, __offset, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + ((__offset) * 16) + 4, 0, 32, __val)
-#define SET_TXBUFFER_DESC_ADD_HIGH_WITH_OFFSET(pbd, off, val, dma64) \
- (dma64 ? SET_BITS_TO_LE_4BYTE((pbd) + ((off) * 16) + 8, 0, 32, val) : 0)
-#define GET_TXBUFFER_DESC_ADDR_LOW(__pdesc, __offset) \
- LE_BITS_TO_4BYTE((__pdesc) + ((__offset) * 16) + 4, 0, 32)
-#define GET_TXBUFFER_DESC_ADDR_HIGH(pbd, off, dma64) \
- (dma64 ? LE_BITS_TO_4BYTE((pbd) + ((off) * 16) + 8, 0, 32) : 0)
+static inline void set_txbuffer_desc_len_with_offset(__le32 *__pdesc,
+ u8 __offset, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4 * __offset), __val,
+ GENMASK(15, 0));
+}
+
+static inline void set_txbuffer_desc_amsdu_with_offset(__le32 *__pdesc,
+ u8 __offset, u32 __val)
+{
+ le32p_replace_bits((__pdesc + 4 * __offset), __val, BIT(31));
+}
+
+static inline void set_txbuffer_desc_add_low_with_offset(__le32 *__pdesc,
+ u8 __offset,
+ u32 __val)
+{
+ *(__pdesc + 4 * __offset + 1) = cpu_to_le32(__val);
+}
+
+static inline void set_txbuffer_desc_add_high_with_offset(__le32 *pbd, u8 off,
+ u32 val, bool dma64)
+{
+ if (dma64)
+ *(pbd + 4 * off + 2) = cpu_to_le32(val);
+ else
+ *(pbd + 4 * off + 2) = 0;
+}
+
+static inline u32 get_txbuffer_desc_addr_low(__le32 *__pdesc, u8 __offset)
+{
+ return le32_to_cpu(*((__pdesc + 4 * __offset + 1)));
+}
+
+static inline u32 get_txbuffer_desc_addr_high(__le32 *pbd, u32 off, bool dma64)
+{
+ if (dma64)
+ return le32_to_cpu(*((pbd + 4 * off + 2)));
+ return 0;
+}
/* Dword 0 */
-#define SET_TX_BUFF_DESC_LEN_0(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 0, 14, __val)
-#define SET_TX_BUFF_DESC_PSB(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 16, 15, __val)
-#define SET_TX_BUFF_DESC_OWN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 31, 1, __val)
+static inline void set_tx_buff_desc_len_0(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, GENMASK(13, 0));
+}
+
+static inline void set_tx_buff_desc_psb(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, GENMASK(30, 16));
+}
+
+static inline void set_tx_buff_desc_own(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(31));
+}
/* Dword 1 */
-#define SET_TX_BUFF_DESC_ADDR_LOW_0(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE((__pdesc) + 4, 0, 32, __val)
+static inline void set_tx_buff_desc_addr_low_0(__le32 *__pdesc, u32 __val)
+{
+ *(__pdesc + 1) = cpu_to_le32(__val);
+}
+
/* Dword 2 */
-#define SET_TX_BUFF_DESC_ADDR_HIGH_0(bdesc, val, dma64) \
- SET_TXBUFFER_DESC_ADD_HIGH_WITH_OFFSET(bdesc, 0, val, dma64)
-/* Dword 3 / RESERVED 0 */
+static inline void set_tx_buff_desc_addr_high_0(__le32 *pdesc, u32 val,
+ bool dma64)
+{
+ if (dma64)
+ *(pdesc + 2) = cpu_to_le32(val);
+ else
+ *(pdesc + 2) = 0;
+}
/* RX buffer */
/* DWORD 0 */
-#define SET_RX_BUFFER_DESC_DATA_LENGTH(__status, __val) \
- SET_BITS_TO_LE_4BYTE(__status, 0, 14, __val)
-#define SET_RX_BUFFER_DESC_LS(__status, __val) \
- SET_BITS_TO_LE_4BYTE(__status, 15, 1, __val)
-#define SET_RX_BUFFER_DESC_FS(__status, __val) \
- SET_BITS_TO_LE_4BYTE(__status, 16, 1, __val)
-#define SET_RX_BUFFER_DESC_TOTAL_LENGTH(__status, __val) \
- SET_BITS_TO_LE_4BYTE(__status, 16, 15, __val)
+static inline void set_rx_buffer_desc_data_length(__le32 *__status, u32 __val)
+{
+ le32p_replace_bits(__status, __val, GENMASK(13, 0));
+}
-#define GET_RX_BUFFER_DESC_OWN(__status) \
- LE_BITS_TO_4BYTE(__status, 31, 1)
-#define GET_RX_BUFFER_DESC_LS(__status) \
- LE_BITS_TO_4BYTE(__status, 15, 1)
-#define GET_RX_BUFFER_DESC_FS(__status) \
- LE_BITS_TO_4BYTE(__status, 16, 1)
-#define GET_RX_BUFFER_DESC_TOTAL_LENGTH(__status) \
- LE_BITS_TO_4BYTE(__status, 16, 15)
+static inline void set_rx_buffer_desc_ls(__le32 *__status, u32 __val)
+{
+ le32p_replace_bits(__status, __val, BIT(15));
+}
+
+static inline void set_rx_buffer_desc_fs(__le32 *__status, u32 __val)
+{
+ le32p_replace_bits(__status, __val, BIT(16));
+}
+
+static inline void set_rx_buffer_desc_total_length(__le32 *__status, u32 __val)
+{
+ le32p_replace_bits(__status, __val, GENMASK(30, 16));
+}
+
+static inline int get_rx_buffer_desc_ls(__le32 *__status)
+{
+ return le32_get_bits(*(__status), BIT(15));
+}
+
+static inline int get_rx_buffer_desc_fs(__le32 *__status)
+{
+ return le32_get_bits(*(__status), BIT(16));
+}
+
+static inline int get_rx_buffer_desc_total_length(__le32 *__status)
+{
+ return le32_get_bits(*(__status), GENMASK(30, 16));
+}
/* DWORD 1 */
-#define SET_RX_BUFFER_PHYSICAL_LOW(__status, __val) \
- SET_BITS_TO_LE_4BYTE(__status+4, 0, 32, __val)
+static inline void set_rx_buffer_physical_low(__le32 *__status, u32 __val)
+{
+ *(__status + 1) = cpu_to_le32(__val);
+}
/* DWORD 2 */
-#define SET_RX_BUFFER_PHYSICAL_HIGH(__rx_status_desc, __val, dma64) \
- (dma64 ? SET_BITS_TO_LE_4BYTE((__rx_status_desc) + 8, 0, 32, __val) : 0)
+static inline void set_rx_buffer_physical_high(__le32 *__rx_status_desc,
+ u32 __val, bool dma64)
+{
+ if (dma64)
+ *(__rx_status_desc + 2) = cpu_to_le32(__val);
+ else
+ *(__rx_status_desc + 2) = 0;
+}
-#define GET_RX_DESC_PKT_LEN(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 0, 14)
-#define GET_RX_DESC_CRC32(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 14, 1)
-#define GET_RX_DESC_ICV(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 15, 1)
-#define GET_RX_DESC_DRV_INFO_SIZE(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 16, 4)
-#define GET_RX_DESC_SECURITY(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 20, 3)
-#define GET_RX_DESC_QOS(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 23, 1)
-#define GET_RX_DESC_SHIFT(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 24, 2)
-#define GET_RX_DESC_PHYST(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 26, 1)
-#define GET_RX_DESC_SWDEC(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 27, 1)
-#define GET_RX_DESC_LS(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 28, 1)
-#define GET_RX_DESC_FS(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 29, 1)
-#define GET_RX_DESC_EOR(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 30, 1)
-#define GET_RX_DESC_OWN(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc, 31, 1)
+static inline int get_rx_desc_pkt_len(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, GENMASK(13, 0));
+}
-#define SET_RX_DESC_PKT_LEN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 0, 14, __val)
-#define SET_RX_DESC_EOR(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 30, 1, __val)
-#define SET_RX_DESC_OWN(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc, 31, 1, __val)
+static inline int get_rx_desc_crc32(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, BIT(14));
+}
-#define GET_RX_DESC_MACID(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 0, 7)
-#define GET_RX_DESC_TID(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 8, 4)
-#define GET_RX_DESC_MACID_VLD(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 12, 1)
-#define GET_RX_DESC_AMSDU(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 13, 1)
-#define GET_RX_DESC_RXID_MATCH(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 14, 1)
-#define GET_RX_DESC_PAGGR(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 15, 1)
-#define GET_RX_DESC_A1_FIT(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 16, 4)
-#define GET_RX_DESC_TCPOFFLOAD_CHKERR(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 20, 1)
-#define GET_RX_DESC_TCPOFFLOAD_IPVER(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 21, 1)
-#define GET_RX_DESC_TCPOFFLOAD_IS_TCPUDP(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 22, 1)
-#define GET_RX_DESC_TCPOFFLOAD_CHK_VLD(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 23, 1)
-#define GET_RX_DESC_PAM(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 24, 1)
-#define GET_RX_DESC_PWR(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 25, 1)
-#define GET_RX_DESC_MD(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 26, 1)
-#define GET_RX_DESC_MF(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 27, 1)
-#define GET_RX_DESC_TYPE(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 28, 2)
-#define GET_RX_DESC_MC(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 30, 1)
-#define GET_RX_DESC_BC(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+4, 31, 1)
-#define GET_RX_DESC_SEQ(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+8, 0, 12)
-#define GET_RX_DESC_FRAG(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+8, 12, 4)
-#define GET_RX_DESC_RX_IS_QOS(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+8, 16, 1)
-#define GET_RX_STATUS_DESC_RPT_SEL(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+8, 28, 1)
+static inline int get_rx_desc_icv(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, BIT(15));
+}
-#define GET_RX_DESC_RXMCS(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 0, 7)
-#define GET_RX_DESC_HTC(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 10, 1)
-#define GET_RX_STATUS_DESC_EOSP(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 11, 1)
-#define GET_RX_STATUS_DESC_BSSID_FIT(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 12, 2)
-#define GET_RX_STATUS_DESC_DMA_AGG_NUM(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 16, 8)
-#define GET_RX_STATUS_DESC_PATTERN_MATCH(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 29, 1)
-#define GET_RX_STATUS_DESC_UNICAST_MATCH(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 30, 1)
-#define GET_RX_STATUS_DESC_MAGIC_MATCH(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+12, 31, 1)
+static inline int get_rx_desc_drv_info_size(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, GENMASK(19, 16));
+}
-#define GET_RX_DESC_TSFL(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+20, 0, 32)
+static inline int get_rx_desc_shift(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, GENMASK(25, 24));
+}
-#define GET_RX_DESC_BUFF_ADDR(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+24, 0, 32)
-#define GET_RX_DESC_BUFF_ADDR64(__pdesc) \
- LE_BITS_TO_4BYTE(__pdesc+28, 0, 32)
+static inline int get_rx_desc_physt(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, BIT(26));
+}
-#define SET_RX_DESC_BUFF_ADDR(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+24, 0, 32, __val)
-#define SET_RX_DESC_BUFF_ADDR64(__pdesc, __val) \
- SET_BITS_TO_LE_4BYTE(__pdesc+28, 0, 32, __val)
+static inline int get_rx_desc_swdec(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, BIT(27));
+}
+
+static inline int get_rx_desc_own(__le32 *__pdesc)
+{
+ return le32_get_bits(*__pdesc, BIT(31));
+}
+
+static inline void set_rx_desc_eor(__le32 *__pdesc, u32 __val)
+{
+ le32p_replace_bits(__pdesc, __val, BIT(30));
+}
+
+static inline int get_rx_desc_macid(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 1), GENMASK(6, 0));
+}
+
+static inline int get_rx_desc_paggr(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 1), BIT(15));
+}
+
+static inline int get_rx_status_desc_rpt_sel(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 2), BIT(28));
+}
+
+static inline int get_rx_desc_rxmcs(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 3), GENMASK(6, 0));
+}
+
+static inline int get_rx_status_desc_pattern_match(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 3), BIT(29));
+}
+
+static inline int get_rx_status_desc_unicast_match(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 3), BIT(30));
+}
+
+static inline int get_rx_status_desc_magic_match(__le32 *__pdesc)
+{
+ return le32_get_bits(*(__pdesc + 3), BIT(31));
+}
+
+static inline u32 get_rx_desc_tsfl(__le32 *__pdesc)
+{
+ return le32_to_cpu(*((__pdesc + 5)));
+}
+
+static inline u32 get_rx_desc_buff_addr(__le32 *__pdesc)
+{
+ return le32_to_cpu(*((__pdesc + 6)));
+}
/* TX report 2 format in Rx desc*/
-#define GET_RX_RPT2_DESC_PKT_LEN(__status) \
- LE_BITS_TO_4BYTE(__status, 0, 9)
-#define GET_RX_RPT2_DESC_MACID_VALID_1(__status) \
- LE_BITS_TO_4BYTE(__status+16, 0, 32)
-#define GET_RX_RPT2_DESC_MACID_VALID_2(__status) \
- LE_BITS_TO_4BYTE(__status+20, 0, 32)
+static inline u32 get_rx_rpt2_desc_macid_valid_1(__le32 *__status)
+{
+ return le32_to_cpu(*((__status + 4)));
+}
-#define CLEAR_PCI_TX_DESC_CONTENT(__pdesc, _size) \
-do { \
- if (_size > TX_DESC_NEXT_DESC_OFFSET) \
- memset(__pdesc, 0, TX_DESC_NEXT_DESC_OFFSET); \
- else \
- memset(__pdesc, 0, _size); \
-} while (0)
+static inline u32 get_rx_rpt2_desc_macid_valid_2(__le32 *__status)
+{
+ return le32_to_cpu(*((__status + 5)));
+}
+
+static inline void clear_pci_tx_desc_content(__le32 *__pdesc, int _size)
+{
+ if (_size > TX_DESC_NEXT_DESC_OFFSET)
+ memset(__pdesc, 0, TX_DESC_NEXT_DESC_OFFSET);
+ else
+ memset(__pdesc, 0, _size);
+}
#define RTL92EE_RX_HAL_IS_CCK_RATE(rxmcs)\
(rxmcs == DESC_RATE1M ||\
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h
index 81951f0..a9ed6fd4 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8821ae/trx.h
@@ -214,7 +214,7 @@ static inline void set_tx_desc_tx_buffer_address(__le32 *__pdesc, u32 __val)
*(__pdesc + 10) = cpu_to_le32(__val);
}
-static inline int get_tx_desc_tx_buffer_address(__le32 *__pdesc)
+static inline u32 get_tx_desc_tx_buffer_address(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 10));
}
@@ -324,12 +324,12 @@ static inline int get_rx_desc_bw(__le32 *__pdesc)
return le32_get_bits(*(__pdesc + 4), GENMASK(5, 4));
}
-static inline int get_rx_desc_tsfl(__le32 *__pdesc)
+static inline u32 get_rx_desc_tsfl(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 5));
}
-static inline int get_rx_desc_buff_addr(__le32 *__pdesc)
+static inline u32 get_rx_desc_buff_addr(__le32 *__pdesc)
{
return le32_to_cpu(*(__pdesc + 6));
}
@@ -341,12 +341,12 @@ static inline void set_rx_desc_buff_addr(__le32 *__pdesc, u32 __val)
/* TX report 2 format in Rx desc*/
-static inline int get_rx_rpt2_desc_macid_valid_1(__le32 *__status)
+static inline u32 get_rx_rpt2_desc_macid_valid_1(__le32 *__status)
{
return le32_to_cpu(*(__status + 4));
}
-static inline int get_rx_rpt2_desc_macid_valid_2(__le32 *__status)
+static inline u32 get_rx_rpt2_desc_macid_valid_2(__le32 *__status)
{
return le32_to_cpu(*(__status + 5));
}
diff --git a/drivers/net/wireless/realtek/rtlwifi/usb.c b/drivers/net/wireless/realtek/rtlwifi/usb.c
index 34d68db..4b59f3b 100644
--- a/drivers/net/wireless/realtek/rtlwifi/usb.c
+++ b/drivers/net/wireless/realtek/rtlwifi/usb.c
@@ -239,10 +239,7 @@ static void _rtl_usb_io_handler_release(struct ieee80211_hw *hw)
mutex_destroy(&rtlpriv->io.bb_mutex);
}
-/**
- *
- * Default aggregation handler. Do nothing and just return the oldest skb.
- */
+/* Default aggregation handler. Do nothing and just return the oldest skb. */
static struct sk_buff *_none_usb_tx_aggregate_hdl(struct ieee80211_hw *hw,
struct sk_buff_head *list)
{
@@ -756,11 +753,6 @@ static int rtl_usb_start(struct ieee80211_hw *hw)
return err;
}
-/**
- *
- *
- */
-
/*======================= tx =========================================*/
static void rtl_usb_cleanup(struct ieee80211_hw *hw)
{
@@ -786,11 +778,7 @@ static void rtl_usb_cleanup(struct ieee80211_hw *hw)
usb_kill_anchored_urbs(&rtlusb->tx_submitted);
}
-/**
- *
- * We may add some struct into struct rtl_usb later. Do deinit here.
- *
- */
+/* We may add some struct into struct rtl_usb later. Do deinit here. */
static void rtl_usb_deinit(struct ieee80211_hw *hw)
{
rtl_usb_cleanup(hw);
diff --git a/drivers/net/wireless/realtek/rtw88/coex.c b/drivers/net/wireless/realtek/rtw88/coex.c
index 4577fce..26e16ad 100644
--- a/drivers/net/wireless/realtek/rtw88/coex.c
+++ b/drivers/net/wireless/realtek/rtw88/coex.c
@@ -1059,7 +1059,7 @@ static void rtw_coex_set_ant_path(struct rtw_dev *rtwdev, bool force, u8 phase)
pos_type = COEX_SWITCH_TO_WLG_BT;
break;
default:
- WARN_ON("unknown phase when setting antenna path\n");
+ WARN(1, "unknown phase when setting antenna path\n");
return;
}
@@ -2454,11 +2454,6 @@ void rtw_coex_wl_fwdbginfo_notify(struct rtw_dev *rtwdev, u8 *buf, u8 length)
rtw_coex_wl_ccklock_detect(rtwdev);
}
-void rtw_coex_coex_dm_reset(struct rtw_dev *rtwdev)
-{
- __rtw_coex_init_hw_config(rtwdev, false);
-}
-
void rtw_coex_wl_status_change_notify(struct rtw_dev *rtwdev)
{
struct rtw_coex *coex = &rtwdev->coex;
diff --git a/drivers/net/wireless/realtek/rtw88/debug.c b/drivers/net/wireless/realtek/rtw88/debug.c
index 383b04c..6ad985e 100644
--- a/drivers/net/wireless/realtek/rtw88/debug.c
+++ b/drivers/net/wireless/realtek/rtw88/debug.c
@@ -77,7 +77,7 @@ static const struct file_operations file_ops_single_r = {
.open = rtw_debugfs_single_open_rw,
.read = seq_read,
.llseek = seq_lseek,
- .release = seq_release,
+ .release = single_release,
};
static const struct file_operations file_ops_single_rw = {
@@ -672,7 +672,7 @@ static struct rtw_debugfs_priv rtw_debug_priv_rsvd_page = {
void rtw_debugfs_init(struct rtw_dev *rtwdev)
{
- struct dentry *debugfs_topdir = rtwdev->debugfs;
+ struct dentry *debugfs_topdir;
debugfs_topdir = debugfs_create_dir("rtw88",
rtwdev->hw->wiphy->debugfsdir);
diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c
index 00ef229..3fdb52a 100644
--- a/drivers/net/wireless/realtek/rtw88/pci.c
+++ b/drivers/net/wireless/realtek/rtw88/pci.c
@@ -11,6 +11,10 @@
#include "fw.h"
#include "debug.h"
+static bool rtw_disable_msi;
+module_param_named(disable_msi, rtw_disable_msi, bool, 0644);
+MODULE_PARM_DESC(disable_msi, "Set Y to disable MSI interrupt support");
+
static u32 rtw_pci_tx_queue_idx_addr[] = {
[RTW_TX_QUEUE_BK] = RTK_PCI_TXBD_IDX_BKQ,
[RTW_TX_QUEUE_BE] = RTK_PCI_TXBD_IDX_BEQ,
@@ -866,12 +870,34 @@ static irqreturn_t rtw_pci_interrupt_handler(int irq, void *dev)
{
struct rtw_dev *rtwdev = dev;
struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv;
- u32 irq_status[4];
spin_lock(&rtwpci->irq_lock);
if (!rtwpci->irq_enabled)
goto out;
+ /* disable RTW PCI interrupt to avoid more interrupts before the end of
+ * thread function
+ *
+ * disable HIMR here to also avoid new HISR flag being raised before
+ * the HISRs have been Write-1-cleared for MSI. If not all of the HISRs
+ * are cleared, the edge-triggered interrupt will not be generated when
+ * a new HISR flag is set.
+ */
+ rtw_pci_disable_interrupt(rtwdev, rtwpci);
+out:
+ spin_unlock(&rtwpci->irq_lock);
+
+ return IRQ_WAKE_THREAD;
+}
+
+static irqreturn_t rtw_pci_interrupt_threadfn(int irq, void *dev)
+{
+ struct rtw_dev *rtwdev = dev;
+ struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv;
+ unsigned long flags;
+ u32 irq_status[4];
+
+ spin_lock_irqsave(&rtwpci->irq_lock, flags);
rtw_pci_irq_recognized(rtwdev, rtwpci, irq_status);
if (irq_status[0] & IMR_MGNTDOK)
@@ -891,8 +917,9 @@ static irqreturn_t rtw_pci_interrupt_handler(int irq, void *dev)
if (irq_status[0] & IMR_ROK)
rtw_pci_rx_isr(rtwdev, rtwpci, RTW_RX_QUEUE_MPDU);
-out:
- spin_unlock(&rtwpci->irq_lock);
+ /* all of the jobs for this interrupt have been done */
+ rtw_pci_enable_interrupt(rtwdev, rtwpci);
+ spin_unlock_irqrestore(&rtwpci->irq_lock, flags);
return IRQ_HANDLED;
}
@@ -1098,6 +1125,38 @@ static struct rtw_hci_ops rtw_pci_ops = {
.write_data_h2c = rtw_pci_write_data_h2c,
};
+static int rtw_pci_request_irq(struct rtw_dev *rtwdev, struct pci_dev *pdev)
+{
+ unsigned int flags = PCI_IRQ_LEGACY;
+ int ret;
+
+ if (!rtw_disable_msi)
+ flags |= PCI_IRQ_MSI;
+
+ ret = pci_alloc_irq_vectors(pdev, 1, 1, flags);
+ if (ret < 0) {
+ rtw_err(rtwdev, "failed to alloc PCI irq vectors\n");
+ return ret;
+ }
+
+ ret = devm_request_threaded_irq(rtwdev->dev, pdev->irq,
+ rtw_pci_interrupt_handler,
+ rtw_pci_interrupt_threadfn,
+ IRQF_SHARED, KBUILD_MODNAME, rtwdev);
+ if (ret) {
+ rtw_err(rtwdev, "failed to request irq %d\n", ret);
+ pci_free_irq_vectors(pdev);
+ }
+
+ return ret;
+}
+
+static void rtw_pci_free_irq(struct rtw_dev *rtwdev, struct pci_dev *pdev)
+{
+ devm_free_irq(rtwdev->dev, pdev->irq, rtwdev);
+ pci_free_irq_vectors(pdev);
+}
+
static int rtw_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
@@ -1152,8 +1211,7 @@ static int rtw_pci_probe(struct pci_dev *pdev,
goto err_destroy_pci;
}
- ret = request_irq(pdev->irq, &rtw_pci_interrupt_handler,
- IRQF_SHARED, KBUILD_MODNAME, rtwdev);
+ ret = rtw_pci_request_irq(rtwdev, pdev);
if (ret) {
ieee80211_unregister_hw(hw);
goto err_destroy_pci;
@@ -1192,7 +1250,7 @@ static void rtw_pci_remove(struct pci_dev *pdev)
rtw_pci_disable_interrupt(rtwdev, rtwpci);
rtw_pci_destroy(rtwdev, pdev);
rtw_pci_declaim(rtwdev, pdev);
- free_irq(rtwpci->pdev->irq, rtwdev);
+ rtw_pci_free_irq(rtwdev, pdev);
rtw_core_deinit(rtwdev);
ieee80211_free_hw(hw);
}
diff --git a/drivers/net/wireless/zydas/zd1211rw/zd_chip.c b/drivers/net/wireless/zydas/zd1211rw/zd_chip.c
index 40c0a86..0af4b19 100644
--- a/drivers/net/wireless/zydas/zd1211rw/zd_chip.c
+++ b/drivers/net/wireless/zydas/zd1211rw/zd_chip.c
@@ -41,8 +41,7 @@ void zd_chip_clear(struct zd_chip *chip)
static int scnprint_mac_oui(struct zd_chip *chip, char *buffer, size_t size)
{
u8 *addr = zd_mac_get_perm_addr(zd_chip_to_mac(chip));
- return scnprintf(buffer, size, "%02x-%02x-%02x",
- addr[0], addr[1], addr[2]);
+ return scnprintf(buffer, size, "%3phD", addr);
}
/* Prints an identifier line, which will support debugging. */
diff --git a/drivers/net/wireless/zydas/zd1211rw/zd_mac.c b/drivers/net/wireless/zydas/zd1211rw/zd_mac.c
index da7e63f..a9999d1 100644
--- a/drivers/net/wireless/zydas/zd1211rw/zd_mac.c
+++ b/drivers/net/wireless/zydas/zd1211rw/zd_mac.c
@@ -223,7 +223,6 @@ void zd_mac_clear(struct zd_mac *mac)
{
flush_workqueue(zd_workqueue);
zd_chip_clear(&mac->chip);
- lockdep_assert_held(&mac->lock);
ZD_MEMCLEAR(mac, sizeof(struct zd_mac));
}
diff --git a/drivers/net/wireless/zydas/zd1211rw/zd_usb.c b/drivers/net/wireless/zydas/zd1211rw/zd_usb.c
index 1965cd0..4e44ea8 100644
--- a/drivers/net/wireless/zydas/zd1211rw/zd_usb.c
+++ b/drivers/net/wireless/zydas/zd1211rw/zd_usb.c
@@ -1597,11 +1597,6 @@ static int zd_ep_regs_out_msg(struct usb_device *udev, void *data, int len,
}
}
-static int usb_int_regs_length(unsigned int count)
-{
- return sizeof(struct usb_int_regs) + count * sizeof(struct reg_data);
-}
-
static void prepare_read_regs_int(struct zd_usb *usb,
struct usb_req_read_regs *req,
unsigned int count)
@@ -1636,10 +1631,10 @@ static bool check_read_regs(struct zd_usb *usb, struct usb_req_read_regs *req,
/* The created block size seems to be larger than expected.
* However results appear to be correct.
*/
- if (rr->length < usb_int_regs_length(count)) {
+ if (rr->length < struct_size(regs, regs, count)) {
dev_dbg_f(zd_usb_dev(usb),
- "error: actual length %d less than expected %d\n",
- rr->length, usb_int_regs_length(count));
+ "error: actual length %d less than expected %ld\n",
+ rr->length, struct_size(regs, regs, count));
return false;
}
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c258a1c..d3d6b7b 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2257,6 +2257,16 @@ static const struct nvme_core_quirk_entry core_quirks[] = {
.vid = 0x1179,
.mn = "THNSF5256GPUK TOSHIBA",
.quirks = NVME_QUIRK_NO_APST,
+ },
+ {
+ /*
+ * This LiteON CL1-3D*-Q11 firmware version has a race
+ * condition associated with actions related to suspend to idle
+ * LiteON has resolved the problem in future firmware
+ */
+ .vid = 0x14a4,
+ .fr = "22301111",
+ .quirks = NVME_QUIRK_SIMPLE_SUSPEND,
}
};
@@ -2597,6 +2607,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
goto out_free;
}
+ if (!(ctrl->ops->flags & NVME_F_FABRICS))
+ ctrl->cntlid = le16_to_cpu(id->cntlid);
+
if (!ctrl->identified) {
int i;
@@ -2697,7 +2710,6 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
goto out_free;
}
} else {
- ctrl->cntlid = le16_to_cpu(id->cntlid);
ctrl->hmpre = le32_to_cpu(id->hmpre);
ctrl->hmmin = le32_to_cpu(id->hmmin);
ctrl->hmminds = le32_to_cpu(id->hmminds);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 888d454..af831d3 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -428,6 +428,7 @@ static void nvme_mpath_set_live(struct nvme_ns *ns)
srcu_read_unlock(&head->srcu, srcu_idx);
}
+ synchronize_srcu(&ns->head->srcu);
kblockd_schedule_work(&ns->head->requeue_work);
}
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 778b3a0..2d678fb 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -92,6 +92,11 @@ enum nvme_quirks {
* Broken Write Zeroes.
*/
NVME_QUIRK_DISABLE_WRITE_ZEROES = (1 << 9),
+
+ /*
+ * Force simple suspend/resume path.
+ */
+ NVME_QUIRK_SIMPLE_SUSPEND = (1 << 10),
};
/*
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 6bd9b10..732d5b6 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2876,7 +2876,8 @@ static int nvme_suspend(struct device *dev)
* state (which may not be possible if the link is up).
*/
if (pm_suspend_via_firmware() || !ctrl->npss ||
- !pcie_aspm_enabled(pdev)) {
+ !pcie_aspm_enabled(pdev) ||
+ (ndev->ctrl.quirks & NVME_QUIRK_SIMPLE_SUSPEND)) {
nvme_dev_disable(ndev, true);
return 0;
}
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 208aacf3..44c4ae1 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -5256,7 +5256,7 @@ static void quirk_reset_lenovo_thinkpad_p50_nvgpu(struct pci_dev *pdev)
*/
if (ioread32(map + 0x2240c) & 0x2) {
pci_info(pdev, FW_BUG "GPU left initialized by EFI, resetting\n");
- ret = pci_reset_function(pdev);
+ ret = pci_reset_bus(pdev);
if (ret < 0)
pci_err(pdev, "Failed to reset GPU: %d\n", ret);
}
diff --git a/drivers/platform/chrome/cros_ec_ishtp.c b/drivers/platform/chrome/cros_ec_ishtp.c
index e504d25..430731c 100644
--- a/drivers/platform/chrome/cros_ec_ishtp.c
+++ b/drivers/platform/chrome/cros_ec_ishtp.c
@@ -707,7 +707,7 @@ static int cros_ec_ishtp_reset(struct ishtp_cl_device *cl_device)
*/
static int __maybe_unused cros_ec_ishtp_suspend(struct device *device)
{
- struct ishtp_cl_device *cl_device = dev_get_drvdata(device);
+ struct ishtp_cl_device *cl_device = ishtp_dev_to_cl_device(device);
struct ishtp_cl *cros_ish_cl = ishtp_get_drvdata(cl_device);
struct ishtp_cl_data *client_data = ishtp_get_client_data(cros_ish_cl);
@@ -722,7 +722,7 @@ static int __maybe_unused cros_ec_ishtp_suspend(struct device *device)
*/
static int __maybe_unused cros_ec_ishtp_resume(struct device *device)
{
- struct ishtp_cl_device *cl_device = dev_get_drvdata(device);
+ struct ishtp_cl_device *cl_device = ishtp_dev_to_cl_device(device);
struct ishtp_cl *cros_ish_cl = ishtp_get_drvdata(cl_device);
struct ishtp_cl_data *client_data = ishtp_get_client_data(cros_ish_cl);
diff --git a/drivers/power/supply/ab8500_charger.c b/drivers/power/supply/ab8500_charger.c
index 30de448..86d88ae 100644
--- a/drivers/power/supply/ab8500_charger.c
+++ b/drivers/power/supply/ab8500_charger.c
@@ -742,6 +742,7 @@ static int ab8500_charger_max_usb_curr(struct ab8500_charger *di,
USB_CH_IP_CUR_LVL_1P5;
break;
}
+ /* Else, fall through */
case USB_STAT_HM_IDGND:
dev_err(di->dev, "USB Type - Charging not allowed\n");
di->max_usb_in_curr.usb_type_max = USB_CH_IP_CUR_LVL_0P05;
diff --git a/drivers/s390/cio/qdio.h b/drivers/s390/cio/qdio.h
index a069443..a58b45d 100644
--- a/drivers/s390/cio/qdio.h
+++ b/drivers/s390/cio/qdio.h
@@ -206,8 +206,6 @@ struct qdio_output_q {
struct qdio_outbuf_state *sbal_state;
/* timer to check for more outbound work */
struct timer_list timer;
- /* used SBALs before tasklet schedule */
- int scan_threshold;
};
/*
@@ -295,6 +293,7 @@ struct qdio_irq {
struct qdio_ssqd_desc ssqd_desc;
void (*orig_handler) (struct ccw_device *, unsigned long, struct irb *);
+ unsigned int scan_threshold; /* used SBALs before tasklet schedule */
int perf_stat_enabled;
struct qdr *qdr;
diff --git a/drivers/s390/cio/qdio_main.c b/drivers/s390/cio/qdio_main.c
index 4142c85..5b63c50 100644
--- a/drivers/s390/cio/qdio_main.c
+++ b/drivers/s390/cio/qdio_main.c
@@ -647,8 +647,6 @@ static void qdio_kick_handler(struct qdio_q *q, unsigned int count)
qperf_inc(q, outbound_handler);
DBF_DEV_EVENT(DBF_INFO, q->irq_ptr, "koh: s:%02x c:%02x",
start, count);
- if (q->u.out.use_cq)
- qdio_handle_aobs(q, start, count);
}
q->handler(q->irq_ptr->cdev, q->qdio_error, q->nr, start, count,
@@ -774,8 +772,11 @@ static inline int qdio_outbound_q_moved(struct qdio_q *q, unsigned int start)
count = get_outbound_buffer_frontier(q, start);
- if (count)
+ if (count) {
DBF_DEV_EVENT(DBF_INFO, q->irq_ptr, "out moved:%1d", q->nr);
+ if (q->u.out.use_cq)
+ qdio_handle_aobs(q, start, count);
+ }
return count;
}
@@ -879,7 +880,7 @@ static inline void qdio_check_outbound_pci_queues(struct qdio_irq *irq)
struct qdio_q *out;
int i;
- if (!pci_out_supported(irq))
+ if (!pci_out_supported(irq) || !irq->scan_threshold)
return;
for_each_output_queue(irq, out, i)
@@ -972,7 +973,7 @@ static void qdio_int_handler_pci(struct qdio_irq *irq_ptr)
}
}
- if (!pci_out_supported(irq_ptr))
+ if (!pci_out_supported(irq_ptr) || !irq_ptr->scan_threshold)
return;
for_each_output_queue(irq_ptr, q, i) {
@@ -1527,6 +1528,7 @@ static int handle_inbound(struct qdio_q *q, unsigned int callflags,
static int handle_outbound(struct qdio_q *q, unsigned int callflags,
int bufnr, int count)
{
+ const unsigned int scan_threshold = q->irq_ptr->scan_threshold;
unsigned char state = 0;
int used, rc = 0;
@@ -1565,8 +1567,12 @@ static int handle_outbound(struct qdio_q *q, unsigned int callflags,
rc = qdio_kick_outbound_q(q, 0);
}
+ /* Let drivers implement their own completion scanning: */
+ if (!scan_threshold)
+ return rc;
+
/* in case of SIGA errors we must process the error immediately */
- if (used >= q->u.out.scan_threshold || rc)
+ if (used >= scan_threshold || rc)
qdio_tasklet_schedule(q);
else
/* free the SBALs in case of no further traffic */
@@ -1655,6 +1661,44 @@ int qdio_start_irq(struct ccw_device *cdev, int nr)
}
EXPORT_SYMBOL(qdio_start_irq);
+static int __qdio_inspect_queue(struct qdio_q *q, unsigned int *bufnr,
+ unsigned int *error)
+{
+ unsigned int start = q->first_to_check;
+ int count;
+
+ count = q->is_input_q ? qdio_inbound_q_moved(q, start) :
+ qdio_outbound_q_moved(q, start);
+ if (count == 0)
+ return 0;
+
+ *bufnr = start;
+ *error = q->qdio_error;
+
+ /* for the next time */
+ q->first_to_check = add_buf(start, count);
+ q->qdio_error = 0;
+
+ return count;
+}
+
+int qdio_inspect_queue(struct ccw_device *cdev, unsigned int nr, bool is_input,
+ unsigned int *bufnr, unsigned int *error)
+{
+ struct qdio_irq *irq_ptr = cdev->private->qdio_data;
+ struct qdio_q *q;
+
+ if (!irq_ptr)
+ return -ENODEV;
+ q = is_input ? irq_ptr->input_qs[nr] : irq_ptr->output_qs[nr];
+
+ if (need_siga_sync(q))
+ qdio_siga_sync_q(q);
+
+ return __qdio_inspect_queue(q, bufnr, error);
+}
+EXPORT_SYMBOL_GPL(qdio_inspect_queue);
+
/**
* qdio_get_next_buffers - process input buffers
* @cdev: associated ccw_device for the qdio subchannel
@@ -1672,13 +1716,10 @@ int qdio_get_next_buffers(struct ccw_device *cdev, int nr, int *bufnr,
{
struct qdio_q *q;
struct qdio_irq *irq_ptr = cdev->private->qdio_data;
- unsigned int start;
- int count;
if (!irq_ptr)
return -ENODEV;
q = irq_ptr->input_qs[nr];
- start = q->first_to_check;
/*
* Cannot rely on automatic sync after interrupt since queues may
@@ -1689,25 +1730,11 @@ int qdio_get_next_buffers(struct ccw_device *cdev, int nr, int *bufnr,
qdio_check_outbound_pci_queues(irq_ptr);
- count = qdio_inbound_q_moved(q, start);
- if (count == 0)
- return 0;
-
- start = add_buf(start, count);
- q->first_to_check = start;
-
/* Note: upper-layer MUST stop processing immediately here ... */
if (unlikely(q->irq_ptr->state != QDIO_IRQ_STATE_ACTIVE))
return -EIO;
- *bufnr = q->first_to_kick;
- *error = q->qdio_error;
-
- /* for the next time */
- q->first_to_kick = add_buf(q->first_to_kick, count);
- q->qdio_error = 0;
-
- return count;
+ return __qdio_inspect_queue(q, bufnr, error);
}
EXPORT_SYMBOL(qdio_get_next_buffers);
diff --git a/drivers/s390/cio/qdio_setup.c b/drivers/s390/cio/qdio_setup.c
index d4101ce..f4ca1d2 100644
--- a/drivers/s390/cio/qdio_setup.c
+++ b/drivers/s390/cio/qdio_setup.c
@@ -248,7 +248,6 @@ static void setup_queues(struct qdio_irq *irq_ptr,
output_sbal_state_array += QDIO_MAX_BUFFERS_PER_Q;
q->is_input_q = 0;
- q->u.out.scan_threshold = qdio_init->scan_threshold;
setup_storage_lists(q, irq_ptr, output_sbal_array, i);
output_sbal_array += QDIO_MAX_BUFFERS_PER_Q;
@@ -474,6 +473,7 @@ int qdio_setup_irq(struct qdio_initialize *init_data)
irq_ptr->nr_input_qs = init_data->no_input_qs;
irq_ptr->nr_output_qs = init_data->no_output_qs;
irq_ptr->cdev = init_data->cdev;
+ irq_ptr->scan_threshold = init_data->scan_threshold;
ccw_device_get_schid(irq_ptr->cdev, &irq_ptr->schid);
setup_queues(irq_ptr, init_data);
diff --git a/drivers/s390/net/qeth_core.h b/drivers/s390/net/qeth_core.h
index 72755a0..e4b55f9 100644
--- a/drivers/s390/net/qeth_core.h
+++ b/drivers/s390/net/qeth_core.h
@@ -22,6 +22,7 @@
#include <linux/hashtable.h>
#include <linux/ip.h>
#include <linux/refcount.h>
+#include <linux/timer.h>
#include <linux/wait.h>
#include <linux/workqueue.h>
@@ -30,6 +31,7 @@
#include <net/ipv6.h>
#include <net/if_inet6.h>
#include <net/addrconf.h>
+#include <net/sch_generic.h>
#include <net/tcp.h>
#include <asm/debug.h>
@@ -376,6 +378,28 @@ enum qeth_header_ids {
#define QETH_HDR_EXT_CSUM_TRANSP_REQ 0x20
#define QETH_HDR_EXT_UDP 0x40 /*bit off for TCP*/
+static inline bool qeth_l2_same_vlan(struct qeth_hdr_layer2 *h1,
+ struct qeth_hdr_layer2 *h2)
+{
+ return !((h1->flags[2] ^ h2->flags[2]) & QETH_LAYER2_FLAG_VLAN) &&
+ h1->vlan_id == h2->vlan_id;
+}
+
+static inline bool qeth_l3_iqd_same_vlan(struct qeth_hdr_layer3 *h1,
+ struct qeth_hdr_layer3 *h2)
+{
+ return !((h1->ext_flags ^ h2->ext_flags) & QETH_HDR_EXT_VLAN_FRAME) &&
+ h1->vlan_id == h2->vlan_id;
+}
+
+static inline bool qeth_l3_same_next_hop(struct qeth_hdr_layer3 *h1,
+ struct qeth_hdr_layer3 *h2)
+{
+ return !((h1->flags ^ h2->flags) & QETH_HDR_IPV6) &&
+ ipv6_addr_equal(&h1->next_hop.ipv6_addr,
+ &h2->next_hop.ipv6_addr);
+}
+
enum qeth_qdio_info_states {
QETH_QDIO_UNINITIALIZED,
QETH_QDIO_ALLOCATED,
@@ -424,6 +448,7 @@ struct qeth_qdio_out_buffer {
struct qdio_buffer *buffer;
atomic_t state;
int next_element_to_fill;
+ unsigned int bytes;
struct sk_buff_head skb_list;
int is_header[QDIO_MAX_ELEMENTS_PER_BUFFER];
@@ -473,6 +498,8 @@ struct qeth_out_q_stats {
u64 tso_bytes;
u64 packing_mode_switch;
u64 stopped;
+ u64 completion_yield;
+ u64 completion_timer;
/* rtnl_link_stats64 */
u64 tx_packets;
@@ -481,6 +508,8 @@ struct qeth_out_q_stats {
u64 tx_dropped;
};
+#define QETH_TX_TIMER_USECS 500
+
struct qeth_qdio_out_q {
struct qdio_buffer *qdio_bufs[QDIO_MAX_BUFFERS_PER_Q];
struct qeth_qdio_out_buffer *bufs[QDIO_MAX_BUFFERS_PER_Q];
@@ -499,13 +528,36 @@ struct qeth_qdio_out_q {
atomic_t used_buffers;
/* indicates whether PCI flag must be set (or if one is outstanding) */
atomic_t set_pci_flags_count;
+ struct napi_struct napi;
+ struct timer_list timer;
+ struct qeth_hdr *prev_hdr;
+ u8 bulk_start;
};
+#define qeth_for_each_output_queue(card, q, i) \
+ for (i = 0; i < card->qdio.no_out_queues && \
+ (q = card->qdio.out_qs[i]); i++)
+
+#define qeth_napi_to_out_queue(n) container_of(n, struct qeth_qdio_out_q, napi)
+
+static inline void qeth_tx_arm_timer(struct qeth_qdio_out_q *queue)
+{
+ if (timer_pending(&queue->timer))
+ return;
+ mod_timer(&queue->timer, usecs_to_jiffies(QETH_TX_TIMER_USECS) +
+ jiffies);
+}
+
static inline bool qeth_out_queue_is_full(struct qeth_qdio_out_q *queue)
{
return atomic_read(&queue->used_buffers) >= QDIO_MAX_BUFFERS_PER_Q;
}
+static inline bool qeth_out_queue_is_empty(struct qeth_qdio_out_q *queue)
+{
+ return atomic_read(&queue->used_buffers) == 0;
+}
+
struct qeth_qdio_info {
atomic_t state;
/* input */
diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c
index 44fbaa4..a7868c8 100644
--- a/drivers/s390/net/qeth_core_main.c
+++ b/drivers/s390/net/qeth_core_main.c
@@ -71,7 +71,8 @@ static void qeth_free_qdio_queues(struct qeth_card *card);
static void qeth_notify_skbs(struct qeth_qdio_out_q *queue,
struct qeth_qdio_out_buffer *buf,
enum iucv_tx_notify notification);
-static void qeth_release_skbs(struct qeth_qdio_out_buffer *buf);
+static void qeth_tx_complete_buf(struct qeth_qdio_out_buffer *buf, bool error,
+ int budget);
static int qeth_init_qdio_out_buf(struct qeth_qdio_out_q *, int);
static void qeth_close_dev_handler(struct work_struct *work)
@@ -411,7 +412,7 @@ static void qeth_cleanup_handled_pending(struct qeth_qdio_out_q *q, int bidx,
/* release here to avoid interleaving between
outbound tasklet and inbound tasklet
regarding notifications and lifecycle */
- qeth_release_skbs(c);
+ qeth_tx_complete_buf(c, forced_cleanup, 0);
c = f->next_pending;
WARN_ON_ONCE(head->next_pending != f);
@@ -1077,22 +1078,52 @@ static void qeth_notify_skbs(struct qeth_qdio_out_q *q,
}
}
-static void qeth_release_skbs(struct qeth_qdio_out_buffer *buf)
+static void qeth_tx_complete_buf(struct qeth_qdio_out_buffer *buf, bool error,
+ int budget)
{
+ struct qeth_qdio_out_q *queue = buf->q;
struct sk_buff *skb;
/* release may never happen from within CQ tasklet scope */
WARN_ON_ONCE(atomic_read(&buf->state) == QETH_QDIO_BUF_IN_CQ);
if (atomic_read(&buf->state) == QETH_QDIO_BUF_PENDING)
- qeth_notify_skbs(buf->q, buf, TX_NOTIFY_GENERALERROR);
+ qeth_notify_skbs(queue, buf, TX_NOTIFY_GENERALERROR);
- while ((skb = __skb_dequeue(&buf->skb_list)) != NULL)
- consume_skb(skb);
+ /* Empty buffer? */
+ if (buf->next_element_to_fill == 0)
+ return;
+
+ QETH_TXQ_STAT_INC(queue, bufs);
+ QETH_TXQ_STAT_ADD(queue, buf_elements, buf->next_element_to_fill);
+ while ((skb = __skb_dequeue(&buf->skb_list)) != NULL) {
+ unsigned int bytes = qdisc_pkt_len(skb);
+ bool is_tso = skb_is_gso(skb);
+ unsigned int packets;
+
+ packets = is_tso ? skb_shinfo(skb)->gso_segs : 1;
+ if (error) {
+ QETH_TXQ_STAT_ADD(queue, tx_errors, packets);
+ } else {
+ QETH_TXQ_STAT_ADD(queue, tx_packets, packets);
+ QETH_TXQ_STAT_ADD(queue, tx_bytes, bytes);
+ if (skb->ip_summed == CHECKSUM_PARTIAL)
+ QETH_TXQ_STAT_ADD(queue, skbs_csum, packets);
+ if (skb_is_nonlinear(skb))
+ QETH_TXQ_STAT_INC(queue, skbs_sg);
+ if (is_tso) {
+ QETH_TXQ_STAT_INC(queue, skbs_tso);
+ QETH_TXQ_STAT_ADD(queue, tso_bytes, bytes);
+ }
+ }
+
+ napi_consume_skb(skb, budget);
+ }
}
static void qeth_clear_output_buffer(struct qeth_qdio_out_q *queue,
- struct qeth_qdio_out_buffer *buf)
+ struct qeth_qdio_out_buffer *buf,
+ bool error, int budget)
{
int i;
@@ -1100,7 +1131,7 @@ static void qeth_clear_output_buffer(struct qeth_qdio_out_q *queue,
if (buf->buffer->element[0].sflags & SBAL_SFLAGS0_PCI_REQ)
atomic_dec(&queue->set_pci_flags_count);
- qeth_release_skbs(buf);
+ qeth_tx_complete_buf(buf, error, budget);
for (i = 0; i < queue->max_elements; ++i) {
if (buf->buffer->element[i].addr && buf->is_header[i])
@@ -1111,6 +1142,7 @@ static void qeth_clear_output_buffer(struct qeth_qdio_out_q *queue,
qeth_scrub_qdio_buffer(buf->buffer, queue->max_elements);
buf->next_element_to_fill = 0;
+ buf->bytes = 0;
atomic_set(&buf->state, QETH_QDIO_BUF_EMPTY);
}
@@ -1122,7 +1154,7 @@ static void qeth_drain_output_queue(struct qeth_qdio_out_q *q, bool free)
if (!q->bufs[j])
continue;
qeth_cleanup_handled_pending(q, j, 1);
- qeth_clear_output_buffer(q, q->bufs[j]);
+ qeth_clear_output_buffer(q, q->bufs[j], true, 0);
if (free) {
kmem_cache_free(qeth_qdio_outbuf_cache, q->bufs[j]);
q->bufs[j] = NULL;
@@ -2255,6 +2287,14 @@ static struct qeth_qdio_out_q *qeth_alloc_output_queue(void)
return q;
}
+static void qeth_tx_completion_timer(struct timer_list *timer)
+{
+ struct qeth_qdio_out_q *queue = from_timer(queue, timer, timer);
+
+ napi_schedule(&queue->napi);
+ QETH_TXQ_STAT_INC(queue, completion_timer);
+}
+
static int qeth_alloc_qdio_queues(struct qeth_card *card)
{
int i, j;
@@ -2276,17 +2316,22 @@ static int qeth_alloc_qdio_queues(struct qeth_card *card)
/* outbound */
for (i = 0; i < card->qdio.no_out_queues; ++i) {
- card->qdio.out_qs[i] = qeth_alloc_output_queue();
- if (!card->qdio.out_qs[i])
+ struct qeth_qdio_out_q *queue;
+
+ queue = qeth_alloc_output_queue();
+ if (!queue)
goto out_freeoutq;
QETH_CARD_TEXT_(card, 2, "outq %i", i);
- QETH_CARD_HEX(card, 2, &card->qdio.out_qs[i], sizeof(void *));
- card->qdio.out_qs[i]->card = card;
- card->qdio.out_qs[i]->queue_no = i;
+ QETH_CARD_HEX(card, 2, &queue, sizeof(void *));
+ card->qdio.out_qs[i] = queue;
+ queue->card = card;
+ queue->queue_no = i;
+ timer_setup(&queue->timer, qeth_tx_completion_timer, 0);
+
/* give outbound qeth_qdio_buffers their qdio_buffers */
for (j = 0; j < QDIO_MAX_BUFFERS_PER_Q; ++j) {
- WARN_ON(card->qdio.out_qs[i]->bufs[j] != NULL);
- if (qeth_init_qdio_out_buf(card->qdio.out_qs[i], j))
+ WARN_ON(queue->bufs[j]);
+ if (qeth_init_qdio_out_buf(queue, j))
goto out_freeoutqbufs;
}
}
@@ -2626,9 +2671,12 @@ int qeth_init_qdio_queues(struct qeth_card *card)
queue->max_elements = QETH_MAX_BUFFER_ELEMENTS(card);
queue->next_buf_to_fill = 0;
queue->do_pack = 0;
+ queue->prev_hdr = NULL;
+ queue->bulk_start = 0;
atomic_set(&queue->used_buffers, 0);
atomic_set(&queue->set_pci_flags_count, 0);
atomic_set(&queue->state, QETH_OUT_Q_UNLOCKED);
+ netdev_tx_reset_queue(netdev_get_tx_queue(card->dev, i));
}
return 0;
}
@@ -3197,6 +3245,7 @@ static int qeth_switch_to_nonpacking_if_needed(struct qeth_qdio_out_q *queue)
static void qeth_flush_buffers(struct qeth_qdio_out_q *queue, int index,
int count)
{
+ struct qeth_card *card = queue->card;
struct qeth_qdio_out_buffer *buf;
int rc;
int i;
@@ -3240,14 +3289,17 @@ static void qeth_flush_buffers(struct qeth_qdio_out_q *queue, int index,
}
}
- QETH_TXQ_STAT_ADD(queue, bufs, count);
qdio_flags = QDIO_FLAG_SYNC_OUTPUT;
if (atomic_read(&queue->set_pci_flags_count))
qdio_flags |= QDIO_FLAG_PCI_OUT;
rc = do_QDIO(CARD_DDEV(queue->card), qdio_flags,
queue->queue_no, index, count);
+
+ /* Fake the TX completion interrupt: */
+ if (IS_IQD(card))
+ napi_schedule(&queue->napi);
+
if (rc) {
- QETH_TXQ_STAT_ADD(queue, tx_errors, count);
/* ignore temporary SIGA errors without busy condition */
if (rc == -ENOBUFS)
return;
@@ -3264,6 +3316,14 @@ static void qeth_flush_buffers(struct qeth_qdio_out_q *queue, int index,
}
}
+static void qeth_flush_queue(struct qeth_qdio_out_q *queue)
+{
+ qeth_flush_buffers(queue, queue->bulk_start, 1);
+
+ queue->bulk_start = QDIO_BUFNR(queue->bulk_start + 1);
+ queue->prev_hdr = NULL;
+}
+
static void qeth_check_outbound_queue(struct qeth_qdio_out_q *queue)
{
int index;
@@ -3425,48 +3485,12 @@ static void qeth_qdio_output_handler(struct ccw_device *ccwdev,
int bidx = i % QDIO_MAX_BUFFERS_PER_Q;
buffer = queue->bufs[bidx];
qeth_handle_send_error(card, buffer, qdio_error);
-
- if (queue->bufstates &&
- (queue->bufstates[bidx].flags &
- QDIO_OUTBUF_STATE_FLAG_PENDING) != 0) {
- WARN_ON_ONCE(card->options.cq != QETH_CQ_ENABLED);
-
- if (atomic_cmpxchg(&buffer->state,
- QETH_QDIO_BUF_PRIMED,
- QETH_QDIO_BUF_PENDING) ==
- QETH_QDIO_BUF_PRIMED) {
- qeth_notify_skbs(queue, buffer,
- TX_NOTIFY_PENDING);
- }
- QETH_CARD_TEXT_(queue->card, 5, "pel%d", bidx);
-
- /* prepare the queue slot for re-use: */
- qeth_scrub_qdio_buffer(buffer->buffer,
- queue->max_elements);
- if (qeth_init_qdio_out_buf(queue, bidx)) {
- QETH_CARD_TEXT(card, 2, "outofbuf");
- qeth_schedule_recovery(card);
- }
- } else {
- if (card->options.cq == QETH_CQ_ENABLED) {
- enum iucv_tx_notify n;
-
- n = qeth_compute_cq_notification(
- buffer->buffer->element[15].sflags, 0);
- qeth_notify_skbs(queue, buffer, n);
- }
-
- qeth_clear_output_buffer(queue, buffer);
- }
- qeth_cleanup_handled_pending(queue, bidx, 0);
+ qeth_clear_output_buffer(queue, buffer, qdio_error, 0);
}
- atomic_sub(count, &queue->used_buffers);
- /* check if we need to do something on this outbound queue */
- if (!IS_IQD(card))
- qeth_check_outbound_queue(queue);
- if (IS_IQD(card))
- __queue = qeth_iqd_translate_txq(dev, __queue);
+ atomic_sub(count, &queue->used_buffers);
+ qeth_check_outbound_queue(queue);
+
txq = netdev_get_tx_queue(dev, __queue);
/* xmit may have observed the full-condition, but not yet stopped the
* txq. In which case the code below won't trigger. So before returning,
@@ -3655,9 +3679,32 @@ static int qeth_add_hw_header(struct qeth_qdio_out_q *queue,
return 0;
}
-static void __qeth_fill_buffer(struct sk_buff *skb,
- struct qeth_qdio_out_buffer *buf,
- bool is_first_elem, unsigned int offset)
+static bool qeth_iqd_may_bulk(struct qeth_qdio_out_q *queue,
+ struct qeth_qdio_out_buffer *buffer,
+ struct sk_buff *curr_skb,
+ struct qeth_hdr *curr_hdr)
+{
+ struct qeth_hdr *prev_hdr = queue->prev_hdr;
+
+ if (!prev_hdr)
+ return true;
+
+ /* All packets must have the same target: */
+ if (curr_hdr->hdr.l2.id == QETH_HEADER_TYPE_LAYER2) {
+ struct sk_buff *prev_skb = skb_peek(&buffer->skb_list);
+
+ return ether_addr_equal(eth_hdr(prev_skb)->h_dest,
+ eth_hdr(curr_skb)->h_dest) &&
+ qeth_l2_same_vlan(&prev_hdr->hdr.l2, &curr_hdr->hdr.l2);
+ }
+
+ return qeth_l3_same_next_hop(&prev_hdr->hdr.l3, &curr_hdr->hdr.l3) &&
+ qeth_l3_iqd_same_vlan(&prev_hdr->hdr.l3, &curr_hdr->hdr.l3);
+}
+
+static unsigned int __qeth_fill_buffer(struct sk_buff *skb,
+ struct qeth_qdio_out_buffer *buf,
+ bool is_first_elem, unsigned int offset)
{
struct qdio_buffer *buffer = buf->buffer;
int element = buf->next_element_to_fill;
@@ -3714,24 +3761,21 @@ static void __qeth_fill_buffer(struct sk_buff *skb,
if (buffer->element[element - 1].eflags)
buffer->element[element - 1].eflags = SBAL_EFLAGS_LAST_FRAG;
buf->next_element_to_fill = element;
+ return element;
}
/**
* qeth_fill_buffer() - map skb into an output buffer
- * @queue: QDIO queue to submit the buffer on
* @buf: buffer to transport the skb
* @skb: skb to map into the buffer
* @hdr: qeth_hdr for this skb. Either at skb->data, or allocated
* from qeth_core_header_cache.
* @offset: when mapping the skb, start at skb->data + offset
* @hd_len: if > 0, build a dedicated header element of this size
- * flush: Prepare the buffer to be flushed, regardless of its fill level.
*/
-static int qeth_fill_buffer(struct qeth_qdio_out_q *queue,
- struct qeth_qdio_out_buffer *buf,
- struct sk_buff *skb, struct qeth_hdr *hdr,
- unsigned int offset, unsigned int hd_len,
- bool flush)
+static unsigned int qeth_fill_buffer(struct qeth_qdio_out_buffer *buf,
+ struct sk_buff *skb, struct qeth_hdr *hdr,
+ unsigned int offset, unsigned int hd_len)
{
struct qdio_buffer *buffer = buf->buffer;
bool is_first_elem = true;
@@ -3751,35 +3795,22 @@ static int qeth_fill_buffer(struct qeth_qdio_out_q *queue,
buf->next_element_to_fill++;
}
- __qeth_fill_buffer(skb, buf, is_first_elem, offset);
-
- if (!queue->do_pack) {
- QETH_CARD_TEXT(queue->card, 6, "fillbfnp");
- } else {
- QETH_CARD_TEXT(queue->card, 6, "fillbfpa");
-
- QETH_TXQ_STAT_INC(queue, skbs_pack);
- /* If the buffer still has free elements, keep using it. */
- if (!flush &&
- buf->next_element_to_fill < queue->max_elements)
- return 0;
- }
-
- /* flush out the buffer */
- atomic_set(&buf->state, QETH_QDIO_BUF_PRIMED);
- queue->next_buf_to_fill = (queue->next_buf_to_fill + 1) %
- QDIO_MAX_BUFFERS_PER_Q;
- return 1;
+ return __qeth_fill_buffer(skb, buf, is_first_elem, offset);
}
-static int qeth_do_send_packet_fast(struct qeth_qdio_out_q *queue,
- struct sk_buff *skb, struct qeth_hdr *hdr,
- unsigned int offset, unsigned int hd_len)
+static int __qeth_xmit(struct qeth_card *card, struct qeth_qdio_out_q *queue,
+ struct sk_buff *skb, unsigned int elements,
+ struct qeth_hdr *hdr, unsigned int offset,
+ unsigned int hd_len)
{
- int index = queue->next_buf_to_fill;
- struct qeth_qdio_out_buffer *buffer = queue->bufs[index];
+ struct qeth_qdio_out_buffer *buffer = queue->bufs[queue->bulk_start];
+ unsigned int bytes = qdisc_pkt_len(skb);
+ unsigned int next_element;
struct netdev_queue *txq;
bool stopped = false;
+ bool flush;
+
+ txq = netdev_get_tx_queue(card->dev, skb_get_queue_mapping(skb));
/* Just a sanity check, the wake/stop logic should ensure that we always
* get a free buffer.
@@ -3787,9 +3818,19 @@ static int qeth_do_send_packet_fast(struct qeth_qdio_out_q *queue,
if (atomic_read(&buffer->state) != QETH_QDIO_BUF_EMPTY)
return -EBUSY;
- txq = netdev_get_tx_queue(queue->card->dev, skb_get_queue_mapping(skb));
+ if ((buffer->next_element_to_fill + elements > queue->max_elements) ||
+ !qeth_iqd_may_bulk(queue, buffer, skb, hdr)) {
+ atomic_set(&buffer->state, QETH_QDIO_BUF_PRIMED);
+ qeth_flush_queue(queue);
+ buffer = queue->bufs[queue->bulk_start];
- if (atomic_inc_return(&queue->used_buffers) >= QDIO_MAX_BUFFERS_PER_Q) {
+ /* Sanity-check again: */
+ if (atomic_read(&buffer->state) != QETH_QDIO_BUF_EMPTY)
+ return -EBUSY;
+ }
+
+ if (buffer->next_element_to_fill == 0 &&
+ atomic_inc_return(&queue->used_buffers) >= QDIO_MAX_BUFFERS_PER_Q) {
/* If a TX completion happens right _here_ and misses to wake
* the txq, then our re-check below will catch the race.
*/
@@ -3798,8 +3839,17 @@ static int qeth_do_send_packet_fast(struct qeth_qdio_out_q *queue,
stopped = true;
}
- qeth_fill_buffer(queue, buffer, skb, hdr, offset, hd_len, stopped);
- qeth_flush_buffers(queue, index, 1);
+ next_element = qeth_fill_buffer(buffer, skb, hdr, offset, hd_len);
+ buffer->bytes += bytes;
+ queue->prev_hdr = hdr;
+
+ flush = __netdev_tx_sent_queue(txq, bytes,
+ !stopped && netdev_xmit_more());
+
+ if (flush || next_element >= queue->max_elements) {
+ atomic_set(&buffer->state, QETH_QDIO_BUF_PRIMED);
+ qeth_flush_queue(queue);
+ }
if (stopped && !qeth_out_queue_is_full(queue))
netif_tx_start_queue(txq);
@@ -3812,6 +3862,7 @@ int qeth_do_send_packet(struct qeth_card *card, struct qeth_qdio_out_q *queue,
int elements_needed)
{
struct qeth_qdio_out_buffer *buffer;
+ unsigned int next_element;
struct netdev_queue *txq;
bool stopped = false;
int start_index;
@@ -3874,8 +3925,17 @@ int qeth_do_send_packet(struct qeth_card *card, struct qeth_qdio_out_q *queue,
stopped = true;
}
- flush_count += qeth_fill_buffer(queue, buffer, skb, hdr, offset, hd_len,
- stopped);
+ next_element = qeth_fill_buffer(buffer, skb, hdr, offset, hd_len);
+
+ if (queue->do_pack)
+ QETH_TXQ_STAT_INC(queue, skbs_pack);
+ if (!queue->do_pack || stopped || next_element >= queue->max_elements) {
+ flush_count++;
+ atomic_set(&buffer->state, QETH_QDIO_BUF_PRIMED);
+ queue->next_buf_to_fill = (queue->next_buf_to_fill + 1) %
+ QDIO_MAX_BUFFERS_PER_Q;
+ }
+
if (flush_count)
qeth_flush_buffers(queue, start_index, flush_count);
else if (!atomic_read(&queue->set_pci_flags_count))
@@ -3942,7 +4002,6 @@ int qeth_xmit(struct qeth_card *card, struct sk_buff *skb,
unsigned int hd_len = 0;
unsigned int elements;
int push_len, rc;
- bool is_sg;
if (is_tso) {
hw_hdr_len = sizeof(struct qeth_hdr_tso);
@@ -3971,10 +4030,9 @@ int qeth_xmit(struct qeth_card *card, struct sk_buff *skb,
qeth_fill_tso_ext((struct qeth_hdr_tso *) hdr,
frame_len - proto_len, skb, proto_len);
- is_sg = skb_is_nonlinear(skb);
if (IS_IQD(card)) {
- rc = qeth_do_send_packet_fast(queue, skb, hdr, data_offset,
- hd_len);
+ rc = __qeth_xmit(card, queue, skb, elements, hdr, data_offset,
+ hd_len);
} else {
/* TODO: drop skb_orphan() once TX completion is fast enough */
skb_orphan(skb);
@@ -3982,18 +4040,9 @@ int qeth_xmit(struct qeth_card *card, struct sk_buff *skb,
hd_len, elements);
}
- if (!rc) {
- QETH_TXQ_STAT_ADD(queue, buf_elements, elements);
- if (is_sg)
- QETH_TXQ_STAT_INC(queue, skbs_sg);
- if (is_tso) {
- QETH_TXQ_STAT_INC(queue, skbs_tso);
- QETH_TXQ_STAT_ADD(queue, tso_bytes, frame_len);
- }
- } else {
- if (!push_len)
- kmem_cache_free(qeth_core_header_cache, hdr);
- }
+ if (rc && !push_len)
+ kmem_cache_free(qeth_core_header_cache, hdr);
+
return rc;
}
EXPORT_SYMBOL_GPL(qeth_xmit);
@@ -4361,6 +4410,10 @@ static int qeth_snmp_command(struct qeth_card *card, char __user *udata)
get_user(req_len, &ureq->hdr.req_len))
return -EFAULT;
+ /* Sanitize user input, to avoid overflows in iob size calculation: */
+ if (req_len > QETH_BUFSIZE)
+ return -EINVAL;
+
iob = qeth_get_adapter_cmd(card, IPA_SETADP_SET_SNMP_CONTROL, req_len);
if (!iob)
return -ENOMEM;
@@ -4724,7 +4777,7 @@ static int qeth_qdio_establish(struct qeth_card *card)
init_data.input_sbal_addr_array = in_sbal_ptrs;
init_data.output_sbal_addr_array = out_sbal_ptrs;
init_data.output_sbal_state_array = card->qdio.out_bufstates;
- init_data.scan_threshold = IS_IQD(card) ? 1 : 32;
+ init_data.scan_threshold = IS_IQD(card) ? 0 : 32;
if (atomic_cmpxchg(&card->qdio.state, QETH_QDIO_ALLOCATED,
QETH_QDIO_ESTABLISHED) == QETH_QDIO_ALLOCATED) {
@@ -5138,6 +5191,107 @@ int qeth_poll(struct napi_struct *napi, int budget)
}
EXPORT_SYMBOL_GPL(qeth_poll);
+static void qeth_iqd_tx_complete(struct qeth_qdio_out_q *queue,
+ unsigned int bidx, bool error, int budget)
+{
+ struct qeth_qdio_out_buffer *buffer = queue->bufs[bidx];
+ u8 sflags = buffer->buffer->element[15].sflags;
+ struct qeth_card *card = queue->card;
+
+ if (queue->bufstates && (queue->bufstates[bidx].flags &
+ QDIO_OUTBUF_STATE_FLAG_PENDING)) {
+ WARN_ON_ONCE(card->options.cq != QETH_CQ_ENABLED);
+
+ if (atomic_cmpxchg(&buffer->state, QETH_QDIO_BUF_PRIMED,
+ QETH_QDIO_BUF_PENDING) ==
+ QETH_QDIO_BUF_PRIMED)
+ qeth_notify_skbs(queue, buffer, TX_NOTIFY_PENDING);
+
+ QETH_CARD_TEXT_(card, 5, "pel%u", bidx);
+
+ /* prepare the queue slot for re-use: */
+ qeth_scrub_qdio_buffer(buffer->buffer, queue->max_elements);
+ if (qeth_init_qdio_out_buf(queue, bidx)) {
+ QETH_CARD_TEXT(card, 2, "outofbuf");
+ qeth_schedule_recovery(card);
+ }
+
+ return;
+ }
+
+ if (card->options.cq == QETH_CQ_ENABLED)
+ qeth_notify_skbs(queue, buffer,
+ qeth_compute_cq_notification(sflags, 0));
+ qeth_clear_output_buffer(queue, buffer, error, budget);
+}
+
+static int qeth_tx_poll(struct napi_struct *napi, int budget)
+{
+ struct qeth_qdio_out_q *queue = qeth_napi_to_out_queue(napi);
+ unsigned int queue_no = queue->queue_no;
+ struct qeth_card *card = queue->card;
+ struct net_device *dev = card->dev;
+ unsigned int work_done = 0;
+ struct netdev_queue *txq;
+
+ txq = netdev_get_tx_queue(dev, qeth_iqd_translate_txq(dev, queue_no));
+
+ while (1) {
+ unsigned int start, error, i;
+ unsigned int packets = 0;
+ unsigned int bytes = 0;
+ int completed;
+
+ if (qeth_out_queue_is_empty(queue)) {
+ napi_complete(napi);
+ return 0;
+ }
+
+ /* Give the CPU a breather: */
+ if (work_done >= QDIO_MAX_BUFFERS_PER_Q) {
+ QETH_TXQ_STAT_INC(queue, completion_yield);
+ if (napi_complete_done(napi, 0))
+ napi_schedule(napi);
+ return 0;
+ }
+
+ completed = qdio_inspect_queue(CARD_DDEV(card), queue_no, false,
+ &start, &error);
+ if (completed <= 0) {
+ /* Ensure we see TX completion for pending work: */
+ if (napi_complete_done(napi, 0))
+ qeth_tx_arm_timer(queue);
+ return 0;
+ }
+
+ for (i = start; i < start + completed; i++) {
+ struct qeth_qdio_out_buffer *buffer;
+ unsigned int bidx = QDIO_BUFNR(i);
+
+ buffer = queue->bufs[bidx];
+ packets += skb_queue_len(&buffer->skb_list);
+ bytes += buffer->bytes;
+
+ qeth_handle_send_error(card, buffer, error);
+ qeth_iqd_tx_complete(queue, bidx, error, budget);
+ qeth_cleanup_handled_pending(queue, bidx, false);
+ }
+
+ netdev_tx_completed_queue(txq, packets, bytes);
+ atomic_sub(completed, &queue->used_buffers);
+ work_done += completed;
+
+ /* xmit may have observed the full-condition, but not yet
+ * stopped the txq. In which case the code below won't trigger.
+ * So before returning, xmit will re-check the txq's fill level
+ * and wake it up if needed.
+ */
+ if (netif_tx_queue_stopped(txq) &&
+ !qeth_out_queue_is_full(queue))
+ netif_tx_wake_queue(txq);
+ }
+}
+
static int qeth_setassparms_inspect_rc(struct qeth_ipa_cmd *cmd)
{
if (!cmd->hdr.return_code)
@@ -6084,6 +6238,17 @@ int qeth_open(struct net_device *dev)
napi_enable(&card->napi);
local_bh_disable();
napi_schedule(&card->napi);
+ if (IS_IQD(card)) {
+ struct qeth_qdio_out_q *queue;
+ unsigned int i;
+
+ qeth_for_each_output_queue(card, queue, i) {
+ netif_tx_napi_add(dev, &queue->napi, qeth_tx_poll,
+ QETH_NAPI_WEIGHT);
+ napi_enable(&queue->napi);
+ napi_schedule(&queue->napi);
+ }
+ }
/* kick-start the NAPI softirq: */
local_bh_enable();
return 0;
@@ -6095,7 +6260,26 @@ int qeth_stop(struct net_device *dev)
struct qeth_card *card = dev->ml_priv;
QETH_CARD_TEXT(card, 4, "qethstop");
- netif_tx_disable(dev);
+ if (IS_IQD(card)) {
+ struct qeth_qdio_out_q *queue;
+ unsigned int i;
+
+ /* Quiesce the NAPI instances: */
+ qeth_for_each_output_queue(card, queue, i) {
+ napi_disable(&queue->napi);
+ del_timer_sync(&queue->timer);
+ }
+
+ /* Stop .ndo_start_xmit, might still access queue->napi. */
+ netif_tx_disable(dev);
+
+ /* Queues may get re-allocated, so remove the NAPIs here. */
+ qeth_for_each_output_queue(card, queue, i)
+ netif_napi_del(&queue->napi);
+ } else {
+ netif_tx_disable(dev);
+ }
+
napi_disable(&card->napi);
return 0;
}
diff --git a/drivers/s390/net/qeth_ethtool.c b/drivers/s390/net/qeth_ethtool.c
index 4166eb2..096698d 100644
--- a/drivers/s390/net/qeth_ethtool.c
+++ b/drivers/s390/net/qeth_ethtool.c
@@ -39,6 +39,8 @@ static const struct qeth_stats txq_stats[] = {
QETH_TXQ_STAT("TSO bytes", tso_bytes),
QETH_TXQ_STAT("Packing mode switches", packing_mode_switch),
QETH_TXQ_STAT("Queue stopped", stopped),
+ QETH_TXQ_STAT("Completion yield", completion_yield),
+ QETH_TXQ_STAT("Completion timer", completion_timer),
};
static const struct qeth_stats card_stats[] = {
diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c
index 662bd51..b8799cd 100644
--- a/drivers/s390/net/qeth_l2_main.c
+++ b/drivers/s390/net/qeth_l2_main.c
@@ -175,10 +175,8 @@ static void qeth_l2_fill_header(struct qeth_qdio_out_q *queue,
hdr->hdr.l2.id = QETH_HEADER_TYPE_L2_TSO;
} else {
hdr->hdr.l2.id = QETH_HEADER_TYPE_LAYER2;
- if (skb->ip_summed == CHECKSUM_PARTIAL) {
+ if (skb->ip_summed == CHECKSUM_PARTIAL)
qeth_tx_csum(skb, &hdr->hdr.l2.flags[1], ipv);
- QETH_TXQ_STAT_INC(queue, skbs_csum);
- }
}
/* set byte byte 3 to casting flags */
@@ -588,9 +586,10 @@ static netdev_tx_t qeth_l2_hard_start_xmit(struct sk_buff *skb,
struct qeth_card *card = dev->ml_priv;
u16 txq = skb_get_queue_mapping(skb);
struct qeth_qdio_out_q *queue;
- int tx_bytes = skb->len;
int rc;
+ if (!skb_is_gso(skb))
+ qdisc_skb_cb(skb)->pkt_len = skb->len;
if (IS_IQD(card))
txq = qeth_iqd_translate_txq(dev, txq);
queue = card->qdio.out_qs[txq];
@@ -601,11 +600,8 @@ static netdev_tx_t qeth_l2_hard_start_xmit(struct sk_buff *skb,
rc = qeth_xmit(card, skb, queue, qeth_get_ip_version(skb),
qeth_l2_fill_header);
- if (!rc) {
- QETH_TXQ_STAT_INC(queue, tx_packets);
- QETH_TXQ_STAT_ADD(queue, tx_bytes, tx_bytes);
+ if (!rc)
return NETDEV_TX_OK;
- }
QETH_TXQ_STAT_INC(queue, tx_dropped);
kfree_skb(skb);
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 54799fe..d7bfc7a 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -1957,7 +1957,6 @@ static void qeth_l3_fill_header(struct qeth_qdio_out_q *queue,
/* some HW requires combined L3+L4 csum offload: */
if (ipv == 4)
hdr->hdr.l3.ext_flags |= QETH_HDR_EXT_CSUM_HDR_REQ;
- QETH_TXQ_STAT_INC(queue, skbs_csum);
}
}
@@ -2044,9 +2043,10 @@ static netdev_tx_t qeth_l3_hard_start_xmit(struct sk_buff *skb,
u16 txq = skb_get_queue_mapping(skb);
int ipv = qeth_get_ip_version(skb);
struct qeth_qdio_out_q *queue;
- int tx_bytes = skb->len;
int rc;
+ if (!skb_is_gso(skb))
+ qdisc_skb_cb(skb)->pkt_len = skb->len;
if (IS_IQD(card)) {
queue = card->qdio.out_qs[qeth_iqd_translate_txq(dev, txq)];
@@ -2069,11 +2069,8 @@ static netdev_tx_t qeth_l3_hard_start_xmit(struct sk_buff *skb,
else
rc = qeth_xmit(card, skb, queue, ipv, qeth_l3_fill_header);
- if (!rc) {
- QETH_TXQ_STAT_INC(queue, tx_packets);
- QETH_TXQ_STAT_ADD(queue, tx_bytes, tx_bytes);
+ if (!rc)
return NETDEV_TX_OK;
- }
tx_drop:
QETH_TXQ_STAT_INC(queue, tx_dropped);
diff --git a/drivers/scsi/libsas/sas_discover.c b/drivers/scsi/libsas/sas_discover.c
index abcad09..f47b4b2 100644
--- a/drivers/scsi/libsas/sas_discover.c
+++ b/drivers/scsi/libsas/sas_discover.c
@@ -459,6 +459,7 @@ static void sas_discover_domain(struct work_struct *work)
pr_notice("ATA device seen but CONFIG_SCSI_SAS_ATA=N so cannot attach\n");
/* Fall through */
#endif
+ /* Fall through - only for the #else condition above. */
default:
error = -ENXIO;
pr_err("unhandled device %d\n", dev->dev_type);
diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 2c3bb8a..bade2e0 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -824,6 +824,7 @@ struct lpfc_hba {
uint32_t cfg_cq_poll_threshold;
uint32_t cfg_cq_max_proc_limit;
uint32_t cfg_fcp_cpu_map;
+ uint32_t cfg_fcp_mq_threshold;
uint32_t cfg_hdw_queue;
uint32_t cfg_irq_chann;
uint32_t cfg_suppress_rsp;
diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index ea62322..8d8c495 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -5709,6 +5709,19 @@ LPFC_ATTR_RW(nvme_embed_cmd, 1, 0, 2,
"Embed NVME Command in WQE");
/*
+ * lpfc_fcp_mq_threshold: Set the maximum number of Hardware Queues
+ * the driver will advertise it supports to the SCSI layer.
+ *
+ * 0 = Set nr_hw_queues by the number of CPUs or HW queues.
+ * 1,128 = Manually specify the maximum nr_hw_queue value to be set,
+ *
+ * Value range is [0,128]. Default value is 8.
+ */
+LPFC_ATTR_R(fcp_mq_threshold, LPFC_FCP_MQ_THRESHOLD_DEF,
+ LPFC_FCP_MQ_THRESHOLD_MIN, LPFC_FCP_MQ_THRESHOLD_MAX,
+ "Set the number of SCSI Queues advertised");
+
+/*
* lpfc_hdw_queue: Set the number of Hardware Queues the driver
* will advertise it supports to the NVME and SCSI layers. This also
* will map to the number of CQ/WQ pairs the driver will create.
@@ -6030,6 +6043,7 @@ struct device_attribute *lpfc_hba_attrs[] = {
&dev_attr_lpfc_cq_poll_threshold,
&dev_attr_lpfc_cq_max_proc_limit,
&dev_attr_lpfc_fcp_cpu_map,
+ &dev_attr_lpfc_fcp_mq_threshold,
&dev_attr_lpfc_hdw_queue,
&dev_attr_lpfc_irq_chann,
&dev_attr_lpfc_suppress_rsp,
@@ -7112,6 +7126,7 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)
/* Initialize first burst. Target vs Initiator are different. */
lpfc_nvme_enable_fb_init(phba, lpfc_nvme_enable_fb);
lpfc_nvmet_fb_size_init(phba, lpfc_nvmet_fb_size);
+ lpfc_fcp_mq_threshold_init(phba, lpfc_fcp_mq_threshold);
lpfc_hdw_queue_init(phba, lpfc_hdw_queue);
lpfc_irq_chann_init(phba, lpfc_irq_chann);
lpfc_enable_bbcr_init(phba, lpfc_enable_bbcr);
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index a7549ae..1ac98be 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -4309,10 +4309,12 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev)
shost->max_cmd_len = 16;
if (phba->sli_rev == LPFC_SLI_REV4) {
- if (phba->cfg_fcp_io_sched == LPFC_FCP_SCHED_BY_HDWQ)
- shost->nr_hw_queues = phba->cfg_hdw_queue;
- else
- shost->nr_hw_queues = phba->sli4_hba.num_present_cpu;
+ if (!phba->cfg_fcp_mq_threshold ||
+ phba->cfg_fcp_mq_threshold > phba->cfg_hdw_queue)
+ phba->cfg_fcp_mq_threshold = phba->cfg_hdw_queue;
+
+ shost->nr_hw_queues = min_t(int, 2 * num_possible_nodes(),
+ phba->cfg_fcp_mq_threshold);
shost->dma_boundary =
phba->sli4_hba.pc_sli4_params.sge_supp_len-1;
diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h
index 3aeca38..329f7aa 100644
--- a/drivers/scsi/lpfc/lpfc_sli4.h
+++ b/drivers/scsi/lpfc/lpfc_sli4.h
@@ -44,6 +44,11 @@
#define LPFC_HBA_HDWQ_MAX 128
#define LPFC_HBA_HDWQ_DEF 0
+/* FCP MQ queue count limiting */
+#define LPFC_FCP_MQ_THRESHOLD_MIN 0
+#define LPFC_FCP_MQ_THRESHOLD_MAX 128
+#define LPFC_FCP_MQ_THRESHOLD_DEF 8
+
/* Common buffer size to accomidate SCSI and NVME IO buffers */
#define LPFC_COMMON_IO_BUF_SZ 768
diff --git a/drivers/scsi/qla2xxx/qla_attr.c b/drivers/scsi/qla2xxx/qla_attr.c
index 8d560c5..6b7b390 100644
--- a/drivers/scsi/qla2xxx/qla_attr.c
+++ b/drivers/scsi/qla2xxx/qla_attr.c
@@ -2956,6 +2956,8 @@ qla24xx_vport_delete(struct fc_vport *fc_vport)
dma_free_coherent(&ha->pdev->dev, vha->gnl.size, vha->gnl.l,
vha->gnl.ldma);
+ vha->gnl.l = NULL;
+
vfree(vha->scan.l);
if (vha->qpair && vha->qpair->vp_idx == vha->vp_idx) {
diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index 2e58cff..98e60a3 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -3440,6 +3440,12 @@ qla2x00_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
return 0;
probe_failed:
+ if (base_vha->gnl.l) {
+ dma_free_coherent(&ha->pdev->dev, base_vha->gnl.size,
+ base_vha->gnl.l, base_vha->gnl.ldma);
+ base_vha->gnl.l = NULL;
+ }
+
if (base_vha->timer_active)
qla2x00_stop_timer(base_vha);
base_vha->flags.online = 0;
@@ -3673,7 +3679,7 @@ qla2x00_remove_one(struct pci_dev *pdev)
if (!atomic_read(&pdev->enable_cnt)) {
dma_free_coherent(&ha->pdev->dev, base_vha->gnl.size,
base_vha->gnl.l, base_vha->gnl.ldma);
-
+ base_vha->gnl.l = NULL;
scsi_host_put(base_vha->host);
kfree(ha);
pci_set_drvdata(pdev, NULL);
@@ -3713,6 +3719,8 @@ qla2x00_remove_one(struct pci_dev *pdev)
dma_free_coherent(&ha->pdev->dev,
base_vha->gnl.size, base_vha->gnl.l, base_vha->gnl.ldma);
+ base_vha->gnl.l = NULL;
+
vfree(base_vha->scan.l);
if (IS_QLAFX00(ha))
@@ -4816,6 +4824,7 @@ struct scsi_qla_host *qla2x00_create_host(struct scsi_host_template *sht,
"Alloc failed for scan database.\n");
dma_free_coherent(&ha->pdev->dev, vha->gnl.size,
vha->gnl.l, vha->gnl.ldma);
+ vha->gnl.l = NULL;
scsi_remove_host(vha->host);
return NULL;
}
diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
index e274053..029da74 100644
--- a/drivers/scsi/ufs/ufshcd.c
+++ b/drivers/scsi/ufs/ufshcd.c
@@ -7062,6 +7062,9 @@ static inline int ufshcd_config_vreg_lpm(struct ufs_hba *hba,
static inline int ufshcd_config_vreg_hpm(struct ufs_hba *hba,
struct ufs_vreg *vreg)
{
+ if (!vreg)
+ return 0;
+
return ufshcd_config_vreg_load(hba->dev, vreg, vreg->max_uA);
}
diff --git a/drivers/soc/ixp4xx/Kconfig b/drivers/soc/ixp4xx/Kconfig
index de2e62c..e3eb19b 100644
--- a/drivers/soc/ixp4xx/Kconfig
+++ b/drivers/soc/ixp4xx/Kconfig
@@ -1,4 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
+if ARCH_IXP4XX || COMPILE_TEST
+
menu "IXP4xx SoC drivers"
config IXP4XX_QMGR
@@ -15,3 +17,5 @@
and is automatically selected by Ethernet and HSS drivers.
endmenu
+
+endif
diff --git a/drivers/soc/ti/pm33xx.c b/drivers/soc/ti/pm33xx.c
index bb77c22..ccc6d53 100644
--- a/drivers/soc/ti/pm33xx.c
+++ b/drivers/soc/ti/pm33xx.c
@@ -141,7 +141,7 @@ static int __init am43xx_map_gic(void)
}
#ifdef CONFIG_SUSPEND
-struct wkup_m3_wakeup_src rtc_wake_src(void)
+static struct wkup_m3_wakeup_src rtc_wake_src(void)
{
u32 i;
@@ -157,7 +157,7 @@ struct wkup_m3_wakeup_src rtc_wake_src(void)
return rtc_ext_wakeup;
}
-int am33xx_rtc_only_idle(unsigned long wfi_flags)
+static int am33xx_rtc_only_idle(unsigned long wfi_flags)
{
omap_rtc_power_off_program(&omap_rtc->dev);
am33xx_do_wfi_sram(wfi_flags);
@@ -252,7 +252,7 @@ static int am33xx_pm_begin(suspend_state_t state)
if (state == PM_SUSPEND_MEM && pm_ops->check_off_mode_enable()) {
nvmem = devm_nvmem_device_get(&omap_rtc->dev,
"omap_rtc_scratch0");
- if (nvmem)
+ if (!IS_ERR(nvmem))
nvmem_device_write(nvmem, RTC_SCRATCH_MAGIC_REG * 4, 4,
(void *)&rtc_magic_val);
rtc_only_idle = 1;
@@ -278,9 +278,12 @@ static void am33xx_pm_end(void)
struct nvmem_device *nvmem;
nvmem = devm_nvmem_device_get(&omap_rtc->dev, "omap_rtc_scratch0");
+ if (IS_ERR(nvmem))
+ return;
+
m3_ipc->ops->finish_low_power(m3_ipc);
if (rtc_only_idle) {
- if (retrigger_irq)
+ if (retrigger_irq) {
/*
* 32 bits of Interrupt Set-Pending correspond to 32
* 32 interrupts. Compute the bit offset of the
@@ -291,8 +294,10 @@ static void am33xx_pm_end(void)
writel_relaxed(1 << (retrigger_irq & 31),
gic_dist_base + GIC_INT_SET_PENDING_BASE
+ retrigger_irq / 32 * 4);
- nvmem_device_write(nvmem, RTC_SCRATCH_MAGIC_REG * 4, 4,
- (void *)&val);
+ }
+
+ nvmem_device_write(nvmem, RTC_SCRATCH_MAGIC_REG * 4, 4,
+ (void *)&val);
}
rtc_only_idle = 0;
@@ -415,7 +420,7 @@ static int am33xx_pm_rtc_setup(void)
nvmem = devm_nvmem_device_get(&omap_rtc->dev,
"omap_rtc_scratch0");
- if (nvmem) {
+ if (!IS_ERR(nvmem)) {
nvmem_device_read(nvmem, RTC_SCRATCH_MAGIC_REG * 4,
4, (void *)&rtc_magic_val);
if ((rtc_magic_val & 0xffff) != RTC_REG_BOOT_MAGIC)
diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c
index 04eda11..661bb93 100644
--- a/drivers/target/target_core_user.c
+++ b/drivers/target/target_core_user.c
@@ -1132,14 +1132,16 @@ static void tcmu_handle_completion(struct tcmu_cmd *cmd, struct tcmu_cmd_entry *
struct se_cmd *se_cmd = cmd->se_cmd;
struct tcmu_dev *udev = cmd->tcmu_dev;
bool read_len_valid = false;
- uint32_t read_len = se_cmd->data_length;
+ uint32_t read_len;
/*
* cmd has been completed already from timeout, just reclaim
* data area space and free cmd
*/
- if (test_bit(TCMU_CMD_BIT_EXPIRED, &cmd->flags))
+ if (test_bit(TCMU_CMD_BIT_EXPIRED, &cmd->flags)) {
+ WARN_ON_ONCE(se_cmd);
goto out;
+ }
list_del_init(&cmd->queue_entry);
@@ -1152,6 +1154,7 @@ static void tcmu_handle_completion(struct tcmu_cmd *cmd, struct tcmu_cmd_entry *
goto done;
}
+ read_len = se_cmd->data_length;
if (se_cmd->data_direction == DMA_FROM_DEVICE &&
(entry->hdr.uflags & TCMU_UFLAG_READ_LEN) && entry->rsp.read_len) {
read_len_valid = true;
@@ -1307,6 +1310,7 @@ static int tcmu_check_expired_cmd(int id, void *p, void *data)
*/
scsi_status = SAM_STAT_CHECK_CONDITION;
list_del_init(&cmd->queue_entry);
+ cmd->se_cmd = NULL;
} else {
list_del_init(&cmd->queue_entry);
idr_remove(&udev->commands, id);
@@ -2022,6 +2026,7 @@ static void tcmu_reset_ring(struct tcmu_dev *udev, u8 err_level)
idr_remove(&udev->commands, i);
if (!test_bit(TCMU_CMD_BIT_EXPIRED, &cmd->flags)) {
+ WARN_ON(!cmd->se_cmd);
list_del_init(&cmd->queue_entry);
if (err_level == 1) {
/*
diff --git a/drivers/usb/chipidea/udc.c b/drivers/usb/chipidea/udc.c
index 6a5ee8e..67ad40b 100644
--- a/drivers/usb/chipidea/udc.c
+++ b/drivers/usb/chipidea/udc.c
@@ -709,12 +709,6 @@ static int _gadget_stop_activity(struct usb_gadget *gadget)
struct ci_hdrc *ci = container_of(gadget, struct ci_hdrc, gadget);
unsigned long flags;
- spin_lock_irqsave(&ci->lock, flags);
- ci->gadget.speed = USB_SPEED_UNKNOWN;
- ci->remote_wakeup = 0;
- ci->suspended = 0;
- spin_unlock_irqrestore(&ci->lock, flags);
-
/* flush all endpoints */
gadget_for_each_ep(ep, gadget) {
usb_ep_fifo_flush(ep);
@@ -732,6 +726,12 @@ static int _gadget_stop_activity(struct usb_gadget *gadget)
ci->status = NULL;
}
+ spin_lock_irqsave(&ci->lock, flags);
+ ci->gadget.speed = USB_SPEED_UNKNOWN;
+ ci->remote_wakeup = 0;
+ ci->suspended = 0;
+ spin_unlock_irqrestore(&ci->lock, flags);
+
return 0;
}
@@ -1303,6 +1303,10 @@ static int ep_disable(struct usb_ep *ep)
return -EBUSY;
spin_lock_irqsave(hwep->lock, flags);
+ if (hwep->ci->gadget.speed == USB_SPEED_UNKNOWN) {
+ spin_unlock_irqrestore(hwep->lock, flags);
+ return 0;
+ }
/* only internal SW should disable ctrl endpts */
@@ -1392,6 +1396,10 @@ static int ep_queue(struct usb_ep *ep, struct usb_request *req,
return -EINVAL;
spin_lock_irqsave(hwep->lock, flags);
+ if (hwep->ci->gadget.speed == USB_SPEED_UNKNOWN) {
+ spin_unlock_irqrestore(hwep->lock, flags);
+ return 0;
+ }
retval = _ep_queue(ep, req, gfp_flags);
spin_unlock_irqrestore(hwep->lock, flags);
return retval;
@@ -1415,8 +1423,8 @@ static int ep_dequeue(struct usb_ep *ep, struct usb_request *req)
return -EINVAL;
spin_lock_irqsave(hwep->lock, flags);
-
- hw_ep_flush(hwep->ci, hwep->num, hwep->dir);
+ if (hwep->ci->gadget.speed != USB_SPEED_UNKNOWN)
+ hw_ep_flush(hwep->ci, hwep->num, hwep->dir);
list_for_each_entry_safe(node, tmpnode, &hwreq->tds, td) {
dma_pool_free(hwep->td_pool, node->ptr, node->dma);
@@ -1487,6 +1495,10 @@ static void ep_fifo_flush(struct usb_ep *ep)
}
spin_lock_irqsave(hwep->lock, flags);
+ if (hwep->ci->gadget.speed == USB_SPEED_UNKNOWN) {
+ spin_unlock_irqrestore(hwep->lock, flags);
+ return;
+ }
hw_ep_flush(hwep->ci, hwep->num, hwep->dir);
@@ -1559,6 +1571,10 @@ static int ci_udc_wakeup(struct usb_gadget *_gadget)
int ret = 0;
spin_lock_irqsave(&ci->lock, flags);
+ if (ci->gadget.speed == USB_SPEED_UNKNOWN) {
+ spin_unlock_irqrestore(&ci->lock, flags);
+ return 0;
+ }
if (!ci->remote_wakeup) {
ret = -EOPNOTSUPP;
goto out;
diff --git a/drivers/usb/class/cdc-wdm.c b/drivers/usb/class/cdc-wdm.c
index a7824a5..70afb2c 100644
--- a/drivers/usb/class/cdc-wdm.c
+++ b/drivers/usb/class/cdc-wdm.c
@@ -587,10 +587,20 @@ static int wdm_flush(struct file *file, fl_owner_t id)
{
struct wdm_device *desc = file->private_data;
- wait_event(desc->wait, !test_bit(WDM_IN_USE, &desc->flags));
+ wait_event(desc->wait,
+ /*
+ * needs both flags. We cannot do with one
+ * because resetting it would cause a race
+ * with write() yet we need to signal
+ * a disconnect
+ */
+ !test_bit(WDM_IN_USE, &desc->flags) ||
+ test_bit(WDM_DISCONNECTING, &desc->flags));
/* cannot dereference desc->intf if WDM_DISCONNECTING */
- if (desc->werr < 0 && !test_bit(WDM_DISCONNECTING, &desc->flags))
+ if (test_bit(WDM_DISCONNECTING, &desc->flags))
+ return -ENODEV;
+ if (desc->werr < 0)
dev_err(&desc->intf->dev, "Error in flush path: %d\n",
desc->werr);
@@ -974,8 +984,6 @@ static void wdm_disconnect(struct usb_interface *intf)
spin_lock_irqsave(&desc->iuspin, flags);
set_bit(WDM_DISCONNECTING, &desc->flags);
set_bit(WDM_READ, &desc->flags);
- /* to terminate pending flushes */
- clear_bit(WDM_IN_USE, &desc->flags);
spin_unlock_irqrestore(&desc->iuspin, flags);
wake_up_all(&desc->wait);
mutex_lock(&desc->rlock);
diff --git a/drivers/usb/class/usbtmc.c b/drivers/usb/class/usbtmc.c
index 4942122b..36858dd 100644
--- a/drivers/usb/class/usbtmc.c
+++ b/drivers/usb/class/usbtmc.c
@@ -2362,8 +2362,11 @@ static int usbtmc_probe(struct usb_interface *intf,
goto err_put;
}
+ retcode = -EINVAL;
data->bulk_in = bulk_in->bEndpointAddress;
data->wMaxPacketSize = usb_endpoint_maxp(bulk_in);
+ if (!data->wMaxPacketSize)
+ goto err_put;
dev_dbg(&intf->dev, "Found bulk in endpoint at %u\n", data->bulk_in);
data->bulk_out = bulk_out->bEndpointAddress;
diff --git a/drivers/usb/core/hcd-pci.c b/drivers/usb/core/hcd-pci.c
index 0343246..7537681 100644
--- a/drivers/usb/core/hcd-pci.c
+++ b/drivers/usb/core/hcd-pci.c
@@ -216,17 +216,18 @@ int usb_hcd_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
/* EHCI, OHCI */
hcd->rsrc_start = pci_resource_start(dev, 0);
hcd->rsrc_len = pci_resource_len(dev, 0);
- if (!request_mem_region(hcd->rsrc_start, hcd->rsrc_len,
- driver->description)) {
+ if (!devm_request_mem_region(&dev->dev, hcd->rsrc_start,
+ hcd->rsrc_len, driver->description)) {
dev_dbg(&dev->dev, "controller already in use\n");
retval = -EBUSY;
goto put_hcd;
}
- hcd->regs = ioremap_nocache(hcd->rsrc_start, hcd->rsrc_len);
+ hcd->regs = devm_ioremap_nocache(&dev->dev, hcd->rsrc_start,
+ hcd->rsrc_len);
if (hcd->regs == NULL) {
dev_dbg(&dev->dev, "error mapping memory\n");
retval = -EFAULT;
- goto release_mem_region;
+ goto put_hcd;
}
} else {
@@ -240,8 +241,8 @@ int usb_hcd_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
hcd->rsrc_start = pci_resource_start(dev, region);
hcd->rsrc_len = pci_resource_len(dev, region);
- if (request_region(hcd->rsrc_start, hcd->rsrc_len,
- driver->description))
+ if (devm_request_region(&dev->dev, hcd->rsrc_start,
+ hcd->rsrc_len, driver->description))
break;
}
if (region == PCI_ROM_RESOURCE) {
@@ -275,20 +276,13 @@ int usb_hcd_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
}
if (retval != 0)
- goto unmap_registers;
+ goto put_hcd;
device_wakeup_enable(hcd->self.controller);
if (pci_dev_run_wake(dev))
pm_runtime_put_noidle(&dev->dev);
return retval;
-unmap_registers:
- if (driver->flags & HCD_MEMORY) {
- iounmap(hcd->regs);
-release_mem_region:
- release_mem_region(hcd->rsrc_start, hcd->rsrc_len);
- } else
- release_region(hcd->rsrc_start, hcd->rsrc_len);
put_hcd:
usb_put_hcd(hcd);
disable_pci:
@@ -347,14 +341,6 @@ void usb_hcd_pci_remove(struct pci_dev *dev)
dev_set_drvdata(&dev->dev, NULL);
up_read(&companions_rwsem);
}
-
- if (hcd->driver->flags & HCD_MEMORY) {
- iounmap(hcd->regs);
- release_mem_region(hcd->rsrc_start, hcd->rsrc_len);
- } else {
- release_region(hcd->rsrc_start, hcd->rsrc_len);
- }
-
usb_put_hcd(hcd);
pci_disable_device(dev);
}
diff --git a/drivers/usb/gadget/udc/lpc32xx_udc.c b/drivers/usb/gadget/udc/lpc32xx_udc.c
index 5f1b14f..bb6af6b 100644
--- a/drivers/usb/gadget/udc/lpc32xx_udc.c
+++ b/drivers/usb/gadget/udc/lpc32xx_udc.c
@@ -2265,7 +2265,7 @@ static void udc_handle_ep0_setup(struct lpc32xx_udc *udc)
default:
break;
}
-
+ break;
case USB_REQ_SET_ADDRESS:
if (reqtype == (USB_TYPE_STANDARD | USB_RECIP_DEVICE)) {
diff --git a/drivers/usb/host/ohci-hcd.c b/drivers/usb/host/ohci-hcd.c
index b457fda..1fe3dee 100644
--- a/drivers/usb/host/ohci-hcd.c
+++ b/drivers/usb/host/ohci-hcd.c
@@ -419,8 +419,7 @@ static void ohci_usb_reset (struct ohci_hcd *ohci)
* other cases where the next software may expect clean state from the
* "firmware". this is bus-neutral, unlike shutdown() methods.
*/
-static void
-ohci_shutdown (struct usb_hcd *hcd)
+static void _ohci_shutdown(struct usb_hcd *hcd)
{
struct ohci_hcd *ohci;
@@ -436,6 +435,16 @@ ohci_shutdown (struct usb_hcd *hcd)
ohci->rh_state = OHCI_RH_HALTED;
}
+static void ohci_shutdown(struct usb_hcd *hcd)
+{
+ struct ohci_hcd *ohci = hcd_to_ohci(hcd);
+ unsigned long flags;
+
+ spin_lock_irqsave(&ohci->lock, flags);
+ _ohci_shutdown(hcd);
+ spin_unlock_irqrestore(&ohci->lock, flags);
+}
+
/*-------------------------------------------------------------------------*
* HC functions
*-------------------------------------------------------------------------*/
@@ -760,7 +769,7 @@ static void io_watchdog_func(struct timer_list *t)
died:
usb_hc_died(ohci_to_hcd(ohci));
ohci_dump(ohci);
- ohci_shutdown(ohci_to_hcd(ohci));
+ _ohci_shutdown(ohci_to_hcd(ohci));
goto done;
} else {
/* No write back because the done queue was empty */
diff --git a/drivers/usb/host/xhci-rcar.c b/drivers/usb/host/xhci-rcar.c
index 8616c52..2b0ccd1 100644
--- a/drivers/usb/host/xhci-rcar.c
+++ b/drivers/usb/host/xhci-rcar.c
@@ -104,7 +104,7 @@ static int xhci_rcar_is_gen2(struct device *dev)
return of_device_is_compatible(node, "renesas,xhci-r8a7790") ||
of_device_is_compatible(node, "renesas,xhci-r8a7791") ||
of_device_is_compatible(node, "renesas,xhci-r8a7793") ||
- of_device_is_compatible(node, "renensas,rcar-gen2-xhci");
+ of_device_is_compatible(node, "renesas,rcar-gen2-xhci");
}
static int xhci_rcar_is_gen3(struct device *dev)
diff --git a/drivers/usb/host/xhci-tegra.c b/drivers/usb/host/xhci-tegra.c
index dafc659..2ff7c91 100644
--- a/drivers/usb/host/xhci-tegra.c
+++ b/drivers/usb/host/xhci-tegra.c
@@ -1194,6 +1194,16 @@ static int tegra_xusb_probe(struct platform_device *pdev)
tegra_xusb_config(tegra, regs);
+ /*
+ * The XUSB Falcon microcontroller can only address 40 bits, so set
+ * the DMA mask accordingly.
+ */
+ err = dma_set_mask_and_coherent(tegra->dev, DMA_BIT_MASK(40));
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to set DMA mask: %d\n", err);
+ goto put_rpm;
+ }
+
err = tegra_xusb_load_firmware(tegra);
if (err < 0) {
dev_err(&pdev->dev, "failed to load firmware: %d\n", err);
diff --git a/drivers/usb/storage/realtek_cr.c b/drivers/usb/storage/realtek_cr.c
index cc794e2..1d9ce9cb 100644
--- a/drivers/usb/storage/realtek_cr.c
+++ b/drivers/usb/storage/realtek_cr.c
@@ -38,7 +38,7 @@ MODULE_LICENSE("GPL");
static int auto_delink_en = 1;
module_param(auto_delink_en, int, S_IRUGO | S_IWUSR);
-MODULE_PARM_DESC(auto_delink_en, "enable auto delink");
+MODULE_PARM_DESC(auto_delink_en, "auto delink mode (0=firmware, 1=software [default])");
#ifdef CONFIG_REALTEK_AUTOPM
static int ss_en = 1;
@@ -996,12 +996,15 @@ static int init_realtek_cr(struct us_data *us)
goto INIT_FAIL;
}
- if (CHECK_FW_VER(chip, 0x5888) || CHECK_FW_VER(chip, 0x5889) ||
- CHECK_FW_VER(chip, 0x5901))
- SET_AUTO_DELINK(chip);
- if (STATUS_LEN(chip) == 16) {
- if (SUPPORT_AUTO_DELINK(chip))
+ if (CHECK_PID(chip, 0x0138) || CHECK_PID(chip, 0x0158) ||
+ CHECK_PID(chip, 0x0159)) {
+ if (CHECK_FW_VER(chip, 0x5888) || CHECK_FW_VER(chip, 0x5889) ||
+ CHECK_FW_VER(chip, 0x5901))
SET_AUTO_DELINK(chip);
+ if (STATUS_LEN(chip) == 16) {
+ if (SUPPORT_AUTO_DELINK(chip))
+ SET_AUTO_DELINK(chip);
+ }
}
#ifdef CONFIG_REALTEK_AUTOPM
if (ss_en)
diff --git a/drivers/usb/storage/unusual_devs.h b/drivers/usb/storage/unusual_devs.h
index ea0d27a..1cd9b63 100644
--- a/drivers/usb/storage/unusual_devs.h
+++ b/drivers/usb/storage/unusual_devs.h
@@ -2100,7 +2100,7 @@ UNUSUAL_DEV( 0x14cd, 0x6600, 0x0201, 0x0201,
US_FL_IGNORE_RESIDUE ),
/* Reported by Michael Büsch <m@bues.ch> */
-UNUSUAL_DEV( 0x152d, 0x0567, 0x0114, 0x0116,
+UNUSUAL_DEV( 0x152d, 0x0567, 0x0114, 0x0117,
"JMicron",
"USB to ATA/ATAPI Bridge",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
diff --git a/drivers/usb/typec/tcpm/tcpm.c b/drivers/usb/typec/tcpm/tcpm.c
index 15abe1d..bcfdb55 100644
--- a/drivers/usb/typec/tcpm/tcpm.c
+++ b/drivers/usb/typec/tcpm/tcpm.c
@@ -1446,7 +1446,7 @@ static enum pdo_err tcpm_caps_err(struct tcpm_port *port, const u32 *pdo,
else if ((pdo_min_voltage(pdo[i]) ==
pdo_min_voltage(pdo[i - 1])) &&
(pdo_max_voltage(pdo[i]) ==
- pdo_min_voltage(pdo[i - 1])))
+ pdo_max_voltage(pdo[i - 1])))
return PDO_ERR_DUPE_PDO;
break;
/*
diff --git a/drivers/video/fbdev/acornfb.c b/drivers/video/fbdev/acornfb.c
index 92f23e3..7cacae5 100644
--- a/drivers/video/fbdev/acornfb.c
+++ b/drivers/video/fbdev/acornfb.c
@@ -858,6 +858,7 @@ static void acornfb_parse_dram(char *opt)
case 'M':
case 'm':
size *= 1024;
+ /* Fall through */
case 'K':
case 'k':
size *= 1024;
diff --git a/drivers/watchdog/wdt285.c b/drivers/watchdog/wdt285.c
index 4eacfb1..eb729d7 100644
--- a/drivers/watchdog/wdt285.c
+++ b/drivers/watchdog/wdt285.c
@@ -168,7 +168,7 @@ static long watchdog_ioctl(struct file *file, unsigned int cmd,
soft_margin = new_margin;
reload = soft_margin * (mem_fclk_21285 / 256);
watchdog_ping();
- /* Fall */
+ /* Fall through */
case WDIOC_GETTIMEOUT:
ret = put_user(soft_margin, int_arg);
break;
diff --git a/fs/afs/cell.c b/fs/afs/cell.c
index a2a8711..fd5133e 100644
--- a/fs/afs/cell.c
+++ b/fs/afs/cell.c
@@ -74,6 +74,7 @@ struct afs_cell *afs_lookup_cell_rcu(struct afs_net *net,
cell = rcu_dereference_raw(net->ws_cell);
if (cell) {
afs_get_cell(cell);
+ ret = 0;
break;
}
ret = -EDESTADDRREQ;
@@ -108,6 +109,9 @@ struct afs_cell *afs_lookup_cell_rcu(struct afs_net *net,
done_seqretry(&net->cells_lock, seq);
+ if (ret != 0 && cell)
+ afs_put_cell(net, cell);
+
return ret == 0 ? cell : ERR_PTR(ret);
}
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 81207dc..139b4e3 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -959,7 +959,8 @@ static struct dentry *afs_lookup(struct inode *dir, struct dentry *dentry,
inode ? AFS_FS_I(inode) : NULL);
} else {
trace_afs_lookup(dvnode, &dentry->d_name,
- inode ? AFS_FS_I(inode) : NULL);
+ IS_ERR_OR_NULL(inode) ? NULL
+ : AFS_FS_I(inode));
}
return d;
}
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index 2575503..ca24528 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -2171,7 +2171,7 @@ int yfs_fs_store_opaque_acl2(struct afs_fs_cursor *fc, const struct afs_acl *acl
key_serial(fc->key), vnode->fid.vid, vnode->fid.vnode);
size = round_up(acl->size, 4);
- call = afs_alloc_flat_call(net, &yfs_RXYFSStoreStatus,
+ call = afs_alloc_flat_call(net, &yfs_RXYFSStoreOpaqueACL2,
sizeof(__be32) * 2 +
sizeof(struct yfs_xdr_YFSFid) +
sizeof(__be32) + size,
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index e078cc5..b3c8b88 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -913,8 +913,9 @@ static int ceph_writepages_start(struct address_space *mapping,
if (page_offset(page) >= ceph_wbc.i_size) {
dout("%p page eof %llu\n",
page, ceph_wbc.i_size);
- if (ceph_wbc.size_stable ||
- page_offset(page) >= i_size_read(inode))
+ if ((ceph_wbc.size_stable ||
+ page_offset(page) >= i_size_read(inode)) &&
+ clear_page_dirty_for_io(page))
mapping->a_ops->invalidatepage(page,
0, PAGE_SIZE);
unlock_page(page);
diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c
index d98dcd9..ce0f565 100644
--- a/fs/ceph/caps.c
+++ b/fs/ceph/caps.c
@@ -1301,6 +1301,7 @@ static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
{
struct ceph_inode_info *ci = cap->ci;
struct inode *inode = &ci->vfs_inode;
+ struct ceph_buffer *old_blob = NULL;
struct cap_msg_args arg;
int held, revoking;
int wake = 0;
@@ -1365,7 +1366,7 @@ static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
ci->i_requested_max_size = arg.max_size;
if (flushing & CEPH_CAP_XATTR_EXCL) {
- __ceph_build_xattrs_blob(ci);
+ old_blob = __ceph_build_xattrs_blob(ci);
arg.xattr_version = ci->i_xattrs.version;
arg.xattr_buf = ci->i_xattrs.blob;
} else {
@@ -1409,6 +1410,8 @@ static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
spin_unlock(&ci->i_ceph_lock);
+ ceph_buffer_put(old_blob);
+
ret = send_cap_msg(&arg);
if (ret < 0) {
dout("error sending cap msg, must requeue %p\n", inode);
diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c
index 791f84a..18500ede 100644
--- a/fs/ceph/inode.c
+++ b/fs/ceph/inode.c
@@ -736,6 +736,7 @@ static int fill_inode(struct inode *inode, struct page *locked_page,
int issued, new_issued, info_caps;
struct timespec64 mtime, atime, ctime;
struct ceph_buffer *xattr_blob = NULL;
+ struct ceph_buffer *old_blob = NULL;
struct ceph_string *pool_ns = NULL;
struct ceph_cap *new_cap = NULL;
int err = 0;
@@ -881,7 +882,7 @@ static int fill_inode(struct inode *inode, struct page *locked_page,
if ((ci->i_xattrs.version == 0 || !(issued & CEPH_CAP_XATTR_EXCL)) &&
le64_to_cpu(info->xattr_version) > ci->i_xattrs.version) {
if (ci->i_xattrs.blob)
- ceph_buffer_put(ci->i_xattrs.blob);
+ old_blob = ci->i_xattrs.blob;
ci->i_xattrs.blob = xattr_blob;
if (xattr_blob)
memcpy(ci->i_xattrs.blob->vec.iov_base,
@@ -1022,8 +1023,8 @@ static int fill_inode(struct inode *inode, struct page *locked_page,
out:
if (new_cap)
ceph_put_cap(mdsc, new_cap);
- if (xattr_blob)
- ceph_buffer_put(xattr_blob);
+ ceph_buffer_put(old_blob);
+ ceph_buffer_put(xattr_blob);
ceph_put_string(pool_ns);
return err;
}
diff --git a/fs/ceph/locks.c b/fs/ceph/locks.c
index ac9b53b..5083e23 100644
--- a/fs/ceph/locks.c
+++ b/fs/ceph/locks.c
@@ -111,8 +111,7 @@ static int ceph_lock_message(u8 lock_type, u16 operation, struct inode *inode,
req->r_wait_for_completion = ceph_lock_wait_for_completion;
err = ceph_mdsc_do_request(mdsc, inode, req);
-
- if (operation == CEPH_MDS_OP_GETFILELOCK) {
+ if (!err && operation == CEPH_MDS_OP_GETFILELOCK) {
fl->fl_pid = -le64_to_cpu(req->r_reply_info.filelock_reply->pid);
if (CEPH_LOCK_SHARED == req->r_reply_info.filelock_reply->type)
fl->fl_type = F_RDLCK;
diff --git a/fs/ceph/snap.c b/fs/ceph/snap.c
index 4c6494e..ccfcc66 100644
--- a/fs/ceph/snap.c
+++ b/fs/ceph/snap.c
@@ -465,6 +465,7 @@ void ceph_queue_cap_snap(struct ceph_inode_info *ci)
struct inode *inode = &ci->vfs_inode;
struct ceph_cap_snap *capsnap;
struct ceph_snap_context *old_snapc, *new_snapc;
+ struct ceph_buffer *old_blob = NULL;
int used, dirty;
capsnap = kzalloc(sizeof(*capsnap), GFP_NOFS);
@@ -541,7 +542,7 @@ void ceph_queue_cap_snap(struct ceph_inode_info *ci)
capsnap->gid = inode->i_gid;
if (dirty & CEPH_CAP_XATTR_EXCL) {
- __ceph_build_xattrs_blob(ci);
+ old_blob = __ceph_build_xattrs_blob(ci);
capsnap->xattr_blob =
ceph_buffer_get(ci->i_xattrs.blob);
capsnap->xattr_version = ci->i_xattrs.version;
@@ -584,6 +585,7 @@ void ceph_queue_cap_snap(struct ceph_inode_info *ci)
}
spin_unlock(&ci->i_ceph_lock);
+ ceph_buffer_put(old_blob);
kfree(capsnap);
ceph_put_snap_context(old_snapc);
}
diff --git a/fs/ceph/super.h b/fs/ceph/super.h
index d2352fd9..6b9f1ee 100644
--- a/fs/ceph/super.h
+++ b/fs/ceph/super.h
@@ -926,7 +926,7 @@ extern int ceph_getattr(const struct path *path, struct kstat *stat,
int __ceph_setxattr(struct inode *, const char *, const void *, size_t, int);
ssize_t __ceph_getxattr(struct inode *, const char *, void *, size_t);
extern ssize_t ceph_listxattr(struct dentry *, char *, size_t);
-extern void __ceph_build_xattrs_blob(struct ceph_inode_info *ci);
+extern struct ceph_buffer *__ceph_build_xattrs_blob(struct ceph_inode_info *ci);
extern void __ceph_destroy_xattrs(struct ceph_inode_info *ci);
extern const struct xattr_handler *ceph_xattr_handlers[];
diff --git a/fs/ceph/xattr.c b/fs/ceph/xattr.c
index 37b458a..939eab7 100644
--- a/fs/ceph/xattr.c
+++ b/fs/ceph/xattr.c
@@ -754,12 +754,15 @@ static int __get_required_blob_size(struct ceph_inode_info *ci, int name_size,
/*
* If there are dirty xattrs, reencode xattrs into the prealloc_blob
- * and swap into place.
+ * and swap into place. It returns the old i_xattrs.blob (or NULL) so
+ * that it can be freed by the caller as the i_ceph_lock is likely to be
+ * held.
*/
-void __ceph_build_xattrs_blob(struct ceph_inode_info *ci)
+struct ceph_buffer *__ceph_build_xattrs_blob(struct ceph_inode_info *ci)
{
struct rb_node *p;
struct ceph_inode_xattr *xattr = NULL;
+ struct ceph_buffer *old_blob = NULL;
void *dest;
dout("__build_xattrs_blob %p\n", &ci->vfs_inode);
@@ -790,12 +793,14 @@ void __ceph_build_xattrs_blob(struct ceph_inode_info *ci)
dest - ci->i_xattrs.prealloc_blob->vec.iov_base;
if (ci->i_xattrs.blob)
- ceph_buffer_put(ci->i_xattrs.blob);
+ old_blob = ci->i_xattrs.blob;
ci->i_xattrs.blob = ci->i_xattrs.prealloc_blob;
ci->i_xattrs.prealloc_blob = NULL;
ci->i_xattrs.dirty = false;
ci->i_xattrs.version++;
}
+
+ return old_blob;
}
static inline int __get_request_mask(struct inode *in) {
@@ -1036,6 +1041,7 @@ int __ceph_setxattr(struct inode *inode, const char *name,
struct ceph_inode_info *ci = ceph_inode(inode);
struct ceph_mds_client *mdsc = ceph_sb_to_client(inode->i_sb)->mdsc;
struct ceph_cap_flush *prealloc_cf = NULL;
+ struct ceph_buffer *old_blob = NULL;
int issued;
int err;
int dirty = 0;
@@ -1109,13 +1115,15 @@ int __ceph_setxattr(struct inode *inode, const char *name,
struct ceph_buffer *blob;
spin_unlock(&ci->i_ceph_lock);
- dout(" preaallocating new blob size=%d\n", required_blob_size);
+ ceph_buffer_put(old_blob); /* Shouldn't be required */
+ dout(" pre-allocating new blob size=%d\n", required_blob_size);
blob = ceph_buffer_new(required_blob_size, GFP_NOFS);
if (!blob)
goto do_sync_unlocked;
spin_lock(&ci->i_ceph_lock);
+ /* prealloc_blob can't be released while holding i_ceph_lock */
if (ci->i_xattrs.prealloc_blob)
- ceph_buffer_put(ci->i_xattrs.prealloc_blob);
+ old_blob = ci->i_xattrs.prealloc_blob;
ci->i_xattrs.prealloc_blob = blob;
goto retry;
}
@@ -1131,6 +1139,7 @@ int __ceph_setxattr(struct inode *inode, const char *name,
}
spin_unlock(&ci->i_ceph_lock);
+ ceph_buffer_put(old_blob);
if (lock_snap_rwsem)
up_read(&mdsc->snap_rwsem);
if (dirty)
diff --git a/fs/cifs/cifsfs.h b/fs/cifs/cifsfs.h
index 4b21a90..99caf77 100644
--- a/fs/cifs/cifsfs.h
+++ b/fs/cifs/cifsfs.h
@@ -152,5 +152,5 @@ extern long cifs_ioctl(struct file *filep, unsigned int cmd, unsigned long arg);
extern const struct export_operations cifs_export_ops;
#endif /* CONFIG_CIFS_NFSD_EXPORT */
-#define CIFS_VERSION "2.21"
+#define CIFS_VERSION "2.22"
#endif /* _CIFSFS_H */
diff --git a/fs/cifs/cifsproto.h b/fs/cifs/cifsproto.h
index e232342..592a6ce 100644
--- a/fs/cifs/cifsproto.h
+++ b/fs/cifs/cifsproto.h
@@ -579,6 +579,7 @@ extern void rqst_page_get_length(struct smb_rqst *rqst, unsigned int page,
unsigned int *len, unsigned int *offset);
void extract_unc_hostname(const char *unc, const char **h, size_t *len);
+int copy_path_name(char *dst, const char *src);
#ifdef CONFIG_CIFS_DFS_UPCALL
static inline int get_dfs_path(const unsigned int xid, struct cifs_ses *ses,
diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c
index e2f9596..3907653 100644
--- a/fs/cifs/cifssmb.c
+++ b/fs/cifs/cifssmb.c
@@ -942,10 +942,8 @@ CIFSPOSIXDelFile(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB add path length overrun check */
- name_len = strnlen(fileName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, fileName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, fileName);
}
params = 6 + name_len;
@@ -1015,10 +1013,8 @@ CIFSSMBDelFile(const unsigned int xid, struct cifs_tcon *tcon, const char *name,
remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve check for buffer overruns BB */
- name_len = strnlen(name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->fileName, name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->fileName, name);
}
pSMB->SearchAttributes =
cpu_to_le16(ATTR_READONLY | ATTR_HIDDEN | ATTR_SYSTEM);
@@ -1062,10 +1058,8 @@ CIFSSMBRmDir(const unsigned int xid, struct cifs_tcon *tcon, const char *name,
remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve check for buffer overruns BB */
- name_len = strnlen(name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->DirName, name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->DirName, name);
}
pSMB->BufferFormat = 0x04;
@@ -1107,10 +1101,8 @@ CIFSSMBMkDir(const unsigned int xid, struct cifs_tcon *tcon, const char *name,
remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve check for buffer overruns BB */
- name_len = strnlen(name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->DirName, name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->DirName, name);
}
pSMB->BufferFormat = 0x04;
@@ -1157,10 +1149,8 @@ CIFSPOSIXCreate(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, name);
}
params = 6 + name_len;
@@ -1324,11 +1314,9 @@ SMBLegacyOpen(const unsigned int xid, struct cifs_tcon *tcon,
fileName, PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve check for buffer overruns BB */
+ } else {
count = 0; /* no pad */
- name_len = strnlen(fileName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->fileName, fileName, name_len);
+ name_len = copy_path_name(pSMB->fileName, fileName);
}
if (*pOplock & REQ_OPLOCK)
pSMB->OpenFlags = cpu_to_le16(REQ_OPLOCK);
@@ -1442,11 +1430,8 @@ CIFS_open(const unsigned int xid, struct cifs_open_parms *oparms, int *oplock,
/* BB improve check for buffer overruns BB */
/* no pad */
count = 0;
- name_len = strnlen(path, PATH_MAX);
- /* trailing null */
- name_len++;
+ name_len = copy_path_name(req->fileName, path);
req->NameLength = cpu_to_le16(name_len);
- strncpy(req->fileName, path, name_len);
}
if (*oplock & REQ_OPLOCK)
@@ -2812,15 +2797,10 @@ CIFSSMBRename(const unsigned int xid, struct cifs_tcon *tcon,
remap);
name_len2 += 1 /* trailing null */ + 1 /* Signature word */ ;
name_len2 *= 2; /* convert to bytes */
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(from_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->OldFileName, from_name, name_len);
- name_len2 = strnlen(to_name, PATH_MAX);
- name_len2++; /* trailing null */
+ } else {
+ name_len = copy_path_name(pSMB->OldFileName, from_name);
+ name_len2 = copy_path_name(pSMB->OldFileName+name_len+1, to_name);
pSMB->OldFileName[name_len] = 0x04; /* 2nd buffer format */
- strncpy(&pSMB->OldFileName[name_len + 1], to_name, name_len2);
- name_len2++; /* trailing null */
name_len2++; /* signature byte */
}
@@ -2962,15 +2942,10 @@ CIFSSMBCopy(const unsigned int xid, struct cifs_tcon *tcon,
toName, PATH_MAX, nls_codepage, remap);
name_len2 += 1 /* trailing null */ + 1 /* Signature word */ ;
name_len2 *= 2; /* convert to bytes */
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(fromName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->OldFileName, fromName, name_len);
- name_len2 = strnlen(toName, PATH_MAX);
- name_len2++; /* trailing null */
+ } else {
+ name_len = copy_path_name(pSMB->OldFileName, fromName);
pSMB->OldFileName[name_len] = 0x04; /* 2nd buffer format */
- strncpy(&pSMB->OldFileName[name_len + 1], toName, name_len2);
- name_len2++; /* trailing null */
+ name_len2 = copy_path_name(pSMB->OldFileName+name_len+1, toName);
name_len2++; /* signature byte */
}
@@ -3021,10 +2996,8 @@ CIFSUnixCreateSymLink(const unsigned int xid, struct cifs_tcon *tcon,
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(fromName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, fromName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, fromName);
}
params = 6 + name_len;
pSMB->MaxSetupCount = 0;
@@ -3044,10 +3017,8 @@ CIFSUnixCreateSymLink(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len_target++; /* trailing null */
name_len_target *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len_target = strnlen(toName, PATH_MAX);
- name_len_target++; /* trailing null */
- strncpy(data_offset, toName, name_len_target);
+ } else {
+ name_len_target = copy_path_name(data_offset, toName);
}
pSMB->MaxParameterCount = cpu_to_le16(2);
@@ -3109,10 +3080,8 @@ CIFSUnixCreateHardLink(const unsigned int xid, struct cifs_tcon *tcon,
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(toName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, toName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, toName);
}
params = 6 + name_len;
pSMB->MaxSetupCount = 0;
@@ -3131,10 +3100,8 @@ CIFSUnixCreateHardLink(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len_target++; /* trailing null */
name_len_target *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len_target = strnlen(fromName, PATH_MAX);
- name_len_target++; /* trailing null */
- strncpy(data_offset, fromName, name_len_target);
+ } else {
+ name_len_target = copy_path_name(data_offset, fromName);
}
pSMB->MaxParameterCount = cpu_to_le16(2);
@@ -3213,15 +3180,10 @@ CIFSCreateHardLink(const unsigned int xid, struct cifs_tcon *tcon,
remap);
name_len2 += 1 /* trailing null */ + 1 /* Signature word */ ;
name_len2 *= 2; /* convert to bytes */
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(from_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->OldFileName, from_name, name_len);
- name_len2 = strnlen(to_name, PATH_MAX);
- name_len2++; /* trailing null */
+ } else {
+ name_len = copy_path_name(pSMB->OldFileName, from_name);
pSMB->OldFileName[name_len] = 0x04; /* 2nd buffer format */
- strncpy(&pSMB->OldFileName[name_len + 1], to_name, name_len2);
- name_len2++; /* trailing null */
+ name_len2 = copy_path_name(pSMB->OldFileName+name_len+1, to_name);
name_len2++; /* signature byte */
}
@@ -3271,10 +3233,8 @@ CIFSSMBUnixQuerySymLink(const unsigned int xid, struct cifs_tcon *tcon,
remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(searchName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, searchName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, searchName);
}
params = 2 /* level */ + 4 /* rsrvd */ + name_len /* incl null */ ;
@@ -3691,10 +3651,8 @@ CIFSSMBGetPosixACL(const unsigned int xid, struct cifs_tcon *tcon,
name_len *= 2;
pSMB->FileName[name_len] = 0;
pSMB->FileName[name_len+1] = 0;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(searchName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, searchName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, searchName);
}
params = 2 /* level */ + 4 /* rsrvd */ + name_len /* incl null */ ;
@@ -3776,10 +3734,8 @@ CIFSSMBSetPosixACL(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(fileName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, fileName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, fileName);
}
params = 6 + name_len;
pSMB->MaxParameterCount = cpu_to_le16(2);
@@ -4184,9 +4140,7 @@ SMBQueryInformation(const unsigned int xid, struct cifs_tcon *tcon,
name_len++; /* trailing null */
name_len *= 2;
} else {
- name_len = strnlen(search_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, search_name, name_len);
+ name_len = copy_path_name(pSMB->FileName, search_name);
}
pSMB->BufferFormat = 0x04;
name_len++; /* account for buffer type byte */
@@ -4321,10 +4275,8 @@ CIFSSMBQPathInfo(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(search_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, search_name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, search_name);
}
params = 2 /* level */ + 4 /* reserved */ + name_len /* includes NUL */;
@@ -4490,10 +4442,8 @@ CIFSSMBUnixQPathInfo(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(searchName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, searchName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, searchName);
}
params = 2 /* level */ + 4 /* reserved */ + name_len /* includes NUL */;
@@ -4593,17 +4543,16 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon,
pSMB->FileName[name_len+1] = 0;
name_len += 2;
}
- } else { /* BB add check for overrun of SMB buf BB */
- name_len = strnlen(searchName, PATH_MAX);
-/* BB fix here and in unicode clause above ie
- if (name_len > buffersize-header)
- free buffer exit; BB */
- strncpy(pSMB->FileName, searchName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, searchName);
if (msearch) {
- pSMB->FileName[name_len] = CIFS_DIR_SEP(cifs_sb);
- pSMB->FileName[name_len+1] = '*';
- pSMB->FileName[name_len+2] = 0;
- name_len += 3;
+ if (WARN_ON_ONCE(name_len > PATH_MAX-2))
+ name_len = PATH_MAX-2;
+ /* overwrite nul byte */
+ pSMB->FileName[name_len-1] = CIFS_DIR_SEP(cifs_sb);
+ pSMB->FileName[name_len] = '*';
+ pSMB->FileName[name_len+1] = 0;
+ name_len += 2;
}
}
@@ -4898,10 +4847,8 @@ CIFSGetSrvInodeNumber(const unsigned int xid, struct cifs_tcon *tcon,
remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(search_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, search_name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, search_name);
}
params = 2 /* level */ + 4 /* rsrvd */ + name_len /* incl null */ ;
@@ -5008,9 +4955,7 @@ CIFSGetDFSRefer(const unsigned int xid, struct cifs_ses *ses,
name_len++; /* trailing null */
name_len *= 2;
} else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(search_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->RequestFileName, search_name, name_len);
+ name_len = copy_path_name(pSMB->RequestFileName, search_name);
}
if (ses->server->sign)
@@ -5663,10 +5608,8 @@ CIFSSMBSetEOF(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, cifs_sb->local_nls, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(file_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, file_name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, file_name);
}
params = 6 + name_len;
data_count = sizeof(struct file_end_of_file_info);
@@ -5959,10 +5902,8 @@ CIFSSMBSetPathInfo(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(fileName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, fileName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, fileName);
}
params = 6 + name_len;
@@ -6040,10 +5981,8 @@ CIFSSMBSetAttrLegacy(unsigned int xid, struct cifs_tcon *tcon, char *fileName,
PATH_MAX, nls_codepage);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(fileName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->fileName, fileName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->fileName, fileName);
}
pSMB->attr = cpu_to_le16(dos_attrs);
pSMB->BufferFormat = 0x04;
@@ -6203,10 +6142,8 @@ CIFSSMBUnixSetPathInfo(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(file_name, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, file_name, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, file_name);
}
params = 6 + name_len;
@@ -6298,10 +6235,8 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
list_len++; /* trailing null */
list_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- list_len = strnlen(searchName, PATH_MAX);
- list_len++; /* trailing null */
- strncpy(pSMB->FileName, searchName, list_len);
+ } else {
+ list_len = copy_path_name(pSMB->FileName, searchName);
}
params = 2 /* level */ + 4 /* reserved */ + list_len /* includes NUL */;
@@ -6480,10 +6415,8 @@ CIFSSMBSetEA(const unsigned int xid, struct cifs_tcon *tcon,
PATH_MAX, nls_codepage, remap);
name_len++; /* trailing null */
name_len *= 2;
- } else { /* BB improve the check for buffer overruns BB */
- name_len = strnlen(fileName, PATH_MAX);
- name_len++; /* trailing null */
- strncpy(pSMB->FileName, fileName, name_len);
+ } else {
+ name_len = copy_path_name(pSMB->FileName, fileName);
}
params = 6 + name_len;
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index a15a6e7..5299eff 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -1113,7 +1113,7 @@ cifs_demultiplex_thread(void *p)
mempool_resize(cifs_req_poolp, length + cifs_min_rcv);
set_freezable();
- allow_signal(SIGKILL);
+ allow_kernel_signal(SIGKILL);
while (server->tcpStatus != CifsExiting) {
if (try_to_freeze())
continue;
@@ -2981,6 +2981,7 @@ static int
cifs_set_cifscreds(struct smb_vol *vol, struct cifs_ses *ses)
{
int rc = 0;
+ int is_domain = 0;
const char *delim, *payload;
char *desc;
ssize_t len;
@@ -3028,6 +3029,7 @@ cifs_set_cifscreds(struct smb_vol *vol, struct cifs_ses *ses)
rc = PTR_ERR(key);
goto out_err;
}
+ is_domain = 1;
}
down_read(&key->sem);
@@ -3085,6 +3087,26 @@ cifs_set_cifscreds(struct smb_vol *vol, struct cifs_ses *ses)
goto out_key_put;
}
+ /*
+ * If we have a domain key then we must set the domainName in the
+ * for the request.
+ */
+ if (is_domain && ses->domainName) {
+ vol->domainname = kstrndup(ses->domainName,
+ strlen(ses->domainName),
+ GFP_KERNEL);
+ if (!vol->domainname) {
+ cifs_dbg(FYI, "Unable to allocate %zd bytes for "
+ "domain\n", len);
+ rc = -ENOMEM;
+ kfree(vol->username);
+ vol->username = NULL;
+ kzfree(vol->password);
+ vol->password = NULL;
+ goto out_key_put;
+ }
+ }
+
out_key_put:
up_read(&key->sem);
key_put(key);
@@ -4209,16 +4231,19 @@ build_unc_path_to_root(const struct smb_vol *vol,
strlen(vol->prepath) + 1 : 0;
unsigned int unc_len = strnlen(vol->UNC, MAX_TREE_SIZE + 1);
+ if (unc_len > MAX_TREE_SIZE)
+ return ERR_PTR(-EINVAL);
+
full_path = kmalloc(unc_len + pplen + 1, GFP_KERNEL);
if (full_path == NULL)
return ERR_PTR(-ENOMEM);
- strncpy(full_path, vol->UNC, unc_len);
+ memcpy(full_path, vol->UNC, unc_len);
pos = full_path + unc_len;
if (pplen) {
*pos = CIFS_DIR_SEP(cifs_sb);
- strncpy(pos + 1, vol->prepath, pplen);
+ memcpy(pos + 1, vol->prepath, pplen);
pos += pplen;
}
diff --git a/fs/cifs/dir.c b/fs/cifs/dir.c
index f26a48d..be424e8 100644
--- a/fs/cifs/dir.c
+++ b/fs/cifs/dir.c
@@ -69,11 +69,10 @@ cifs_build_path_to_root(struct smb_vol *vol, struct cifs_sb_info *cifs_sb,
return full_path;
if (dfsplen)
- strncpy(full_path, tcon->treeName, dfsplen);
+ memcpy(full_path, tcon->treeName, dfsplen);
full_path[dfsplen] = CIFS_DIR_SEP(cifs_sb);
- strncpy(full_path + dfsplen + 1, vol->prepath, pplen);
+ memcpy(full_path + dfsplen + 1, vol->prepath, pplen);
convert_delimiter(full_path, CIFS_DIR_SEP(cifs_sb));
- full_path[dfsplen + pplen] = 0; /* add trailing null */
return full_path;
}
diff --git a/fs/cifs/misc.c b/fs/cifs/misc.c
index f383877..5ad83bd 100644
--- a/fs/cifs/misc.c
+++ b/fs/cifs/misc.c
@@ -1011,3 +1011,25 @@ void extract_unc_hostname(const char *unc, const char **h, size_t *len)
*h = unc;
*len = end - unc;
}
+
+/**
+ * copy_path_name - copy src path to dst, possibly truncating
+ *
+ * returns number of bytes written (including trailing nul)
+ */
+int copy_path_name(char *dst, const char *src)
+{
+ int name_len;
+
+ /*
+ * PATH_MAX includes nul, so if strlen(src) >= PATH_MAX it
+ * will truncate and strlen(dst) will be PATH_MAX-1
+ */
+ name_len = strscpy(dst, src, PATH_MAX);
+ if (WARN_ON_ONCE(name_len < 0))
+ name_len = PATH_MAX-1;
+
+ /* we count the trailing nul */
+ name_len++;
+ return name_len;
+}
diff --git a/fs/cifs/sess.c b/fs/cifs/sess.c
index dcd49ad..4c764ff 100644
--- a/fs/cifs/sess.c
+++ b/fs/cifs/sess.c
@@ -159,13 +159,16 @@ static void ascii_ssetup_strings(char **pbcc_area, struct cifs_ses *ses,
const struct nls_table *nls_cp)
{
char *bcc_ptr = *pbcc_area;
+ int len;
/* copy user */
/* BB what about null user mounts - check that we do this BB */
/* copy user */
if (ses->user_name != NULL) {
- strncpy(bcc_ptr, ses->user_name, CIFS_MAX_USERNAME_LEN);
- bcc_ptr += strnlen(ses->user_name, CIFS_MAX_USERNAME_LEN);
+ len = strscpy(bcc_ptr, ses->user_name, CIFS_MAX_USERNAME_LEN);
+ if (WARN_ON_ONCE(len < 0))
+ len = CIFS_MAX_USERNAME_LEN - 1;
+ bcc_ptr += len;
}
/* else null user mount */
*bcc_ptr = 0;
@@ -173,8 +176,10 @@ static void ascii_ssetup_strings(char **pbcc_area, struct cifs_ses *ses,
/* copy domain */
if (ses->domainName != NULL) {
- strncpy(bcc_ptr, ses->domainName, CIFS_MAX_DOMAINNAME_LEN);
- bcc_ptr += strnlen(ses->domainName, CIFS_MAX_DOMAINNAME_LEN);
+ len = strscpy(bcc_ptr, ses->domainName, CIFS_MAX_DOMAINNAME_LEN);
+ if (WARN_ON_ONCE(len < 0))
+ len = CIFS_MAX_DOMAINNAME_LEN - 1;
+ bcc_ptr += len;
} /* else we will send a null domain name
so the server will default to its own domain */
*bcc_ptr = 0;
@@ -242,9 +247,10 @@ static void decode_ascii_ssetup(char **pbcc_area, __u16 bleft,
kfree(ses->serverOS);
- ses->serverOS = kzalloc(len + 1, GFP_KERNEL);
+ ses->serverOS = kmalloc(len + 1, GFP_KERNEL);
if (ses->serverOS) {
- strncpy(ses->serverOS, bcc_ptr, len);
+ memcpy(ses->serverOS, bcc_ptr, len);
+ ses->serverOS[len] = 0;
if (strncmp(ses->serverOS, "OS/2", 4) == 0)
cifs_dbg(FYI, "OS/2 server\n");
}
@@ -258,9 +264,11 @@ static void decode_ascii_ssetup(char **pbcc_area, __u16 bleft,
kfree(ses->serverNOS);
- ses->serverNOS = kzalloc(len + 1, GFP_KERNEL);
- if (ses->serverNOS)
- strncpy(ses->serverNOS, bcc_ptr, len);
+ ses->serverNOS = kmalloc(len + 1, GFP_KERNEL);
+ if (ses->serverNOS) {
+ memcpy(ses->serverNOS, bcc_ptr, len);
+ ses->serverNOS[len] = 0;
+ }
bcc_ptr += len + 1;
bleft -= len + 1;
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 24bbe3c..cfb48bd 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -679,6 +679,13 @@ static void io_put_req(struct io_kiocb *req)
io_free_req(req);
}
+static unsigned io_cqring_events(struct io_cq_ring *ring)
+{
+ /* See comment at the top of this file */
+ smp_rmb();
+ return READ_ONCE(ring->r.tail) - READ_ONCE(ring->r.head);
+}
+
/*
* Find and free completed poll iocbs
*/
@@ -771,7 +778,7 @@ static int io_do_iopoll(struct io_ring_ctx *ctx, unsigned int *nr_events,
static int io_iopoll_getevents(struct io_ring_ctx *ctx, unsigned int *nr_events,
long min)
{
- while (!list_empty(&ctx->poll_list)) {
+ while (!list_empty(&ctx->poll_list) && !need_resched()) {
int ret;
ret = io_do_iopoll(ctx, nr_events, min);
@@ -798,6 +805,12 @@ static void io_iopoll_reap_events(struct io_ring_ctx *ctx)
unsigned int nr_events = 0;
io_iopoll_getevents(ctx, &nr_events, 1);
+
+ /*
+ * Ensure we allow local-to-the-cpu processing to take place,
+ * in this case we need to ensure that we reap all events.
+ */
+ cond_resched();
}
mutex_unlock(&ctx->uring_lock);
}
@@ -805,11 +818,42 @@ static void io_iopoll_reap_events(struct io_ring_ctx *ctx)
static int io_iopoll_check(struct io_ring_ctx *ctx, unsigned *nr_events,
long min)
{
- int ret = 0;
+ int iters, ret = 0;
+ /*
+ * We disallow the app entering submit/complete with polling, but we
+ * still need to lock the ring to prevent racing with polled issue
+ * that got punted to a workqueue.
+ */
+ mutex_lock(&ctx->uring_lock);
+
+ iters = 0;
do {
int tmin = 0;
+ /*
+ * Don't enter poll loop if we already have events pending.
+ * If we do, we can potentially be spinning for commands that
+ * already triggered a CQE (eg in error).
+ */
+ if (io_cqring_events(ctx->cq_ring))
+ break;
+
+ /*
+ * If a submit got punted to a workqueue, we can have the
+ * application entering polling for a command before it gets
+ * issued. That app will hold the uring_lock for the duration
+ * of the poll right here, so we need to take a breather every
+ * now and then to ensure that the issue has a chance to add
+ * the poll to the issued list. Otherwise we can spin here
+ * forever, while the workqueue is stuck trying to acquire the
+ * very same mutex.
+ */
+ if (!(++iters & 7)) {
+ mutex_unlock(&ctx->uring_lock);
+ mutex_lock(&ctx->uring_lock);
+ }
+
if (*nr_events < min)
tmin = min - *nr_events;
@@ -819,6 +863,7 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, unsigned *nr_events,
ret = 0;
} while (min && !*nr_events && !need_resched());
+ mutex_unlock(&ctx->uring_lock);
return ret;
}
@@ -2280,15 +2325,7 @@ static int io_sq_thread(void *data)
unsigned nr_events = 0;
if (ctx->flags & IORING_SETUP_IOPOLL) {
- /*
- * We disallow the app entering submit/complete
- * with polling, but we still need to lock the
- * ring to prevent racing with polled issue
- * that got punted to a workqueue.
- */
- mutex_lock(&ctx->uring_lock);
io_iopoll_check(ctx, &nr_events, 0);
- mutex_unlock(&ctx->uring_lock);
} else {
/*
* Normal IO, just pretend everything completed.
@@ -2433,13 +2470,6 @@ static int io_ring_submit(struct io_ring_ctx *ctx, unsigned int to_submit)
return submit;
}
-static unsigned io_cqring_events(struct io_cq_ring *ring)
-{
- /* See comment at the top of this file */
- smp_rmb();
- return READ_ONCE(ring->r.tail) - READ_ONCE(ring->r.head);
-}
-
/*
* Wait until events become available, if we don't already have some. The
* application must reap them itself, as they reside on the shared cq ring.
@@ -3190,9 +3220,7 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit,
min_complete = min(min_complete, ctx->cq_entries);
if (ctx->flags & IORING_SETUP_IOPOLL) {
- mutex_lock(&ctx->uring_lock);
ret = io_iopoll_check(ctx, &nr_events, min_complete);
- mutex_unlock(&ctx->uring_lock);
} else {
ret = io_cqring_wait(ctx, min_complete, sig, sigsz);
}
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index 8d50109..0adfd884 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -1487,7 +1487,7 @@ static int nfs_finish_open(struct nfs_open_context *ctx,
if (S_ISREG(file->f_path.dentry->d_inode->i_mode))
nfs_file_set_open_context(file, ctx);
else
- err = -ESTALE;
+ err = -EOPENSTALE;
out:
return err;
}
diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
index 0cb4424..222d711 100644
--- a/fs/nfs/direct.c
+++ b/fs/nfs/direct.c
@@ -401,15 +401,21 @@ static void nfs_direct_read_completion(struct nfs_pgio_header *hdr)
unsigned long bytes = 0;
struct nfs_direct_req *dreq = hdr->dreq;
- if (test_bit(NFS_IOHDR_REDO, &hdr->flags))
- goto out_put;
-
spin_lock(&dreq->lock);
- if (test_bit(NFS_IOHDR_ERROR, &hdr->flags) && (hdr->good_bytes == 0))
+ if (test_bit(NFS_IOHDR_ERROR, &hdr->flags))
dreq->error = hdr->error;
- else
+
+ if (test_bit(NFS_IOHDR_REDO, &hdr->flags)) {
+ spin_unlock(&dreq->lock);
+ goto out_put;
+ }
+
+ if (hdr->good_bytes != 0)
nfs_direct_good_bytes(dreq, hdr);
+ if (test_bit(NFS_IOHDR_EOF, &hdr->flags))
+ dreq->error = 0;
+
spin_unlock(&dreq->lock);
while (!list_empty(&hdr->pages)) {
@@ -782,16 +788,19 @@ static void nfs_direct_write_completion(struct nfs_pgio_header *hdr)
bool request_commit = false;
struct nfs_page *req = nfs_list_entry(hdr->pages.next);
- if (test_bit(NFS_IOHDR_REDO, &hdr->flags))
- goto out_put;
-
nfs_init_cinfo_from_dreq(&cinfo, dreq);
spin_lock(&dreq->lock);
if (test_bit(NFS_IOHDR_ERROR, &hdr->flags))
dreq->error = hdr->error;
- if (dreq->error == 0) {
+
+ if (test_bit(NFS_IOHDR_REDO, &hdr->flags)) {
+ spin_unlock(&dreq->lock);
+ goto out_put;
+ }
+
+ if (hdr->good_bytes != 0) {
nfs_direct_good_bytes(dreq, hdr);
if (nfs_write_need_commit(hdr)) {
if (dreq->flags == NFS_ODIRECT_RESCHED_WRITES)
diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index b04e20d..5657b7f 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -8,6 +8,7 @@
*/
#include <linux/nfs_fs.h>
+#include <linux/nfs_mount.h>
#include <linux/nfs_page.h>
#include <linux/module.h>
#include <linux/sched/mm.h>
@@ -928,7 +929,9 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio,
pgm = &pgio->pg_mirrors[0];
pgm->pg_bsize = mirror->mirror_ds->ds_versions[0].rsize;
- pgio->pg_maxretrans = io_maxretrans;
+ if (NFS_SERVER(pgio->pg_inode)->flags &
+ (NFS_MOUNT_SOFT|NFS_MOUNT_SOFTERR))
+ pgio->pg_maxretrans = io_maxretrans;
return;
out_nolseg:
if (pgio->pg_error < 0)
@@ -940,6 +943,7 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio,
pgio->pg_lseg);
pnfs_put_lseg(pgio->pg_lseg);
pgio->pg_lseg = NULL;
+ pgio->pg_maxretrans = 0;
nfs_pageio_reset_read_mds(pgio);
}
@@ -1000,7 +1004,9 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio,
pgm->pg_bsize = mirror->mirror_ds->ds_versions[0].wsize;
}
- pgio->pg_maxretrans = io_maxretrans;
+ if (NFS_SERVER(pgio->pg_inode)->flags &
+ (NFS_MOUNT_SOFT|NFS_MOUNT_SOFTERR))
+ pgio->pg_maxretrans = io_maxretrans;
return;
out_mds:
@@ -1010,6 +1016,7 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio,
pgio->pg_lseg);
pnfs_put_lseg(pgio->pg_lseg);
pgio->pg_lseg = NULL;
+ pgio->pg_maxretrans = 0;
nfs_pageio_reset_write_mds(pgio);
}
@@ -1148,8 +1155,6 @@ static int ff_layout_async_handle_error_v4(struct rpc_task *task,
break;
case -NFS4ERR_RETRY_UNCACHED_REP:
break;
- case -EAGAIN:
- return -NFS4ERR_RESET_TO_PNFS;
/* Invalidate Layout errors */
case -NFS4ERR_PNFS_NO_LAYOUT:
case -ESTALE: /* mapped NFS4ERR_STALE */
@@ -1210,7 +1215,6 @@ static int ff_layout_async_handle_error_v3(struct rpc_task *task,
case -EBADHANDLE:
case -ELOOP:
case -ENOSPC:
- case -EAGAIN:
break;
case -EJUKEBOX:
nfs_inc_stats(lseg->pls_layout->plh_inode, NFSIOS_DELAY);
@@ -1445,16 +1449,6 @@ static void ff_layout_read_prepare_v4(struct rpc_task *task, void *data)
ff_layout_read_prepare_common(task, hdr);
}
-static void
-ff_layout_io_prepare_transmit(struct rpc_task *task,
- void *data)
-{
- struct nfs_pgio_header *hdr = data;
-
- if (!pnfs_is_valid_lseg(hdr->lseg))
- rpc_exit(task, -EAGAIN);
-}
-
static void ff_layout_read_call_done(struct rpc_task *task, void *data)
{
struct nfs_pgio_header *hdr = data;
@@ -1740,7 +1734,6 @@ static void ff_layout_commit_release(void *data)
static const struct rpc_call_ops ff_layout_read_call_ops_v3 = {
.rpc_call_prepare = ff_layout_read_prepare_v3,
- .rpc_call_prepare_transmit = ff_layout_io_prepare_transmit,
.rpc_call_done = ff_layout_read_call_done,
.rpc_count_stats = ff_layout_read_count_stats,
.rpc_release = ff_layout_read_release,
@@ -1748,7 +1741,6 @@ static const struct rpc_call_ops ff_layout_read_call_ops_v3 = {
static const struct rpc_call_ops ff_layout_read_call_ops_v4 = {
.rpc_call_prepare = ff_layout_read_prepare_v4,
- .rpc_call_prepare_transmit = ff_layout_io_prepare_transmit,
.rpc_call_done = ff_layout_read_call_done,
.rpc_count_stats = ff_layout_read_count_stats,
.rpc_release = ff_layout_read_release,
@@ -1756,7 +1748,6 @@ static const struct rpc_call_ops ff_layout_read_call_ops_v4 = {
static const struct rpc_call_ops ff_layout_write_call_ops_v3 = {
.rpc_call_prepare = ff_layout_write_prepare_v3,
- .rpc_call_prepare_transmit = ff_layout_io_prepare_transmit,
.rpc_call_done = ff_layout_write_call_done,
.rpc_count_stats = ff_layout_write_count_stats,
.rpc_release = ff_layout_write_release,
@@ -1764,7 +1755,6 @@ static const struct rpc_call_ops ff_layout_write_call_ops_v3 = {
static const struct rpc_call_ops ff_layout_write_call_ops_v4 = {
.rpc_call_prepare = ff_layout_write_prepare_v4,
- .rpc_call_prepare_transmit = ff_layout_io_prepare_transmit,
.rpc_call_done = ff_layout_write_call_done,
.rpc_count_stats = ff_layout_write_count_stats,
.rpc_release = ff_layout_write_release,
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 8a17582..c764cfe 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1403,12 +1403,21 @@ static int nfs_check_inode_attributes(struct inode *inode, struct nfs_fattr *fat
if (NFS_PROTO(inode)->have_delegation(inode, FMODE_READ))
return 0;
+ /* No fileid? Just exit */
+ if (!(fattr->valid & NFS_ATTR_FATTR_FILEID))
+ return 0;
/* Has the inode gone and changed behind our back? */
- if ((fattr->valid & NFS_ATTR_FATTR_FILEID) && nfsi->fileid != fattr->fileid)
+ if (nfsi->fileid != fattr->fileid) {
+ /* Is this perhaps the mounted-on fileid? */
+ if ((fattr->valid & NFS_ATTR_FATTR_MOUNTED_ON_FILEID) &&
+ nfsi->fileid == fattr->mounted_on_fileid)
+ return 0;
return -ESTALE;
+ }
if ((fattr->valid & NFS_ATTR_FATTR_TYPE) && (inode->i_mode & S_IFMT) != (fattr->mode & S_IFMT))
return -ESTALE;
+
if (!nfs_file_has_buffered_writers(nfsi)) {
/* Verify a few of the more important attributes */
if ((fattr->valid & NFS_ATTR_FATTR_CHANGE) != 0 && !inode_eq_iversion_raw(inode, fattr->change_attr))
@@ -1768,18 +1777,6 @@ int nfs_post_op_update_inode_force_wcc(struct inode *inode, struct nfs_fattr *fa
EXPORT_SYMBOL_GPL(nfs_post_op_update_inode_force_wcc);
-static inline bool nfs_fileid_valid(struct nfs_inode *nfsi,
- struct nfs_fattr *fattr)
-{
- bool ret1 = true, ret2 = true;
-
- if (fattr->valid & NFS_ATTR_FATTR_FILEID)
- ret1 = (nfsi->fileid == fattr->fileid);
- if (fattr->valid & NFS_ATTR_FATTR_MOUNTED_ON_FILEID)
- ret2 = (nfsi->fileid == fattr->mounted_on_fileid);
- return ret1 || ret2;
-}
-
/*
* Many nfs protocol calls return the new file attributes after
* an operation. Here we update the inode to reflect the state
@@ -1810,7 +1807,15 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr)
nfs_display_fhandle_hash(NFS_FH(inode)),
atomic_read(&inode->i_count), fattr->valid);
- if (!nfs_fileid_valid(nfsi, fattr)) {
+ /* No fileid? Just exit */
+ if (!(fattr->valid & NFS_ATTR_FATTR_FILEID))
+ return 0;
+ /* Has the inode gone and changed behind our back? */
+ if (nfsi->fileid != fattr->fileid) {
+ /* Is this perhaps the mounted-on fileid? */
+ if ((fattr->valid & NFS_ATTR_FATTR_MOUNTED_ON_FILEID) &&
+ nfsi->fileid == fattr->mounted_on_fileid)
+ return 0;
printk(KERN_ERR "NFS: server %s error: fileid changed\n"
"fsid %s: expected fileid 0x%Lx, got 0x%Lx\n",
NFS_SERVER(inode)->nfs_client->cl_hostname,
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index a2346a2..e64f810 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -775,3 +775,13 @@ static inline bool nfs_error_is_fatal(int err)
}
}
+static inline bool nfs_error_is_fatal_on_server(int err)
+{
+ switch (err) {
+ case 0:
+ case -ERESTARTSYS:
+ case -EINTR:
+ return false;
+ }
+ return nfs_error_is_fatal(err);
+}
diff --git a/fs/nfs/nfs4file.c b/fs/nfs/nfs4file.c
index 96db471..339663d 100644
--- a/fs/nfs/nfs4file.c
+++ b/fs/nfs/nfs4file.c
@@ -73,13 +73,13 @@ nfs4_file_open(struct inode *inode, struct file *filp)
if (IS_ERR(inode)) {
err = PTR_ERR(inode);
switch (err) {
- case -EPERM:
- case -EACCES:
- case -EDQUOT:
- case -ENOSPC:
- case -EROFS:
- goto out_put_ctx;
default:
+ goto out_put_ctx;
+ case -ENOENT:
+ case -ESTALE:
+ case -EISDIR:
+ case -ENOTDIR:
+ case -ELOOP:
goto out_drop;
}
}
diff --git a/fs/nfs/pagelist.c b/fs/nfs/pagelist.c
index ed4e1b0..20b3717 100644
--- a/fs/nfs/pagelist.c
+++ b/fs/nfs/pagelist.c
@@ -590,7 +590,7 @@ static void nfs_pgio_rpcsetup(struct nfs_pgio_header *hdr,
}
hdr->res.fattr = &hdr->fattr;
- hdr->res.count = count;
+ hdr->res.count = 0;
hdr->res.eof = 0;
hdr->res.verf = &hdr->verf;
nfs_fattr_init(&hdr->fattr);
@@ -1251,20 +1251,23 @@ static void nfs_pageio_complete_mirror(struct nfs_pageio_descriptor *desc,
int nfs_pageio_resend(struct nfs_pageio_descriptor *desc,
struct nfs_pgio_header *hdr)
{
- LIST_HEAD(failed);
+ LIST_HEAD(pages);
desc->pg_io_completion = hdr->io_completion;
desc->pg_dreq = hdr->dreq;
- while (!list_empty(&hdr->pages)) {
- struct nfs_page *req = nfs_list_entry(hdr->pages.next);
+ list_splice_init(&hdr->pages, &pages);
+ while (!list_empty(&pages)) {
+ struct nfs_page *req = nfs_list_entry(pages.next);
if (!nfs_pageio_add_request(desc, req))
- nfs_list_move_request(req, &failed);
+ break;
}
nfs_pageio_complete(desc);
- if (!list_empty(&failed)) {
- list_move(&failed, &hdr->pages);
- return desc->pg_error < 0 ? desc->pg_error : -EIO;
+ if (!list_empty(&pages)) {
+ int err = desc->pg_error < 0 ? desc->pg_error : -EIO;
+ hdr->completion_ops->error_cleanup(&pages, err);
+ nfs_set_pgio_error(hdr, err, hdr->io_start);
+ return err;
}
return 0;
}
diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c
index c0046c3..82af480 100644
--- a/fs/nfs/pnfs_nfs.c
+++ b/fs/nfs/pnfs_nfs.c
@@ -627,11 +627,16 @@ static int _nfs4_pnfs_v3_ds_connect(struct nfs_server *mds_srv,
/* Add this address as an alias */
rpc_clnt_add_xprt(clp->cl_rpcclient, &xprt_args,
rpc_clnt_test_and_add_xprt, NULL);
- } else
- clp = get_v3_ds_connect(mds_srv,
- (struct sockaddr *)&da->da_addr,
- da->da_addrlen, IPPROTO_TCP,
- timeo, retrans);
+ continue;
+ }
+ clp = get_v3_ds_connect(mds_srv,
+ (struct sockaddr *)&da->da_addr,
+ da->da_addrlen, IPPROTO_TCP,
+ timeo, retrans);
+ if (IS_ERR(clp))
+ continue;
+ clp->cl_rpcclient->cl_softerr = 0;
+ clp->cl_rpcclient->cl_softrtry = 0;
}
if (IS_ERR(clp)) {
diff --git a/fs/nfs/proc.c b/fs/nfs/proc.c
index 5552fa8..0f7288b 100644
--- a/fs/nfs/proc.c
+++ b/fs/nfs/proc.c
@@ -594,7 +594,8 @@ static int nfs_read_done(struct rpc_task *task, struct nfs_pgio_header *hdr)
/* Emulate the eof flag, which isn't normally needed in NFSv2
* as it is guaranteed to always return the file attributes
*/
- if (hdr->args.offset + hdr->res.count >= hdr->res.fattr->size)
+ if ((hdr->res.count == 0 && hdr->args.count > 0) ||
+ hdr->args.offset + hdr->res.count >= hdr->res.fattr->size)
hdr->res.eof = 1;
}
return 0;
@@ -615,8 +616,10 @@ static int nfs_proc_pgio_rpc_prepare(struct rpc_task *task,
static int nfs_write_done(struct rpc_task *task, struct nfs_pgio_header *hdr)
{
- if (task->tk_status >= 0)
+ if (task->tk_status >= 0) {
+ hdr->res.count = hdr->args.count;
nfs_writeback_update_inode(hdr);
+ }
return 0;
}
diff --git a/fs/nfs/read.c b/fs/nfs/read.c
index c19841c..cfe0b58 100644
--- a/fs/nfs/read.c
+++ b/fs/nfs/read.c
@@ -91,19 +91,25 @@ void nfs_pageio_reset_read_mds(struct nfs_pageio_descriptor *pgio)
}
EXPORT_SYMBOL_GPL(nfs_pageio_reset_read_mds);
-static void nfs_readpage_release(struct nfs_page *req)
+static void nfs_readpage_release(struct nfs_page *req, int error)
{
struct inode *inode = d_inode(nfs_req_openctx(req)->dentry);
+ struct page *page = req->wb_page;
dprintk("NFS: read done (%s/%llu %d@%lld)\n", inode->i_sb->s_id,
(unsigned long long)NFS_FILEID(inode), req->wb_bytes,
(long long)req_offset(req));
+ if (nfs_error_is_fatal_on_server(error) && error != -ETIMEDOUT)
+ SetPageError(page);
if (nfs_page_group_sync_on_bit(req, PG_UNLOCKPAGE)) {
- if (PageUptodate(req->wb_page))
- nfs_readpage_to_fscache(inode, req->wb_page, 0);
+ struct address_space *mapping = page_file_mapping(page);
- unlock_page(req->wb_page);
+ if (PageUptodate(page))
+ nfs_readpage_to_fscache(inode, page, 0);
+ else if (!PageError(page) && !PagePrivate(page))
+ generic_error_remove_page(mapping, page);
+ unlock_page(page);
}
nfs_release_request(req);
}
@@ -131,7 +137,7 @@ int nfs_readpage_async(struct nfs_open_context *ctx, struct inode *inode,
&nfs_async_read_completion_ops);
if (!nfs_pageio_add_request(&pgio, new)) {
nfs_list_remove_request(new);
- nfs_readpage_release(new);
+ nfs_readpage_release(new, pgio.pg_error);
}
nfs_pageio_complete(&pgio);
@@ -153,6 +159,7 @@ static void nfs_page_group_set_uptodate(struct nfs_page *req)
static void nfs_read_completion(struct nfs_pgio_header *hdr)
{
unsigned long bytes = 0;
+ int error;
if (test_bit(NFS_IOHDR_REDO, &hdr->flags))
goto out;
@@ -179,14 +186,19 @@ static void nfs_read_completion(struct nfs_pgio_header *hdr)
zero_user_segment(page, start, end);
}
}
+ error = 0;
bytes += req->wb_bytes;
if (test_bit(NFS_IOHDR_ERROR, &hdr->flags)) {
if (bytes <= hdr->good_bytes)
nfs_page_group_set_uptodate(req);
+ else {
+ error = hdr->error;
+ xchg(&nfs_req_openctx(req)->error, error);
+ }
} else
nfs_page_group_set_uptodate(req);
nfs_list_remove_request(req);
- nfs_readpage_release(req);
+ nfs_readpage_release(req, error);
}
out:
hdr->release(hdr);
@@ -213,7 +225,7 @@ nfs_async_read_error(struct list_head *head, int error)
while (!list_empty(head)) {
req = nfs_list_entry(head->next);
nfs_list_remove_request(req);
- nfs_readpage_release(req);
+ nfs_readpage_release(req, error);
}
}
@@ -337,8 +349,13 @@ int nfs_readpage(struct file *file, struct page *page)
goto out;
}
+ xchg(&ctx->error, 0);
error = nfs_readpage_async(ctx, inode, page);
-
+ if (!error) {
+ error = wait_on_page_locked_killable(page);
+ if (!PageUptodate(page) && !error)
+ error = xchg(&ctx->error, 0);
+ }
out:
put_nfs_open_context(ctx);
return error;
@@ -372,8 +389,8 @@ readpage_async_filler(void *data, struct page *page)
zero_user_segment(page, len, PAGE_SIZE);
if (!nfs_pageio_add_request(desc->pgio, new)) {
nfs_list_remove_request(new);
- nfs_readpage_release(new);
error = desc->pgio->pg_error;
+ nfs_readpage_release(new, error);
goto out;
}
return 0;
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 92d9cad..85ca495 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -57,6 +57,7 @@ static const struct rpc_call_ops nfs_commit_ops;
static const struct nfs_pgio_completion_ops nfs_async_write_completion_ops;
static const struct nfs_commit_completion_ops nfs_commit_completion_ops;
static const struct nfs_rw_ops nfs_rw_write_ops;
+static void nfs_inode_remove_request(struct nfs_page *req);
static void nfs_clear_request_commit(struct nfs_page *req);
static void nfs_init_cinfo_from_inode(struct nfs_commit_info *cinfo,
struct inode *inode);
@@ -591,23 +592,13 @@ nfs_lock_and_join_requests(struct page *page)
static void nfs_write_error(struct nfs_page *req, int error)
{
+ nfs_set_pageerror(page_file_mapping(req->wb_page));
nfs_mapping_set_error(req->wb_page, error);
+ nfs_inode_remove_request(req);
nfs_end_page_writeback(req);
nfs_release_request(req);
}
-static bool
-nfs_error_is_fatal_on_server(int err)
-{
- switch (err) {
- case 0:
- case -ERESTARTSYS:
- case -EINTR:
- return false;
- }
- return nfs_error_is_fatal(err);
-}
-
/*
* Find an associated nfs write request, and prepare to flush it out
* May return an error if the user signalled nfs_wait_on_request().
@@ -615,7 +606,6 @@ nfs_error_is_fatal_on_server(int err)
static int nfs_page_async_flush(struct nfs_pageio_descriptor *pgio,
struct page *page)
{
- struct address_space *mapping;
struct nfs_page *req;
int ret = 0;
@@ -630,12 +620,11 @@ static int nfs_page_async_flush(struct nfs_pageio_descriptor *pgio,
WARN_ON_ONCE(test_bit(PG_CLEAN, &req->wb_flags));
/* If there is a fatal error that covers this write, just exit */
- ret = 0;
- mapping = page_file_mapping(page);
- if (test_bit(AS_ENOSPC, &mapping->flags) ||
- test_bit(AS_EIO, &mapping->flags))
+ ret = pgio->pg_error;
+ if (nfs_error_is_fatal_on_server(ret))
goto out_launder;
+ ret = 0;
if (!nfs_pageio_add_request(pgio, req)) {
ret = pgio->pg_error;
/*
@@ -647,6 +636,7 @@ static int nfs_page_async_flush(struct nfs_pageio_descriptor *pgio,
} else
ret = -EAGAIN;
nfs_redirty_request(req);
+ pgio->pg_error = 0;
} else
nfs_add_stats(page_file_mapping(page)->host,
NFSIOS_WRITEPAGES, 1);
@@ -666,7 +656,7 @@ static int nfs_do_writepage(struct page *page, struct writeback_control *wbc,
ret = nfs_page_async_flush(pgio, page);
if (ret == -EAGAIN) {
redirty_page_for_writepage(wbc, page);
- ret = 0;
+ ret = AOP_WRITEPAGE_ACTIVATE;
}
return ret;
}
@@ -685,10 +675,11 @@ static int nfs_writepage_locked(struct page *page,
nfs_pageio_init_write(&pgio, inode, 0,
false, &nfs_async_write_completion_ops);
err = nfs_do_writepage(page, wbc, &pgio);
+ pgio.pg_error = 0;
nfs_pageio_complete(&pgio);
if (err < 0)
return err;
- if (pgio.pg_error < 0)
+ if (nfs_error_is_fatal(pgio.pg_error))
return pgio.pg_error;
return 0;
}
@@ -698,7 +689,8 @@ int nfs_writepage(struct page *page, struct writeback_control *wbc)
int ret;
ret = nfs_writepage_locked(page, wbc);
- unlock_page(page);
+ if (ret != AOP_WRITEPAGE_ACTIVATE)
+ unlock_page(page);
return ret;
}
@@ -707,7 +699,8 @@ static int nfs_writepages_callback(struct page *page, struct writeback_control *
int ret;
ret = nfs_do_writepage(page, wbc, data);
- unlock_page(page);
+ if (ret != AOP_WRITEPAGE_ACTIVATE)
+ unlock_page(page);
return ret;
}
@@ -733,13 +726,14 @@ int nfs_writepages(struct address_space *mapping, struct writeback_control *wbc)
&nfs_async_write_completion_ops);
pgio.pg_io_completion = ioc;
err = write_cache_pages(mapping, wbc, nfs_writepages_callback, &pgio);
+ pgio.pg_error = 0;
nfs_pageio_complete(&pgio);
nfs_io_completion_put(ioc);
if (err < 0)
goto out_err;
err = pgio.pg_error;
- if (err < 0)
+ if (nfs_error_is_fatal(err))
goto out_err;
return 0;
out_err:
diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c
index 26ad75a..96352ab 100644
--- a/fs/nfsd/nfscache.c
+++ b/fs/nfsd/nfscache.c
@@ -571,7 +571,7 @@ nfsd_cache_append(struct svc_rqst *rqstp, struct kvec *data)
*/
static int nfsd_reply_cache_stats_show(struct seq_file *m, void *v)
{
- struct nfsd_net *nn = v;
+ struct nfsd_net *nn = m->private;
seq_printf(m, "max entries: %u\n", nn->max_drc_entries);
seq_printf(m, "num entries: %u\n",
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 13c5487..3cf4f6a 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -1171,13 +1171,17 @@ static struct inode *nfsd_get_inode(struct super_block *sb, umode_t mode)
return inode;
}
-static int __nfsd_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
+static int __nfsd_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode, struct nfsdfs_client *ncl)
{
struct inode *inode;
inode = nfsd_get_inode(dir->i_sb, mode);
if (!inode)
return -ENOMEM;
+ if (ncl) {
+ inode->i_private = ncl;
+ kref_get(&ncl->cl_ref);
+ }
d_add(dentry, inode);
inc_nlink(dir);
fsnotify_mkdir(dir, dentry);
@@ -1194,17 +1198,14 @@ static struct dentry *nfsd_mkdir(struct dentry *parent, struct nfsdfs_client *nc
dentry = d_alloc_name(parent, name);
if (!dentry)
goto out_err;
- ret = __nfsd_mkdir(d_inode(parent), dentry, S_IFDIR | 0600);
+ ret = __nfsd_mkdir(d_inode(parent), dentry, S_IFDIR | 0600, ncl);
if (ret)
goto out_err;
- if (ncl) {
- d_inode(dentry)->i_private = ncl;
- kref_get(&ncl->cl_ref);
- }
out:
inode_unlock(dir);
return dentry;
out_err:
+ dput(dentry);
dentry = ERR_PTR(ret);
goto out;
}
@@ -1214,11 +1215,9 @@ static void clear_ncl(struct inode *inode)
struct nfsdfs_client *ncl = inode->i_private;
inode->i_private = NULL;
- synchronize_rcu();
kref_put(&ncl->cl_ref, ncl->cl_release);
}
-
static struct nfsdfs_client *__get_nfsdfs_client(struct inode *inode)
{
struct nfsdfs_client *nc = inode->i_private;
@@ -1232,9 +1231,9 @@ struct nfsdfs_client *get_nfsdfs_client(struct inode *inode)
{
struct nfsdfs_client *nc;
- rcu_read_lock();
+ inode_lock_shared(inode);
nc = __get_nfsdfs_client(inode);
- rcu_read_unlock();
+ inode_unlock_shared(inode);
return nc;
}
/* from __rpc_unlink */
diff --git a/fs/read_write.c b/fs/read_write.c
index 1f5088d..5bbf587 100644
--- a/fs/read_write.c
+++ b/fs/read_write.c
@@ -1811,10 +1811,7 @@ static int generic_remap_check_len(struct inode *inode_in,
return (remap_flags & REMAP_FILE_DEDUP) ? -EBADE : -EINVAL;
}
-/*
- * Read a page's worth of file data into the page cache. Return the page
- * locked.
- */
+/* Read a page's worth of file data into the page cache. */
static struct page *vfs_dedupe_get_page(struct inode *inode, loff_t offset)
{
struct page *page;
@@ -1826,11 +1823,33 @@ static struct page *vfs_dedupe_get_page(struct inode *inode, loff_t offset)
put_page(page);
return ERR_PTR(-EIO);
}
- lock_page(page);
return page;
}
/*
+ * Lock two pages, ensuring that we lock in offset order if the pages are from
+ * the same file.
+ */
+static void vfs_lock_two_pages(struct page *page1, struct page *page2)
+{
+ /* Always lock in order of increasing index. */
+ if (page1->index > page2->index)
+ swap(page1, page2);
+
+ lock_page(page1);
+ if (page1 != page2)
+ lock_page(page2);
+}
+
+/* Unlock two pages, being careful not to unlock the same page twice. */
+static void vfs_unlock_two_pages(struct page *page1, struct page *page2)
+{
+ unlock_page(page1);
+ if (page1 != page2)
+ unlock_page(page2);
+}
+
+/*
* Compare extents of two files to see if they are the same.
* Caller must have locked both inodes to prevent write races.
*/
@@ -1867,10 +1886,24 @@ static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
dest_page = vfs_dedupe_get_page(dest, destoff);
if (IS_ERR(dest_page)) {
error = PTR_ERR(dest_page);
- unlock_page(src_page);
put_page(src_page);
goto out_error;
}
+
+ vfs_lock_two_pages(src_page, dest_page);
+
+ /*
+ * Now that we've locked both pages, make sure they're still
+ * mapped to the file data we're interested in. If not,
+ * someone is invalidating pages on us and we lose.
+ */
+ if (!PageUptodate(src_page) || !PageUptodate(dest_page) ||
+ src_page->mapping != src->i_mapping ||
+ dest_page->mapping != dest->i_mapping) {
+ same = false;
+ goto unlock;
+ }
+
src_addr = kmap_atomic(src_page);
dest_addr = kmap_atomic(dest_page);
@@ -1882,8 +1915,8 @@ static int vfs_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
kunmap_atomic(dest_addr);
kunmap_atomic(src_addr);
- unlock_page(dest_page);
- unlock_page(src_page);
+unlock:
+ vfs_unlock_two_pages(src_page, dest_page);
put_page(dest_page);
put_page(src_page);
diff --git a/fs/ubifs/budget.c b/fs/ubifs/budget.c
index 80d7301..c0b84e9 100644
--- a/fs/ubifs/budget.c
+++ b/fs/ubifs/budget.c
@@ -51,7 +51,7 @@
static void shrink_liability(struct ubifs_info *c, int nr_to_write)
{
down_read(&c->vfs_sb->s_umount);
- writeback_inodes_sb(c->vfs_sb, WB_REASON_FS_FREE_SPACE);
+ writeback_inodes_sb_nr(c->vfs_sb, nr_to_write, WB_REASON_FS_FREE_SPACE);
up_read(&c->vfs_sb->s_umount);
}
diff --git a/fs/ubifs/orphan.c b/fs/ubifs/orphan.c
index b52624e..3b4b411 100644
--- a/fs/ubifs/orphan.c
+++ b/fs/ubifs/orphan.c
@@ -129,7 +129,6 @@ static void __orphan_drop(struct ubifs_info *c, struct ubifs_orphan *o)
static void orphan_delete(struct ubifs_info *c, struct ubifs_orphan *orph)
{
if (orph->del) {
- spin_unlock(&c->orphan_lock);
dbg_gen("deleted twice ino %lu", orph->inum);
return;
}
@@ -138,7 +137,6 @@ static void orphan_delete(struct ubifs_info *c, struct ubifs_orphan *orph)
orph->del = 1;
orph->dnext = c->orph_dnext;
c->orph_dnext = orph;
- spin_unlock(&c->orphan_lock);
dbg_gen("delete later ino %lu", orph->inum);
return;
}
diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c
index 2c0803b..8c1d571 100644
--- a/fs/ubifs/super.c
+++ b/fs/ubifs/super.c
@@ -609,6 +609,10 @@ static int init_constants_early(struct ubifs_info *c)
c->max_bu_buf_len = UBIFS_MAX_BULK_READ * UBIFS_MAX_DATA_NODE_SZ;
if (c->max_bu_buf_len > c->leb_size)
c->max_bu_buf_len = c->leb_size;
+
+ /* Log is ready, preserve one LEB for commits. */
+ c->min_log_bytes = c->leb_size;
+
return 0;
}
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index ccbdbd6..fe6d804 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -880,6 +880,7 @@ static int userfaultfd_release(struct inode *inode, struct file *file)
/* len == 0 means wake all */
struct userfaultfd_wake_range range = { .len = 0, };
unsigned long new_flags;
+ bool still_valid;
WRITE_ONCE(ctx->released, true);
@@ -895,8 +896,7 @@ static int userfaultfd_release(struct inode *inode, struct file *file)
* taking the mmap_sem for writing.
*/
down_write(&mm->mmap_sem);
- if (!mmget_still_valid(mm))
- goto skip_mm;
+ still_valid = mmget_still_valid(mm);
prev = NULL;
for (vma = mm->mmap; vma; vma = vma->vm_next) {
cond_resched();
@@ -907,19 +907,20 @@ static int userfaultfd_release(struct inode *inode, struct file *file)
continue;
}
new_flags = vma->vm_flags & ~(VM_UFFD_MISSING | VM_UFFD_WP);
- prev = vma_merge(mm, prev, vma->vm_start, vma->vm_end,
- new_flags, vma->anon_vma,
- vma->vm_file, vma->vm_pgoff,
- vma_policy(vma),
- NULL_VM_UFFD_CTX);
- if (prev)
- vma = prev;
- else
- prev = vma;
+ if (still_valid) {
+ prev = vma_merge(mm, prev, vma->vm_start, vma->vm_end,
+ new_flags, vma->anon_vma,
+ vma->vm_file, vma->vm_pgoff,
+ vma_policy(vma),
+ NULL_VM_UFFD_CTX);
+ if (prev)
+ vma = prev;
+ else
+ prev = vma;
+ }
vma->vm_flags = new_flags;
vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
}
-skip_mm:
up_write(&mm->mmap_sem);
mmput(mm);
wakeup:
diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c
index 7fcf756..7bd7534 100644
--- a/fs/xfs/xfs_ioctl32.c
+++ b/fs/xfs/xfs_ioctl32.c
@@ -547,63 +547,12 @@ xfs_file_compat_ioctl(
struct inode *inode = file_inode(filp);
struct xfs_inode *ip = XFS_I(inode);
struct xfs_mount *mp = ip->i_mount;
- void __user *arg = (void __user *)p;
+ void __user *arg = compat_ptr(p);
int error;
trace_xfs_file_compat_ioctl(ip);
switch (cmd) {
- /* No size or alignment issues on any arch */
- case XFS_IOC_DIOINFO:
- case XFS_IOC_FSGEOMETRY_V4:
- case XFS_IOC_FSGEOMETRY:
- case XFS_IOC_AG_GEOMETRY:
- case XFS_IOC_FSGETXATTR:
- case XFS_IOC_FSSETXATTR:
- case XFS_IOC_FSGETXATTRA:
- case XFS_IOC_FSSETDM:
- case XFS_IOC_GETBMAP:
- case XFS_IOC_GETBMAPA:
- case XFS_IOC_GETBMAPX:
- case XFS_IOC_FSCOUNTS:
- case XFS_IOC_SET_RESBLKS:
- case XFS_IOC_GET_RESBLKS:
- case XFS_IOC_FSGROWFSLOG:
- case XFS_IOC_GOINGDOWN:
- case XFS_IOC_ERROR_INJECTION:
- case XFS_IOC_ERROR_CLEARALL:
- case FS_IOC_GETFSMAP:
- case XFS_IOC_SCRUB_METADATA:
- case XFS_IOC_BULKSTAT:
- case XFS_IOC_INUMBERS:
- return xfs_file_ioctl(filp, cmd, p);
-#if !defined(BROKEN_X86_ALIGNMENT) || defined(CONFIG_X86_X32)
- /*
- * These are handled fine if no alignment issues. To support x32
- * which uses native 64-bit alignment we must emit these cases in
- * addition to the ia-32 compat set below.
- */
- case XFS_IOC_ALLOCSP:
- case XFS_IOC_FREESP:
- case XFS_IOC_RESVSP:
- case XFS_IOC_UNRESVSP:
- case XFS_IOC_ALLOCSP64:
- case XFS_IOC_FREESP64:
- case XFS_IOC_RESVSP64:
- case XFS_IOC_UNRESVSP64:
- case XFS_IOC_FSGEOMETRY_V1:
- case XFS_IOC_FSGROWFSDATA:
- case XFS_IOC_FSGROWFSRT:
- case XFS_IOC_ZERO_RANGE:
-#ifdef CONFIG_X86_X32
- /*
- * x32 special: this gets a different cmd number from the ia-32 compat
- * case below; the associated data will match native 64-bit alignment.
- */
- case XFS_IOC_SWAPEXT:
-#endif
- return xfs_file_ioctl(filp, cmd, p);
-#endif
#if defined(BROKEN_X86_ALIGNMENT)
case XFS_IOC_ALLOCSP_32:
case XFS_IOC_FREESP_32:
@@ -705,6 +654,7 @@ xfs_file_compat_ioctl(
case XFS_IOC_FSSETDM_BY_HANDLE_32:
return xfs_compat_fssetdm_by_handle(filp, arg);
default:
- return -ENOIOCTLCMD;
+ /* try the native version */
+ return xfs_file_ioctl(filp, cmd, (unsigned long)arg);
}
}
diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c
index ff3c1fa..fe285d1 100644
--- a/fs/xfs/xfs_iops.c
+++ b/fs/xfs/xfs_iops.c
@@ -793,6 +793,7 @@ xfs_setattr_nonsize(
out_cancel:
xfs_trans_cancel(tp);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
out_dqrele:
xfs_qm_dqrele(udqp);
xfs_qm_dqrele(gdqp);
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index 0c954ca..a339bd5 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -32,7 +32,7 @@ xfs_break_leased_layouts(
struct xfs_inode *ip = XFS_I(inode);
int error;
- while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
+ while ((error = break_layout(inode, false)) == -EWOULDBLOCK) {
xfs_iunlock(ip, *iolock);
*did_unlock = true;
error = break_layout(inode, true);
diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c
index c4ec7af..edbe37b 100644
--- a/fs/xfs/xfs_reflink.c
+++ b/fs/xfs/xfs_reflink.c
@@ -1190,11 +1190,11 @@ xfs_reflink_remap_blocks(
}
/*
- * Grab the exclusive iolock for a data copy from src to dest, making
- * sure to abide vfs locking order (lowest pointer value goes first) and
- * breaking the pnfs layout leases on dest before proceeding. The loop
- * is needed because we cannot call the blocking break_layout() with the
- * src iolock held, and therefore have to back out both locks.
+ * Grab the exclusive iolock for a data copy from src to dest, making sure to
+ * abide vfs locking order (lowest pointer value goes first) and breaking the
+ * layout leases before proceeding. The loop is needed because we cannot call
+ * the blocking break_layout() with the iolocks held, and therefore have to
+ * back out both locks.
*/
static int
xfs_iolock_two_inodes_and_break_layout(
@@ -1203,33 +1203,44 @@ xfs_iolock_two_inodes_and_break_layout(
{
int error;
-retry:
- if (src < dest) {
- inode_lock_shared(src);
- inode_lock_nested(dest, I_MUTEX_NONDIR2);
- } else {
- /* src >= dest */
- inode_lock(dest);
- }
+ if (src > dest)
+ swap(src, dest);
- error = break_layout(dest, false);
- if (error == -EWOULDBLOCK) {
- inode_unlock(dest);
- if (src < dest)
- inode_unlock_shared(src);
+retry:
+ /* Wait to break both inodes' layouts before we start locking. */
+ error = break_layout(src, true);
+ if (error)
+ return error;
+ if (src != dest) {
error = break_layout(dest, true);
if (error)
return error;
- goto retry;
}
+
+ /* Lock one inode and make sure nobody got in and leased it. */
+ inode_lock(src);
+ error = break_layout(src, false);
if (error) {
- inode_unlock(dest);
- if (src < dest)
- inode_unlock_shared(src);
+ inode_unlock(src);
+ if (error == -EWOULDBLOCK)
+ goto retry;
return error;
}
- if (src > dest)
- inode_lock_shared_nested(src, I_MUTEX_NONDIR2);
+
+ if (src == dest)
+ return 0;
+
+ /* Lock the other inode and make sure nobody got in and leased it. */
+ inode_lock_nested(dest, I_MUTEX_NONDIR2);
+ error = break_layout(dest, false);
+ if (error) {
+ inode_unlock(src);
+ inode_unlock(dest);
+ if (error == -EWOULDBLOCK)
+ goto retry;
+ return error;
+ }
+
return 0;
}
@@ -1247,10 +1258,10 @@ xfs_reflink_remap_unlock(
xfs_iunlock(dest, XFS_MMAPLOCK_EXCL);
if (!same_inode)
- xfs_iunlock(src, XFS_MMAPLOCK_SHARED);
+ xfs_iunlock(src, XFS_MMAPLOCK_EXCL);
inode_unlock(inode_out);
if (!same_inode)
- inode_unlock_shared(inode_in);
+ inode_unlock(inode_in);
}
/*
@@ -1325,7 +1336,7 @@ xfs_reflink_remap_prep(
if (same_inode)
xfs_ilock(src, XFS_MMAPLOCK_EXCL);
else
- xfs_lock_two_inodes(src, XFS_MMAPLOCK_SHARED, dest,
+ xfs_lock_two_inodes(src, XFS_MMAPLOCK_EXCL, dest,
XFS_MMAPLOCK_EXCL);
/* Check file eligibility and prepare for block sharing. */
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index f9a5061..5b9d223 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -24,6 +24,9 @@ struct seq_file;
struct btf;
struct btf_type;
+extern struct idr btf_idr;
+extern spinlock_t btf_idr_lock;
+
/* map is generic key/value storage optionally accesible by eBPF programs */
struct bpf_map_ops {
/* funcs callable from userspace (via syscall) */
@@ -647,6 +650,8 @@ void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock);
struct bpf_map *bpf_map_get_with_uref(u32 ufd);
struct bpf_map *__bpf_map_get(struct fd f);
struct bpf_map * __must_check bpf_map_inc(struct bpf_map *map, bool uref);
+struct bpf_map * __must_check bpf_map_inc_not_zero(struct bpf_map *map,
+ bool uref);
void bpf_map_put_with_uref(struct bpf_map *map);
void bpf_map_put(struct bpf_map *map);
int bpf_map_charge_memlock(struct bpf_map *map, u32 pages);
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 5fe99f3..26a6d58 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -355,6 +355,7 @@ struct bpf_verifier_env {
struct bpf_verifier_stack_elem *head; /* stack of verifier states to be processed */
int stack_size; /* number of states to be processed */
bool strict_alignment; /* perform strict pointer alignment checks */
+ bool test_state_freq; /* test verifier with different pruning frequency */
struct bpf_verifier_state *cur_state; /* current verifier state */
struct bpf_verifier_state_list **explored_states; /* search pruning optimization */
struct bpf_verifier_state_list *free_list;
diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h
new file mode 100644
index 0000000..2f5d731
--- /dev/null
+++ b/include/linux/can/can-ml.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR BSD-3-Clause) */
+/* Copyright (c) 2002-2007 Volkswagen Group Electronic Research
+ * Copyright (c) 2017 Pengutronix, Marc Kleine-Budde <kernel@pengutronix.de>
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of Volkswagen nor the names of its contributors
+ * may be used to endorse or promote products derived from this software
+ * without specific prior written permission.
+ *
+ * Alternatively, provided that this notice is retained in full, this
+ * software may be distributed under the terms of the GNU General
+ * Public License ("GPL") version 2, in which case the provisions of the
+ * GPL apply INSTEAD OF those given above.
+ *
+ * The provided data structures and external interfaces from this code
+ * are not restricted to be used by modules with a GPL compatible license.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+ * DAMAGE.
+ *
+ */
+
+#ifndef CAN_ML_H
+#define CAN_ML_H
+
+#include <linux/can.h>
+#include <linux/list.h>
+
+#define CAN_SFF_RCV_ARRAY_SZ (1 << CAN_SFF_ID_BITS)
+#define CAN_EFF_RCV_HASH_BITS 10
+#define CAN_EFF_RCV_ARRAY_SZ (1 << CAN_EFF_RCV_HASH_BITS)
+
+enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX };
+
+struct can_dev_rcv_lists {
+ struct hlist_head rx[RX_MAX];
+ struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ];
+ struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ];
+ int entries;
+};
+
+struct can_ml_priv {
+ struct can_dev_rcv_lists dev_rcv_lists;
+#ifdef CAN_J1939
+ struct j1939_priv *j1939_priv;
+#endif
+};
+
+#endif /* CAN_ML_H */
diff --git a/include/linux/can/core.h b/include/linux/can/core.h
index 708c10d..8339071 100644
--- a/include/linux/can/core.h
+++ b/include/linux/can/core.h
@@ -41,6 +41,14 @@ struct can_proto {
struct proto *prot;
};
+/* required_size
+ * macro to find the minimum size of a struct
+ * that includes a requested member
+ */
+#define CAN_REQUIRED_SIZE(struct_type, member) \
+ (offsetof(typeof(struct_type), member) + \
+ sizeof(((typeof(struct_type) *)(NULL))->member))
+
/* function prototypes for the CAN networklayer core (af_can.c) */
extern int can_proto_register(const struct can_proto *cp);
diff --git a/include/linux/can/dev.h b/include/linux/can/dev.h
index f01623a..9b3c720 100644
--- a/include/linux/can/dev.h
+++ b/include/linux/can/dev.h
@@ -169,7 +169,8 @@ void can_change_state(struct net_device *dev, struct can_frame *cf,
void can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
unsigned int idx);
-struct sk_buff *__can_get_echo_skb(struct net_device *dev, unsigned int idx, u8 *len_ptr);
+struct sk_buff *__can_get_echo_skb(struct net_device *dev, unsigned int idx,
+ u8 *len_ptr);
unsigned int can_get_echo_skb(struct net_device *dev, unsigned int idx);
void can_free_echo_skb(struct net_device *dev, unsigned int idx);
diff --git a/include/linux/can/rx-offload.h b/include/linux/can/rx-offload.h
index 9daa111..01219f2 100644
--- a/include/linux/can/rx-offload.h
+++ b/include/linux/can/rx-offload.h
@@ -15,7 +15,8 @@
struct can_rx_offload {
struct net_device *dev;
- unsigned int (*mailbox_read)(struct can_rx_offload *offload, struct can_frame *cf,
+ unsigned int (*mailbox_read)(struct can_rx_offload *offload,
+ struct can_frame *cf,
u32 *timestamp, unsigned int mb);
struct sk_buff_head skb_queue;
@@ -29,9 +30,13 @@ struct can_rx_offload {
bool inc;
};
-int can_rx_offload_add_timestamp(struct net_device *dev, struct can_rx_offload *offload);
-int can_rx_offload_add_fifo(struct net_device *dev, struct can_rx_offload *offload, unsigned int weight);
-int can_rx_offload_irq_offload_timestamp(struct can_rx_offload *offload, u64 reg);
+int can_rx_offload_add_timestamp(struct net_device *dev,
+ struct can_rx_offload *offload);
+int can_rx_offload_add_fifo(struct net_device *dev,
+ struct can_rx_offload *offload,
+ unsigned int weight);
+int can_rx_offload_irq_offload_timestamp(struct can_rx_offload *offload,
+ u64 reg);
int can_rx_offload_irq_offload_fifo(struct can_rx_offload *offload);
int can_rx_offload_queue_sorted(struct can_rx_offload *offload,
struct sk_buff *skb, u32 timestamp);
diff --git a/include/linux/ceph/buffer.h b/include/linux/ceph/buffer.h
index 5e58bb2..11cdc7c 100644
--- a/include/linux/ceph/buffer.h
+++ b/include/linux/ceph/buffer.h
@@ -30,7 +30,8 @@ static inline struct ceph_buffer *ceph_buffer_get(struct ceph_buffer *b)
static inline void ceph_buffer_put(struct ceph_buffer *b)
{
- kref_put(&b->kref, ceph_buffer_release);
+ if (b)
+ kref_put(&b->kref, ceph_buffer_release);
}
extern int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end);
diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
index c05d4e6..03f8e98 100644
--- a/include/linux/dma-contiguous.h
+++ b/include/linux/dma-contiguous.h
@@ -160,10 +160,7 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
static inline struct page *dma_alloc_contiguous(struct device *dev, size_t size,
gfp_t gfp)
{
- int node = dev ? dev_to_node(dev) : NUMA_NO_NODE;
- size_t align = get_order(PAGE_ALIGN(size));
-
- return alloc_pages_node(node, gfp, align);
+ return NULL;
}
static inline void dma_free_contiguous(struct device *dev, struct page *page,
diff --git a/include/linux/gpio.h b/include/linux/gpio.h
index 40915b4..f757a58 100644
--- a/include/linux/gpio.h
+++ b/include/linux/gpio.h
@@ -241,30 +241,6 @@ static inline int irq_to_gpio(unsigned irq)
return -EINVAL;
}
-static inline int
-gpiochip_add_pin_range(struct gpio_chip *chip, const char *pinctl_name,
- unsigned int gpio_offset, unsigned int pin_offset,
- unsigned int npins)
-{
- WARN_ON(1);
- return -EINVAL;
-}
-
-static inline int
-gpiochip_add_pingroup_range(struct gpio_chip *chip,
- struct pinctrl_dev *pctldev,
- unsigned int gpio_offset, const char *pin_group)
-{
- WARN_ON(1);
- return -EINVAL;
-}
-
-static inline void
-gpiochip_remove_pin_ranges(struct gpio_chip *chip)
-{
- WARN_ON(1);
-}
-
static inline int devm_gpio_request(struct device *dev, unsigned gpio,
const char *label)
{
diff --git a/include/linux/logic_pio.h b/include/linux/logic_pio.h
index cbd9d84..88e1e63 100644
--- a/include/linux/logic_pio.h
+++ b/include/linux/logic_pio.h
@@ -117,6 +117,7 @@ struct logic_pio_hwaddr *find_io_range_by_fwnode(struct fwnode_handle *fwnode);
unsigned long logic_pio_trans_hwaddr(struct fwnode_handle *fwnode,
resource_size_t hw_addr, resource_size_t size);
int logic_pio_register_range(struct logic_pio_hwaddr *newrange);
+void logic_pio_unregister_range(struct logic_pio_hwaddr *range);
resource_size_t logic_pio_to_hwaddr(unsigned long pio);
unsigned long logic_pio_trans_cpuaddr(resource_size_t hw_addr);
diff --git a/include/linux/mdio.h b/include/linux/mdio.h
index e8242ad8..a760424 100644
--- a/include/linux/mdio.h
+++ b/include/linux/mdio.h
@@ -68,6 +68,17 @@ struct mdio_driver {
#define to_mdio_driver(d) \
container_of(to_mdio_common_driver(d), struct mdio_driver, mdiodrv)
+/* device driver data */
+static inline void mdiodev_set_drvdata(struct mdio_device *mdio, void *data)
+{
+ dev_set_drvdata(&mdio->dev, data);
+}
+
+static inline void *mdiodev_get_drvdata(struct mdio_device *mdio)
+{
+ return dev_get_drvdata(&mdio->dev);
+}
+
void mdio_device_free(struct mdio_device *mdiodev);
struct mdio_device *mdio_device_create(struct mii_bus *bus, int addr);
int mdio_device_register(struct mdio_device *mdiodev);
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index c2f056b..f3773e8 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -1162,6 +1162,9 @@ enum mlx5_qcam_feature_groups {
#define MLX5_CAP_FLOWTABLE(mdev, cap) \
MLX5_GET(flow_table_nic_cap, mdev->caps.hca_cur[MLX5_CAP_FLOW_TABLE], cap)
+#define MLX5_CAP64_FLOWTABLE(mdev, cap) \
+ MLX5_GET64(flow_table_nic_cap, (mdev)->caps.hca_cur[MLX5_CAP_FLOW_TABLE], cap)
+
#define MLX5_CAP_FLOWTABLE_MAX(mdev, cap) \
MLX5_GET(flow_table_nic_cap, mdev->caps.hca_max[MLX5_CAP_FLOW_TABLE], cap)
@@ -1225,6 +1228,10 @@ enum mlx5_qcam_feature_groups {
MLX5_GET(e_switch_cap, \
mdev->caps.hca_cur[MLX5_CAP_ESWITCH], cap)
+#define MLX5_CAP64_ESW_FLOWTABLE(mdev, cap) \
+ MLX5_GET64(flow_table_eswitch_cap, \
+ (mdev)->caps.hca_cur[MLX5_CAP_ESWITCH_FLOW_TABLE], cap)
+
#define MLX5_CAP_ESW_MAX(mdev, cap) \
MLX5_GET(e_switch_cap, \
mdev->caps.hca_max[MLX5_CAP_ESWITCH], cap)
@@ -1309,6 +1316,7 @@ enum {
MLX5_PER_PRIORITY_COUNTERS_GROUP = 0x10,
MLX5_PER_TRAFFIC_CLASS_COUNTERS_GROUP = 0x11,
MLX5_PHYSICAL_LAYER_COUNTERS_GROUP = 0x12,
+ MLX5_PER_TRAFFIC_CLASS_CONGESTION_GROUP = 0x13,
MLX5_PHYSICAL_LAYER_STATISTICAL_GROUP = 0x16,
MLX5_INFINIBAND_PORT_COUNTERS_GROUP = 0x20,
};
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 13b4cf2..3e80f03 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -47,6 +47,7 @@
#include <linux/interrupt.h>
#include <linux/idr.h>
#include <linux/notifier.h>
+#include <linux/refcount.h>
#include <linux/mlx5/device.h>
#include <linux/mlx5/doorbell.h>
@@ -390,7 +391,7 @@ enum mlx5_res_type {
struct mlx5_core_rsc_common {
enum mlx5_res_type res;
- atomic_t refcount;
+ refcount_t refcount;
struct completion free;
};
@@ -625,6 +626,11 @@ struct mlx5e_resources {
struct mlx5_sq_bfreg bfreg;
};
+enum mlx5_sw_icm_type {
+ MLX5_SW_ICM_TYPE_STEERING,
+ MLX5_SW_ICM_TYPE_HEADER_MODIFY,
+};
+
#define MLX5_MAX_RESERVED_GIDS 8
struct mlx5_rsvd_gids {
@@ -656,11 +662,15 @@ struct mlx5_clock {
struct mlx5_pps pps_info;
};
+struct mlx5_dm;
struct mlx5_fw_tracer;
struct mlx5_vxlan;
struct mlx5_geneve;
struct mlx5_hv_vhca;
+#define MLX5_LOG_SW_ICM_BLOCK_SIZE(dev) (MLX5_CAP_DEV_MEM(dev, log_sw_icm_alloc_granularity))
+#define MLX5_SW_ICM_BLOCK_SIZE(dev) (1 << MLX5_LOG_SW_ICM_BLOCK_SIZE(dev))
+
struct mlx5_core_dev {
struct device *device;
enum mlx5_coredev_type coredev_type;
@@ -694,6 +704,7 @@ struct mlx5_core_dev {
atomic_t num_qps;
u32 issi;
struct mlx5e_resources mlx5e_res;
+ struct mlx5_dm *dm;
struct mlx5_vxlan *vxlan;
struct mlx5_geneve *geneve;
struct {
@@ -1077,6 +1088,10 @@ int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev,
size_t *offsets);
struct mlx5_uars_page *mlx5_get_uars_page(struct mlx5_core_dev *mdev);
void mlx5_put_uars_page(struct mlx5_core_dev *mdev, struct mlx5_uars_page *up);
+int mlx5_dm_sw_icm_alloc(struct mlx5_core_dev *dev, enum mlx5_sw_icm_type type,
+ u64 length, u16 uid, phys_addr_t *addr, u32 *obj_id);
+int mlx5_dm_sw_icm_dealloc(struct mlx5_core_dev *dev, enum mlx5_sw_icm_type type,
+ u64 length, u16 uid, phys_addr_t addr, u32 obj_id);
#ifdef CONFIG_MLX5_CORE_IPOIB
struct net_device *mlx5_rdma_netdev_alloc(struct mlx5_core_dev *mdev,
diff --git a/include/linux/mlx5/eswitch.h b/include/linux/mlx5/eswitch.h
index 38a70d1..98e667b 100644
--- a/include/linux/mlx5/eswitch.h
+++ b/include/linux/mlx5/eswitch.h
@@ -60,7 +60,6 @@ void *mlx5_eswitch_get_proto_dev(struct mlx5_eswitch *esw,
struct mlx5_eswitch_rep *mlx5_eswitch_vport_rep(struct mlx5_eswitch *esw,
u16 vport_num);
void *mlx5_eswitch_uplink_get_proto_dev(struct mlx5_eswitch *esw, u8 rep_type);
-u8 mlx5_eswitch_mode(struct mlx5_eswitch *esw);
struct mlx5_flow_handle *
mlx5_eswitch_add_send_to_vport_rule(struct mlx5_eswitch *esw,
u16 vport_num, u32 sqn);
@@ -74,7 +73,14 @@ mlx5_eswitch_get_encap_mode(const struct mlx5_core_dev *dev);
bool mlx5_eswitch_vport_match_metadata_enabled(const struct mlx5_eswitch *esw);
u32 mlx5_eswitch_get_vport_metadata_for_match(const struct mlx5_eswitch *esw,
u16 vport_num);
+u8 mlx5_eswitch_mode(struct mlx5_eswitch *esw);
#else /* CONFIG_MLX5_ESWITCH */
+
+static inline u8 mlx5_eswitch_mode(struct mlx5_eswitch *esw)
+{
+ return MLX5_ESWITCH_NONE;
+}
+
static inline enum devlink_eswitch_encap_mode
mlx5_eswitch_get_encap_mode(const struct mlx5_core_dev *dev)
{
diff --git a/include/linux/mlx5/fs.h b/include/linux/mlx5/fs.h
index 1cb1045..724d276 100644
--- a/include/linux/mlx5/fs.h
+++ b/include/linux/mlx5/fs.h
@@ -75,6 +75,7 @@ enum mlx5_flow_namespace_type {
MLX5_FLOW_NAMESPACE_SNIFFER_TX,
MLX5_FLOW_NAMESPACE_EGRESS,
MLX5_FLOW_NAMESPACE_RDMA_RX,
+ MLX5_FLOW_NAMESPACE_RDMA_RX_KERNEL,
};
enum {
@@ -83,6 +84,8 @@ enum {
FDB_SLOW_PATH,
};
+struct mlx5_pkt_reformat;
+struct mlx5_modify_hdr;
struct mlx5_flow_table;
struct mlx5_flow_group;
struct mlx5_flow_namespace;
@@ -120,7 +123,7 @@ struct mlx5_flow_destination {
struct {
u16 num;
u16 vhca_id;
- u32 reformat_id;
+ struct mlx5_pkt_reformat *pkt_reformat;
u8 flags;
} vport;
};
@@ -194,8 +197,8 @@ enum {
struct mlx5_flow_act {
u32 action;
- u32 reformat_id;
- u32 modify_id;
+ struct mlx5_modify_hdr *modify_hdr;
+ struct mlx5_pkt_reformat *pkt_reformat;
uintptr_t esp_id;
u32 flags;
struct mlx5_fs_vlan vlan[MLX5_FS_VLAN_DEPTH];
@@ -204,8 +207,6 @@ struct mlx5_flow_act {
#define MLX5_DECLARE_FLOW_ACT(name) \
struct mlx5_flow_act name = { .action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST,\
- .reformat_id = 0, \
- .modify_id = 0, \
.flags = 0, }
/* Single destination per rule.
@@ -235,19 +236,18 @@ u32 mlx5_fc_id(struct mlx5_fc *counter);
int mlx5_fs_add_rx_underlay_qpn(struct mlx5_core_dev *dev, u32 underlay_qpn);
int mlx5_fs_remove_rx_underlay_qpn(struct mlx5_core_dev *dev, u32 underlay_qpn);
-int mlx5_modify_header_alloc(struct mlx5_core_dev *dev,
- u8 namespace, u8 num_actions,
- void *modify_actions, u32 *modify_header_id);
+struct mlx5_modify_hdr *mlx5_modify_header_alloc(struct mlx5_core_dev *dev,
+ u8 ns_type, u8 num_actions,
+ void *modify_actions);
void mlx5_modify_header_dealloc(struct mlx5_core_dev *dev,
- u32 modify_header_id);
+ struct mlx5_modify_hdr *modify_hdr);
-int mlx5_packet_reformat_alloc(struct mlx5_core_dev *dev,
- int reformat_type,
- size_t size,
- void *reformat_data,
- enum mlx5_flow_namespace_type namespace,
- u32 *packet_reformat_id);
+struct mlx5_pkt_reformat *mlx5_packet_reformat_alloc(struct mlx5_core_dev *dev,
+ int reformat_type,
+ size_t size,
+ void *reformat_data,
+ enum mlx5_flow_namespace_type ns_type);
void mlx5_packet_reformat_dealloc(struct mlx5_core_dev *dev,
- u32 packet_reformat_id);
+ struct mlx5_pkt_reformat *reformat);
#endif
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index a66ed0a..a487b68 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -172,6 +172,8 @@ enum {
MLX5_CMD_OP_QUERY_XRQ_DC_PARAMS_ENTRY = 0x725,
MLX5_CMD_OP_SET_XRQ_DC_PARAMS_ENTRY = 0x726,
MLX5_CMD_OP_QUERY_XRQ_ERROR_PARAMS = 0x727,
+ MLX5_CMD_OP_RELEASE_XRQ_ERROR = 0x729,
+ MLX5_CMD_OP_MODIFY_XRQ = 0x72a,
MLX5_CMD_OP_QUERY_ESW_FUNCTIONS = 0x740,
MLX5_CMD_OP_QUERY_VPORT_STATE = 0x750,
MLX5_CMD_OP_MODIFY_VPORT_STATE = 0x751,
@@ -280,6 +282,7 @@ enum {
MLX5_CMD_OP_ALLOC_MODIFY_HEADER_CONTEXT = 0x940,
MLX5_CMD_OP_DEALLOC_MODIFY_HEADER_CONTEXT = 0x941,
MLX5_CMD_OP_QUERY_MODIFY_HEADER_CONTEXT = 0x942,
+ MLX5_CMD_OP_SYNC_STEERING = 0xb00,
MLX5_CMD_OP_FPGA_CREATE_QP = 0x960,
MLX5_CMD_OP_FPGA_MODIFY_QP = 0x961,
MLX5_CMD_OP_FPGA_QUERY_QP = 0x962,
@@ -483,7 +486,11 @@ union mlx5_ifc_gre_key_bits {
};
struct mlx5_ifc_fte_match_set_misc_bits {
- u8 reserved_at_0[0x8];
+ u8 gre_c_present[0x1];
+ u8 reserved_auto1[0x1];
+ u8 gre_k_present[0x1];
+ u8 gre_s_present[0x1];
+ u8 source_vhca_port[0x4];
u8 source_sqn[0x18];
u8 source_eswitch_owner_vhca_id[0x10];
@@ -563,12 +570,38 @@ struct mlx5_ifc_fte_match_set_misc2_bits {
u8 metadata_reg_a[0x20];
- u8 reserved_at_1a0[0x60];
+ u8 metadata_reg_b[0x20];
+
+ u8 reserved_at_1c0[0x40];
};
struct mlx5_ifc_fte_match_set_misc3_bits {
- u8 reserved_at_0[0x120];
+ u8 inner_tcp_seq_num[0x20];
+
+ u8 outer_tcp_seq_num[0x20];
+
+ u8 inner_tcp_ack_num[0x20];
+
+ u8 outer_tcp_ack_num[0x20];
+
+ u8 reserved_at_80[0x8];
+ u8 outer_vxlan_gpe_vni[0x18];
+
+ u8 outer_vxlan_gpe_next_protocol[0x8];
+ u8 outer_vxlan_gpe_flags[0x8];
+ u8 reserved_at_b0[0x10];
+
+ u8 icmp_header_data[0x20];
+
+ u8 icmpv6_header_data[0x20];
+
+ u8 icmp_type[0x8];
+ u8 icmp_code[0x8];
+ u8 icmpv6_type[0x8];
+ u8 icmpv6_code[0x8];
+
u8 geneve_tlv_option_0_data[0x20];
+
u8 reserved_at_140[0xc0];
};
@@ -664,7 +697,15 @@ struct mlx5_ifc_flow_table_nic_cap_bits {
struct mlx5_ifc_flow_table_prop_layout_bits flow_table_properties_nic_transmit_sniffer;
- u8 reserved_at_e00[0x7200];
+ u8 reserved_at_e00[0x1200];
+
+ u8 sw_steering_nic_rx_action_drop_icm_address[0x40];
+
+ u8 sw_steering_nic_tx_action_drop_icm_address[0x40];
+
+ u8 sw_steering_nic_tx_action_allow_icm_address[0x40];
+
+ u8 reserved_at_20c0[0x5f40];
};
enum {
@@ -696,7 +737,17 @@ struct mlx5_ifc_flow_table_eswitch_cap_bits {
struct mlx5_ifc_flow_table_prop_layout_bits flow_table_properties_esw_acl_egress;
- u8 reserved_at_800[0x7800];
+ u8 reserved_at_800[0x1000];
+
+ u8 sw_steering_fdb_action_drop_icm_address_rx[0x40];
+
+ u8 sw_steering_fdb_action_drop_icm_address_tx[0x40];
+
+ u8 sw_steering_uplink_icm_address_rx[0x40];
+
+ u8 sw_steering_uplink_icm_address_tx[0x40];
+
+ u8 reserved_at_1900[0x6700];
};
enum {
@@ -806,7 +857,9 @@ struct mlx5_ifc_per_protocol_networking_offload_caps_bits {
u8 swp_csum[0x1];
u8 swp_lso[0x1];
u8 cqe_checksum_full[0x1];
- u8 reserved_at_24[0xc];
+ u8 reserved_at_24[0x5];
+ u8 tunnel_stateless_ip_over_ip[0x1];
+ u8 reserved_at_2a[0x6];
u8 max_vxlan_udp_ports[0x8];
u8 reserved_at_38[0x6];
u8 max_geneve_opt_len[0x1];
@@ -845,6 +898,25 @@ struct mlx5_ifc_roce_cap_bits {
u8 reserved_at_100[0x700];
};
+struct mlx5_ifc_sync_steering_in_bits {
+ u8 opcode[0x10];
+ u8 uid[0x10];
+
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+
+ u8 reserved_at_40[0xc0];
+};
+
+struct mlx5_ifc_sync_steering_out_bits {
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+
+ u8 syndrome[0x20];
+
+ u8 reserved_at_40[0x40];
+};
+
struct mlx5_ifc_device_mem_cap_bits {
u8 memic[0x1];
u8 reserved_at_1[0x1f];
@@ -944,7 +1016,9 @@ struct mlx5_ifc_odp_cap_bits {
struct mlx5_ifc_odp_per_transport_service_cap_bits xrc_odp_caps;
- u8 reserved_at_100[0x700];
+ struct mlx5_ifc_odp_per_transport_service_cap_bits dc_odp_caps;
+
+ u8 reserved_at_120[0x6E0];
};
struct mlx5_ifc_calc_op {
@@ -1036,6 +1110,12 @@ enum {
};
enum {
+ MLX5_FLEX_PARSER_VXLAN_GPE_ENABLED = 1 << 7,
+ MLX5_FLEX_PARSER_ICMP_V4_ENABLED = 1 << 8,
+ MLX5_FLEX_PARSER_ICMP_V6_ENABLED = 1 << 9,
+};
+
+enum {
MLX5_UCTX_CAP_RAW_TX = 1UL << 0,
MLX5_UCTX_CAP_INTERNAL_DEV_RES = 1UL << 1,
};
@@ -1114,7 +1194,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 cache_line_128byte[0x1];
u8 reserved_at_165[0x4];
u8 rts2rts_qp_counters_set_id[0x1];
- u8 reserved_at_16a[0x5];
+ u8 reserved_at_16a[0x2];
+ u8 vnic_env_int_rq_oob[0x1];
+ u8 sbcam_reg[0x1];
+ u8 reserved_at_16e[0x1];
u8 qcam_reg[0x1];
u8 gid_table_size[0x10];
@@ -1243,7 +1326,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 reserved_at_263[0x8];
u8 log_bf_reg_size[0x5];
- u8 reserved_at_270[0xb];
+ u8 reserved_at_270[0x8];
+ u8 lag_tx_port_affinity[0x1];
+ u8 reserved_at_279[0x2];
u8 lag_master[0x1];
u8 num_lag_ports[0x4];
@@ -1404,7 +1489,14 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 reserved_at_6c0[0x4];
u8 flex_parser_id_geneve_tlv_option_0[0x4];
- u8 reserved_at_6c8[0x28];
+ u8 flex_parser_id_icmp_dw1[0x4];
+ u8 flex_parser_id_icmp_dw0[0x4];
+ u8 flex_parser_id_icmpv6_dw1[0x4];
+ u8 flex_parser_id_icmpv6_dw0[0x4];
+ u8 flex_parser_id_outer_first_mpls_over_gre[0x4];
+ u8 flex_parser_id_outer_first_mpls_over_udp_label[0x4];
+
+ u8 reserved_at_6e0[0x10];
u8 sf_base_id[0x10];
u8 reserved_at_700[0x80];
@@ -1869,12 +1961,28 @@ struct mlx5_ifc_ib_port_cntrs_grp_data_layout_bits {
u8 port_xmit_wait[0x20];
};
-struct mlx5_ifc_eth_per_traffic_grp_data_layout_bits {
+struct mlx5_ifc_eth_per_tc_prio_grp_data_layout_bits {
u8 transmit_queue_high[0x20];
u8 transmit_queue_low[0x20];
- u8 reserved_at_40[0x780];
+ u8 no_buffer_discard_uc_high[0x20];
+
+ u8 no_buffer_discard_uc_low[0x20];
+
+ u8 reserved_at_80[0x740];
+};
+
+struct mlx5_ifc_eth_per_tc_congest_prio_grp_data_layout_bits {
+ u8 wred_discard_high[0x20];
+
+ u8 wred_discard_low[0x20];
+
+ u8 ecn_marked_tc_high[0x20];
+
+ u8 ecn_marked_tc_low[0x20];
+
+ u8 reserved_at_80[0x740];
};
struct mlx5_ifc_eth_per_prio_grp_data_layout_bits {
@@ -2642,6 +2750,7 @@ union mlx5_ifc_hca_cap_union_bits {
struct mlx5_ifc_debug_cap_bits debug_cap;
struct mlx5_ifc_fpga_cap_bits fpga_cap;
struct mlx5_ifc_tls_cap_bits tls_cap;
+ struct mlx5_ifc_device_mem_cap_bits device_mem_cap;
u8 reserved_at_0[0x8000];
};
@@ -2770,7 +2879,11 @@ struct mlx5_ifc_vnic_diagnostic_statistics_bits {
u8 transmit_discard_vport_down[0x40];
- u8 reserved_at_140[0xec0];
+ u8 reserved_at_140[0xa0];
+
+ u8 internal_rq_out_of_buffer[0x20];
+
+ u8 reserved_at_200[0xe00];
};
struct mlx5_ifc_traffic_counter_bits {
@@ -3241,7 +3354,11 @@ struct mlx5_ifc_esw_vport_context_bits {
u8 cvlan_pcp[0x3];
u8 cvlan_id[0xc];
- u8 reserved_at_60[0x7a0];
+ u8 reserved_at_60[0x720];
+
+ u8 sw_steering_vport_icm_address_rx[0x40];
+
+ u8 sw_steering_vport_icm_address_tx[0x40];
};
enum {
@@ -3542,7 +3659,8 @@ union mlx5_ifc_eth_cntrs_grp_data_layout_auto_bits {
struct mlx5_ifc_eth_3635_cntrs_grp_data_layout_bits eth_3635_cntrs_grp_data_layout;
struct mlx5_ifc_eth_extended_cntrs_grp_data_layout_bits eth_extended_cntrs_grp_data_layout;
struct mlx5_ifc_eth_per_prio_grp_data_layout_bits eth_per_prio_grp_data_layout;
- struct mlx5_ifc_eth_per_traffic_grp_data_layout_bits eth_per_traffic_grp_data_layout;
+ struct mlx5_ifc_eth_per_tc_prio_grp_data_layout_bits eth_per_tc_prio_grp_data_layout;
+ struct mlx5_ifc_eth_per_tc_congest_prio_grp_data_layout_bits eth_per_tc_congest_prio_grp_data_layout;
struct mlx5_ifc_ib_port_cntrs_grp_data_layout_bits ib_port_cntrs_grp_data_layout;
struct mlx5_ifc_phys_layer_cntrs_bits phys_layer_cntrs;
struct mlx5_ifc_phys_layer_statistical_cntrs_bits phys_layer_statistical_cntrs;
@@ -4927,7 +5045,87 @@ struct mlx5_ifc_query_hca_cap_in_bits {
u8 reserved_at_20[0x10];
u8 op_mod[0x10];
- u8 reserved_at_40[0x40];
+ u8 other_function[0x1];
+ u8 reserved_at_41[0xf];
+ u8 function_id[0x10];
+
+ u8 reserved_at_60[0x20];
+};
+
+struct mlx5_ifc_other_hca_cap_bits {
+ u8 roce[0x1];
+ u8 reserved_0[0x27f];
+};
+
+struct mlx5_ifc_query_other_hca_cap_out_bits {
+ u8 status[0x8];
+ u8 reserved_0[0x18];
+
+ u8 syndrome[0x20];
+
+ u8 reserved_1[0x40];
+
+ struct mlx5_ifc_other_hca_cap_bits other_capability;
+};
+
+struct mlx5_ifc_query_other_hca_cap_in_bits {
+ u8 opcode[0x10];
+ u8 reserved_0[0x10];
+
+ u8 reserved_1[0x10];
+ u8 op_mod[0x10];
+
+ u8 reserved_2[0x10];
+ u8 function_id[0x10];
+
+ u8 reserved_3[0x20];
+};
+
+struct mlx5_ifc_modify_other_hca_cap_out_bits {
+ u8 status[0x8];
+ u8 reserved_0[0x18];
+
+ u8 syndrome[0x20];
+
+ u8 reserved_1[0x40];
+};
+
+struct mlx5_ifc_modify_other_hca_cap_in_bits {
+ u8 opcode[0x10];
+ u8 reserved_0[0x10];
+
+ u8 reserved_1[0x10];
+ u8 op_mod[0x10];
+
+ u8 reserved_2[0x10];
+ u8 function_id[0x10];
+ u8 field_select[0x20];
+
+ struct mlx5_ifc_other_hca_cap_bits other_capability;
+};
+
+struct mlx5_ifc_flow_table_context_bits {
+ u8 reformat_en[0x1];
+ u8 decap_en[0x1];
+ u8 sw_owner[0x1];
+ u8 termination_table[0x1];
+ u8 table_miss_action[0x4];
+ u8 level[0x8];
+ u8 reserved_at_10[0x8];
+ u8 log_size[0x8];
+
+ u8 reserved_at_20[0x8];
+ u8 table_miss_id[0x18];
+
+ u8 reserved_at_40[0x8];
+ u8 lag_master_next_table_id[0x18];
+
+ u8 reserved_at_60[0x60];
+
+ u8 sw_owner_icm_root_1[0x40];
+
+ u8 sw_owner_icm_root_0[0x40];
+
};
struct mlx5_ifc_query_flow_table_out_bits {
@@ -4938,12 +5136,7 @@ struct mlx5_ifc_query_flow_table_out_bits {
u8 reserved_at_40[0x80];
- u8 reserved_at_c0[0x8];
- u8 level[0x8];
- u8 reserved_at_d0[0x8];
- u8 log_size[0x8];
-
- u8 reserved_at_e0[0x120];
+ struct mlx5_ifc_flow_table_context_bits flow_table_context;
};
struct mlx5_ifc_query_flow_table_in_bits {
@@ -5213,7 +5406,7 @@ struct mlx5_ifc_alloc_packet_reformat_context_out_bits {
u8 reserved_at_60[0x20];
};
-enum {
+enum mlx5_reformat_ctx_type {
MLX5_REFORMAT_TYPE_L2_TO_VXLAN = 0x0,
MLX5_REFORMAT_TYPE_L2_TO_NVGRE = 0x1,
MLX5_REFORMAT_TYPE_L2_TO_L2_TUNNEL = 0x2,
@@ -5309,7 +5502,16 @@ enum {
MLX5_ACTION_IN_FIELD_OUT_DIPV4 = 0x16,
MLX5_ACTION_IN_FIELD_OUT_FIRST_VID = 0x17,
MLX5_ACTION_IN_FIELD_OUT_IPV6_HOPLIMIT = 0x47,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_A = 0x49,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_B = 0x50,
MLX5_ACTION_IN_FIELD_METADATA_REG_C_0 = 0x51,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_C_1 = 0x52,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_C_2 = 0x53,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_C_3 = 0x54,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_C_4 = 0x55,
+ MLX5_ACTION_IN_FIELD_METADATA_REG_C_5 = 0x56,
+ MLX5_ACTION_IN_FIELD_OUT_TCP_SEQ_NUM = 0x59,
+ MLX5_ACTION_IN_FIELD_OUT_TCP_ACK_NUM = 0x5B,
};
struct mlx5_ifc_alloc_modify_header_context_out_bits {
@@ -7357,35 +7559,26 @@ struct mlx5_ifc_create_mkey_in_bits {
u8 klm_pas_mtt[0][0x20];
};
+enum {
+ MLX5_FLOW_TABLE_TYPE_NIC_RX = 0x0,
+ MLX5_FLOW_TABLE_TYPE_NIC_TX = 0x1,
+ MLX5_FLOW_TABLE_TYPE_ESW_EGRESS_ACL = 0x2,
+ MLX5_FLOW_TABLE_TYPE_ESW_INGRESS_ACL = 0x3,
+ MLX5_FLOW_TABLE_TYPE_FDB = 0X4,
+ MLX5_FLOW_TABLE_TYPE_SNIFFER_RX = 0X5,
+ MLX5_FLOW_TABLE_TYPE_SNIFFER_TX = 0X6,
+};
+
struct mlx5_ifc_create_flow_table_out_bits {
u8 status[0x8];
- u8 reserved_at_8[0x18];
+ u8 icm_address_63_40[0x18];
u8 syndrome[0x20];
- u8 reserved_at_40[0x8];
+ u8 icm_address_39_32[0x8];
u8 table_id[0x18];
- u8 reserved_at_60[0x20];
-};
-
-struct mlx5_ifc_flow_table_context_bits {
- u8 reformat_en[0x1];
- u8 decap_en[0x1];
- u8 reserved_at_2[0x1];
- u8 termination_table[0x1];
- u8 table_miss_action[0x4];
- u8 level[0x8];
- u8 reserved_at_10[0x8];
- u8 log_size[0x8];
-
- u8 reserved_at_20[0x8];
- u8 table_miss_id[0x18];
-
- u8 reserved_at_40[0x8];
- u8 lag_master_next_table_id[0x18];
-
- u8 reserved_at_60[0xe0];
+ u8 icm_address_31_0[0x20];
};
struct mlx5_ifc_create_flow_table_in_bits {
@@ -9247,7 +9440,8 @@ union mlx5_ifc_ports_control_registers_document_bits {
struct mlx5_ifc_eth_802_3_cntrs_grp_data_layout_bits eth_802_3_cntrs_grp_data_layout;
struct mlx5_ifc_eth_extended_cntrs_grp_data_layout_bits eth_extended_cntrs_grp_data_layout;
struct mlx5_ifc_eth_per_prio_grp_data_layout_bits eth_per_prio_grp_data_layout;
- struct mlx5_ifc_eth_per_traffic_grp_data_layout_bits eth_per_traffic_grp_data_layout;
+ struct mlx5_ifc_eth_per_tc_prio_grp_data_layout_bits eth_per_tc_prio_grp_data_layout;
+ struct mlx5_ifc_eth_per_tc_congest_prio_grp_data_layout_bits eth_per_tc_congest_prio_grp_data_layout;
struct mlx5_ifc_lane_2_module_mapping_bits lane_2_module_mapping;
struct mlx5_ifc_pamp_reg_bits pamp_reg;
struct mlx5_ifc_paos_reg_bits paos_reg;
@@ -9594,8 +9788,6 @@ struct mlx5_ifc_query_lag_out_bits {
u8 syndrome[0x20];
- u8 reserved_at_40[0x40];
-
struct mlx5_ifc_lagc_bits ctx;
};
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index d77d717..3f38c30 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -215,8 +215,9 @@ enum node_stat_item {
NR_INACTIVE_FILE, /* " " " " " */
NR_ACTIVE_FILE, /* " " " " " */
NR_UNEVICTABLE, /* " " " " " */
- NR_SLAB_RECLAIMABLE,
- NR_SLAB_UNRECLAIMABLE,
+ NR_SLAB_RECLAIMABLE, /* Please do not reorder this item */
+ NR_SLAB_UNRECLAIMABLE, /* and this one without looking at
+ * memcg_flush_percpu_vmstats() first. */
NR_ISOLATED_ANON, /* Temporary isolated pages from anon lru */
NR_ISOLATED_FILE, /* Temporary isolated pages from file lru */
WORKINGSET_NODES,
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 55ac223..d7d5626 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -901,6 +901,10 @@ struct netdev_bpf {
};
};
+/* Flags for ndo_xsk_wakeup. */
+#define XDP_WAKEUP_RX (1 << 0)
+#define XDP_WAKEUP_TX (1 << 1)
+
#ifdef CONFIG_XFRM_OFFLOAD
struct xfrmdev_ops {
int (*xdo_dev_state_add) (struct xfrm_state *x);
@@ -1227,6 +1231,12 @@ struct tlsdev_ops;
* that got dropped are freed/returned via xdp_return_frame().
* Returns negative number, means general error invoking ndo, meaning
* no frames were xmit'ed and core-caller will free all frames.
+ * int (*ndo_xsk_wakeup)(struct net_device *dev, u32 queue_id, u32 flags);
+ * This function is used to wake up the softirq, ksoftirqd or kthread
+ * responsible for sending and/or receiving packets on a specific
+ * queue id bound to an AF_XDP socket. The flags field specifies if
+ * only RX, only Tx, or both should be woken up using the flags
+ * XDP_WAKEUP_RX and XDP_WAKEUP_TX.
* struct devlink_port *(*ndo_get_devlink_port)(struct net_device *dev);
* Get devlink port instance associated with a given netdev.
* Called with a reference on the netdevice and devlink locks only,
@@ -1426,8 +1436,8 @@ struct net_device_ops {
int (*ndo_xdp_xmit)(struct net_device *dev, int n,
struct xdp_frame **xdp,
u32 flags);
- int (*ndo_xsk_async_xmit)(struct net_device *dev,
- u32 queue_id);
+ int (*ndo_xsk_wakeup)(struct net_device *dev,
+ u32 queue_id, u32 flags);
struct devlink_port * (*ndo_get_devlink_port)(struct net_device *dev);
};
@@ -3272,7 +3282,7 @@ static inline void netdev_tx_completed_queue(struct netdev_queue *dev_queue,
*/
smp_mb();
- if (dql_avail(&dev_queue->dql) < 0)
+ if (unlikely(dql_avail(&dev_queue->dql) < 0))
return;
if (test_and_clear_bit(__QUEUE_STATE_STACK_XOFF, &dev_queue->state))
diff --git a/include/linux/netfilter/nf_conntrack_h323_types.h b/include/linux/netfilter/nf_conntrack_h323_types.h
index 7a6871a..74c6f92 100644
--- a/include/linux/netfilter/nf_conntrack_h323_types.h
+++ b/include/linux/netfilter/nf_conntrack_h323_types.h
@@ -4,6 +4,9 @@
* Copyright (c) 2006 Jing Min Zhao <zhaojingmin@users.sourceforge.net>
*/
+#ifndef _NF_CONNTRACK_H323_TYPES_H
+#define _NF_CONNTRACK_H323_TYPES_H
+
typedef struct TransportAddress_ipAddress { /* SEQUENCE */
int options; /* No use */
unsigned int ip;
@@ -931,3 +934,5 @@ typedef struct RasMessage { /* CHOICE */
InfoRequestResponse infoRequestResponse;
};
} RasMessage;
+
+#endif /* _NF_CONNTRACK_H323_TYPES_H */
diff --git a/include/linux/phy.h b/include/linux/phy.h
index d26779f..a7ecbe0 100644
--- a/include/linux/phy.h
+++ b/include/linux/phy.h
@@ -1117,6 +1117,7 @@ int genphy_c45_an_disable_aneg(struct phy_device *phydev);
int genphy_c45_read_mdix(struct phy_device *phydev);
int genphy_c45_pma_read_abilities(struct phy_device *phydev);
int genphy_c45_read_status(struct phy_device *phydev);
+int genphy_c45_config_aneg(struct phy_device *phydev);
/* The gen10g_* functions are the old Clause 45 stub */
int gen10g_config_aneg(struct phy_device *phydev);
diff --git a/include/linux/qed/qed_if.h b/include/linux/qed/qed_if.h
index e366399..e354638 100644
--- a/include/linux/qed/qed_if.h
+++ b/include/linux/qed/qed_if.h
@@ -1132,6 +1132,26 @@ struct qed_common_ops {
* @param cdev
*/
u8 (*get_affin_hwfn_idx)(struct qed_dev *cdev);
+
+/**
+ * @brief read_nvm_cfg - Read NVM config attribute value.
+ * @param cdev
+ * @param buf - buffer
+ * @param cmd - NVM CFG command id
+ * @param entity_id - Entity id
+ *
+ */
+ int (*read_nvm_cfg)(struct qed_dev *cdev, u8 **buf, u32 cmd,
+ u32 entity_id);
+
+/**
+ * @brief set_grc_config - Configure value for grc config id.
+ * @param cdev
+ * @param cfg_id - grc config id
+ * @param val - grc config value
+ *
+ */
+ int (*set_grc_config)(struct qed_dev *cdev, u32 cfg_id, u32 val);
};
#define MASK_FIELD(_name, _value) \
diff --git a/include/linux/signal.h b/include/linux/signal.h
index b5d99482..1a5f883 100644
--- a/include/linux/signal.h
+++ b/include/linux/signal.h
@@ -282,6 +282,9 @@ extern void signal_setup_done(int failed, struct ksignal *ksig, int stepping);
extern void exit_signals(struct task_struct *tsk);
extern void kernel_sigaction(int, __sighandler_t);
+#define SIG_KTHREAD ((__force __sighandler_t)2)
+#define SIG_KTHREAD_KERNEL ((__force __sighandler_t)3)
+
static inline void allow_signal(int sig)
{
/*
@@ -289,7 +292,17 @@ static inline void allow_signal(int sig)
* know it'll be handled, so that they don't get converted to
* SIGKILL or just silently dropped.
*/
- kernel_sigaction(sig, (__force __sighandler_t)2);
+ kernel_sigaction(sig, SIG_KTHREAD);
+}
+
+static inline void allow_kernel_signal(int sig)
+{
+ /*
+ * Kernel threads handle their own signals. Let the signal code
+ * know signals sent by the kernel will be handled, so that they
+ * don't get silently dropped.
+ */
+ kernel_sigaction(sig, SIG_KTHREAD_KERNEL);
}
static inline void disallow_signal(int sig)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 77c6dc8..028e684 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -279,6 +279,16 @@ struct nf_bridge_info {
};
#endif
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+/* Chain in tc_skb_ext will be used to share the tc chain with
+ * ovs recirc_id. It will be set to the current chain by tc
+ * and read by ovs to recirc_id.
+ */
+struct tc_skb_ext {
+ __u32 chain;
+};
+#endif
+
struct sk_buff_head {
/* These two members must be first. */
struct sk_buff *next;
@@ -4058,6 +4068,9 @@ enum skb_ext_id {
#ifdef CONFIG_XFRM
SKB_EXT_SEC_PATH,
#endif
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ TC_SKB_EXT,
+#endif
SKB_EXT_NUM, /* must be last */
};
diff --git a/include/linux/stmmac.h b/include/linux/stmmac.h
index 5cc6b6f..7ad7ae3 100644
--- a/include/linux/stmmac.h
+++ b/include/linux/stmmac.h
@@ -168,6 +168,7 @@ struct plat_stmmacenet_data {
struct clk *clk_ptp_ref;
unsigned int clk_ptp_rate;
unsigned int clk_ref_rate;
+ s32 ptp_max_adj;
struct reset_control *stmmac_rst;
struct stmmac_axi *axi;
int has_gmac4;
diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h
index baa3ecd..27536b9 100644
--- a/include/linux/sunrpc/sched.h
+++ b/include/linux/sunrpc/sched.h
@@ -98,7 +98,6 @@ typedef void (*rpc_action)(struct rpc_task *);
struct rpc_call_ops {
void (*rpc_call_prepare)(struct rpc_task *, void *);
- void (*rpc_call_prepare_transmit)(struct rpc_task *, void *);
void (*rpc_call_done)(struct rpc_task *, void *);
void (*rpc_count_stats)(struct rpc_task *, void *);
void (*rpc_release)(void *);
diff --git a/include/linux/timekeeper_internal.h b/include/linux/timekeeper_internal.h
index 7acb953..84ff284 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -57,6 +57,7 @@ struct tk_read_base {
* @cs_was_changed_seq: The sequence number of clocksource change events
* @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second
* @raw_sec: CLOCK_MONOTONIC_RAW time in seconds
+ * @monotonic_to_boot: CLOCK_MONOTONIC to CLOCK_BOOTTIME offset
* @cycle_interval: Number of clock cycles in one NTP interval
* @xtime_interval: Number of clock shifted nano seconds in one NTP
* interval.
@@ -84,6 +85,9 @@ struct tk_read_base {
*
* wall_to_monotonic is no longer the boot time, getboottime must be
* used instead.
+ *
+ * @monotonic_to_boottime is a timespec64 representation of @offs_boot to
+ * accelerate the VDSO update for CLOCK_BOOTTIME.
*/
struct timekeeper {
struct tk_read_base tkr_mono;
@@ -99,6 +103,7 @@ struct timekeeper {
u8 cs_was_changed_seq;
ktime_t next_leap_ktime;
u64 raw_sec;
+ struct timespec64 monotonic_to_boot;
/* The following members are for timekeeping internal use */
u64 cycle_interval;
diff --git a/include/linux/tnum.h b/include/linux/tnum.h
index c7dc2b5..c17af77 100644
--- a/include/linux/tnum.h
+++ b/include/linux/tnum.h
@@ -5,6 +5,10 @@
* propagate the unknown bits such that the tnum result represents all the
* possible results for possible values of the operands.
*/
+
+#ifndef _LINUX_TNUM_H
+#define _LINUX_TNUM_H
+
#include <linux/types.h>
struct tnum {
@@ -81,3 +85,5 @@ bool tnum_in(struct tnum a, struct tnum b);
int tnum_strn(char *str, size_t size, struct tnum a);
/* Format a tnum as tristate binary expansion */
int tnum_sbin(char *str, size_t size, struct tnum a);
+
+#endif /* _LINUX_TNUM_H */
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 5150436..30a8cdc 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -548,6 +548,7 @@ extern int trace_event_get_offsets(struct trace_event_call *call);
#define is_signed_type(type) (((type)(-1)) < (type)1)
+int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);
int trace_set_clr_event(const char *system, const char *event, int set);
/*
diff --git a/include/math-emu/op-common.h b/include/math-emu/op-common.h
index f37d128..adcc6a9 100644
--- a/include/math-emu/op-common.h
+++ b/include/math-emu/op-common.h
@@ -308,6 +308,7 @@ do { \
\
case _FP_CLS_COMBINE(FP_CLS_NORMAL,FP_CLS_ZERO): \
R##_e = X##_e; \
+ /* Fall through */ \
case _FP_CLS_COMBINE(FP_CLS_NAN,FP_CLS_NORMAL): \
case _FP_CLS_COMBINE(FP_CLS_NAN,FP_CLS_INF): \
case _FP_CLS_COMBINE(FP_CLS_NAN,FP_CLS_ZERO): \
@@ -318,6 +319,7 @@ do { \
\
case _FP_CLS_COMBINE(FP_CLS_ZERO,FP_CLS_NORMAL): \
R##_e = Y##_e; \
+ /* Fall through */ \
case _FP_CLS_COMBINE(FP_CLS_NORMAL,FP_CLS_NAN): \
case _FP_CLS_COMBINE(FP_CLS_INF,FP_CLS_NAN): \
case _FP_CLS_COMBINE(FP_CLS_ZERO,FP_CLS_NAN): \
@@ -415,6 +417,7 @@ do { \
case _FP_CLS_COMBINE(FP_CLS_NAN,FP_CLS_INF): \
case _FP_CLS_COMBINE(FP_CLS_NAN,FP_CLS_ZERO): \
R##_s = X##_s; \
+ /* Fall through */ \
\
case _FP_CLS_COMBINE(FP_CLS_INF,FP_CLS_INF): \
case _FP_CLS_COMBINE(FP_CLS_INF,FP_CLS_NORMAL): \
@@ -428,6 +431,7 @@ do { \
case _FP_CLS_COMBINE(FP_CLS_INF,FP_CLS_NAN): \
case _FP_CLS_COMBINE(FP_CLS_ZERO,FP_CLS_NAN): \
R##_s = Y##_s; \
+ /* Fall through */ \
\
case _FP_CLS_COMBINE(FP_CLS_NORMAL,FP_CLS_INF): \
case _FP_CLS_COMBINE(FP_CLS_NORMAL,FP_CLS_ZERO): \
@@ -493,6 +497,7 @@ do { \
\
case _FP_CLS_COMBINE(FP_CLS_NORMAL,FP_CLS_ZERO): \
FP_SET_EXCEPTION(FP_EX_DIVZERO); \
+ /* Fall through */ \
case _FP_CLS_COMBINE(FP_CLS_INF,FP_CLS_ZERO): \
case _FP_CLS_COMBINE(FP_CLS_INF,FP_CLS_NORMAL): \
R##_c = FP_CLS_INF; \
diff --git a/include/net/act_api.h b/include/net/act_api.h
index c61a1bf..3a1a729 100644
--- a/include/net/act_api.h
+++ b/include/net/act_api.h
@@ -15,6 +15,7 @@
struct tcf_idrinfo {
struct mutex lock;
struct idr action_idr;
+ struct net *net;
};
struct tc_action_ops;
@@ -108,7 +109,7 @@ struct tc_action_net {
};
static inline
-int tc_action_net_init(struct tc_action_net *tn,
+int tc_action_net_init(struct net *net, struct tc_action_net *tn,
const struct tc_action_ops *ops)
{
int err = 0;
@@ -117,6 +118,7 @@ int tc_action_net_init(struct tc_action_net *tn,
if (!tn->idrinfo)
return -ENOMEM;
tn->ops = ops;
+ tn->idrinfo->net = net;
mutex_init(&tn->idrinfo->lock);
idr_init(&tn->idrinfo->action_idr);
return err;
diff --git a/include/net/addrconf.h b/include/net/addrconf.h
index becdad5..3f62b34 100644
--- a/include/net/addrconf.h
+++ b/include/net/addrconf.h
@@ -206,7 +206,7 @@ static inline int ipv6_mc_may_pull(struct sk_buff *skb,
unsigned int len)
{
if (skb_transport_offset(skb) + ipv6_transport_len(skb) < len)
- return -EINVAL;
+ return 0;
return pskb_may_pull(skb, len);
}
diff --git a/include/net/bluetooth/hci_core.h b/include/net/bluetooth/hci_core.h
index ffc95b3..b689ace 100644
--- a/include/net/bluetooth/hci_core.h
+++ b/include/net/bluetooth/hci_core.h
@@ -1517,6 +1517,8 @@ void hci_mgmt_chan_unregister(struct hci_mgmt_chan *c);
#define DISCOV_INTERLEAVED_INQUIRY_LEN 0x04
#define DISCOV_BREDR_INQUIRY_LEN 0x08
#define DISCOV_LE_RESTART_DELAY msecs_to_jiffies(200) /* msec */
+#define DISCOV_LE_FAST_ADV_INT_MIN 100 /* msec */
+#define DISCOV_LE_FAST_ADV_INT_MAX 150 /* msec */
void mgmt_fill_version_info(void *ver);
int mgmt_new_settings(struct hci_dev *hdev);
diff --git a/include/net/bpf_sk_storage.h b/include/net/bpf_sk_storage.h
index b9dcb02..8e4f831 100644
--- a/include/net/bpf_sk_storage.h
+++ b/include/net/bpf_sk_storage.h
@@ -10,4 +10,14 @@ void bpf_sk_storage_free(struct sock *sk);
extern const struct bpf_func_proto bpf_sk_storage_get_proto;
extern const struct bpf_func_proto bpf_sk_storage_delete_proto;
+#ifdef CONFIG_BPF_SYSCALL
+int bpf_sk_storage_clone(const struct sock *sk, struct sock *newsk);
+#else
+static inline int bpf_sk_storage_clone(const struct sock *sk,
+ struct sock *newsk)
+{
+ return 0;
+}
+#endif
+
#endif /* _BPF_SK_STORAGE_H */
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 7f43c48..03e4d92 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -75,7 +75,7 @@ struct devlink_port {
struct list_head list;
struct list_head param_list;
struct devlink *devlink;
- unsigned index;
+ unsigned int index;
bool registered;
spinlock_t type_lock; /* Protects type and type_dev
* pointer consistency.
@@ -398,6 +398,7 @@ enum devlink_param_generic_id {
DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MAX,
DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MIN,
DEVLINK_PARAM_GENERIC_ID_FW_LOAD_POLICY,
+ DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE,
/* add new param generic ids above here*/
__DEVLINK_PARAM_GENERIC_ID_MAX,
@@ -428,6 +429,10 @@ enum devlink_param_generic_id {
#define DEVLINK_PARAM_GENERIC_FW_LOAD_POLICY_NAME "fw_load_policy"
#define DEVLINK_PARAM_GENERIC_FW_LOAD_POLICY_TYPE DEVLINK_PARAM_TYPE_U8
+#define DEVLINK_PARAM_GENERIC_RESET_DEV_ON_DRV_PROBE_NAME \
+ "reset_dev_on_drv_probe"
+#define DEVLINK_PARAM_GENERIC_RESET_DEV_ON_DRV_PROBE_TYPE DEVLINK_PARAM_TYPE_U8
+
#define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \
{ \
.id = DEVLINK_PARAM_GENERIC_ID_##_id, \
@@ -458,6 +463,13 @@ enum devlink_param_generic_id {
/* Maker of the board */
#define DEVLINK_INFO_VERSION_GENERIC_BOARD_MANUFACTURE "board.manufacture"
+/* Part number, identifier of asic design */
+#define DEVLINK_INFO_VERSION_GENERIC_ASIC_ID "asic.id"
+/* Revision of asic design */
+#define DEVLINK_INFO_VERSION_GENERIC_ASIC_REV "asic.rev"
+
+/* Overall FW version */
+#define DEVLINK_INFO_VERSION_GENERIC_FW "fw"
/* Control processor FW version */
#define DEVLINK_INFO_VERSION_GENERIC_FW_MGMT "fw.mgmt"
/* Data path microcode controlling high-speed packet processing */
diff --git a/include/net/dsa.h b/include/net/dsa.h
index 147b757..96acb14 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -275,9 +275,6 @@ struct dsa_switch {
*/
bool vlan_filtering;
- unsigned long *bitmap;
- unsigned long _bitmap;
-
/* Dynamically allocated ports, keep last */
size_t num_ports;
struct dsa_port ports[];
diff --git a/include/net/flow_offload.h b/include/net/flow_offload.h
index 757fa84..fc88187 100644
--- a/include/net/flow_offload.h
+++ b/include/net/flow_offload.h
@@ -284,6 +284,7 @@ struct flow_block_offload {
enum flow_block_command command;
enum flow_block_binder_type binder_type;
bool block_shared;
+ bool unlocked_driver_cb;
struct net *net;
struct flow_block *block;
struct list_head cb_list;
diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h
index 5003791..a01981d 100644
--- a/include/net/if_inet6.h
+++ b/include/net/if_inet6.h
@@ -89,9 +89,9 @@ struct ip6_sf_socklist {
struct ipv6_mc_socklist {
struct in6_addr addr;
int ifindex;
+ unsigned int sfmode; /* MCAST_{INCLUDE,EXCLUDE} */
struct ipv6_mc_socklist __rcu *next;
rwlock_t sflock;
- unsigned int sfmode; /* MCAST_{INCLUDE,EXCLUDE} */
struct ip6_sf_socklist *sflist;
struct rcu_head rcu;
};
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
index c57d53e..8955460 100644
--- a/include/net/inet_connection_sock.h
+++ b/include/net/inet_connection_sock.h
@@ -97,7 +97,7 @@ struct inet_connection_sock {
const struct tcp_congestion_ops *icsk_ca_ops;
const struct inet_connection_sock_af_ops *icsk_af_ops;
const struct tcp_ulp_ops *icsk_ulp_ops;
- void *icsk_ulp_data;
+ void __rcu *icsk_ulp_data;
void (*icsk_clean_acked)(struct sock *sk, u32 acked_seq);
struct hlist_node icsk_listen_portaddr_node;
unsigned int (*icsk_sync_mss)(struct sock *sk, u32 pmtu);
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index cb668bc..64bcb58 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -52,7 +52,7 @@ struct bpf_prog;
#define NETDEV_HASHENTRIES (1 << NETDEV_HASHBITS)
struct net {
- refcount_t passive; /* To decided when the network
+ refcount_t passive; /* To decide when the network
* namespace should be freed.
*/
refcount_t count; /* To decided when the network
@@ -103,9 +103,6 @@ struct net {
/* core fib_rules */
struct list_head rules_ops;
- struct list_head fib_notifier_ops; /* Populated by
- * register_pernet_subsys()
- */
struct net_device *loopback_dev; /* The loopback */
struct netns_core core;
struct netns_mib mib;
diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h
index e73d16f..3d9e66a 100644
--- a/include/net/netfilter/nf_tables.h
+++ b/include/net/netfilter/nf_tables.h
@@ -2,6 +2,7 @@
#ifndef _NET_NF_TABLES_H
#define _NET_NF_TABLES_H
+#include <asm/unaligned.h>
#include <linux/list.h>
#include <linux/netfilter.h>
#include <linux/netfilter/nfnetlink.h>
@@ -102,33 +103,43 @@ struct nft_regs {
};
};
-/* Store/load an u16 or u8 integer to/from the u32 data register.
+/* Store/load an u8, u16 or u64 integer to/from the u32 data register.
*
* Note, when using concatenations, register allocation happens at 32-bit
* level. So for store instruction, pad the rest part with zero to avoid
* garbage values.
*/
-static inline void nft_reg_store16(u32 *dreg, u16 val)
-{
- *dreg = 0;
- *(u16 *)dreg = val;
-}
-
static inline void nft_reg_store8(u32 *dreg, u8 val)
{
*dreg = 0;
*(u8 *)dreg = val;
}
+static inline u8 nft_reg_load8(u32 *sreg)
+{
+ return *(u8 *)sreg;
+}
+
+static inline void nft_reg_store16(u32 *dreg, u16 val)
+{
+ *dreg = 0;
+ *(u16 *)dreg = val;
+}
+
static inline u16 nft_reg_load16(u32 *sreg)
{
return *(u16 *)sreg;
}
-static inline u8 nft_reg_load8(u32 *sreg)
+static inline void nft_reg_store64(u32 *dreg, u64 val)
{
- return *(u8 *)sreg;
+ put_unaligned(val, (u64 *)dreg);
+}
+
+static inline u64 nft_reg_load64(u32 *sreg)
+{
+ return get_unaligned((u64 *)sreg);
}
static inline void nft_data_copy(u32 *dst, const struct nft_data *src,
@@ -291,17 +302,23 @@ struct nft_expr;
* struct nft_set_ops - nf_tables set operations
*
* @lookup: look up an element within the set
+ * @update: update an element if exists, add it if doesn't exist
+ * @delete: delete an element
* @insert: insert new element into set
* @activate: activate new element in the next generation
* @deactivate: lookup for element and deactivate it in the next generation
* @flush: deactivate element in the next generation
* @remove: remove element from set
- * @walk: iterate over all set elemeennts
+ * @walk: iterate over all set elements
* @get: get set elements
* @privsize: function to return size of set private data
* @init: initialize private data of new set instance
* @destroy: destroy private data of set instance
* @elemsize: element private size
+ *
+ * Operations lookup, update and delete have simpler interfaces, are faster
+ * and currently only used in the packet path. All the rest are slower,
+ * control plane functions.
*/
struct nft_set_ops {
bool (*lookup)(const struct net *net,
@@ -316,6 +333,8 @@ struct nft_set_ops {
const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_set_ext **ext);
+ bool (*delete)(const struct nft_set *set,
+ const u32 *key);
int (*insert)(const struct net *net,
const struct nft_set *set,
@@ -1108,6 +1127,7 @@ struct nft_object_type {
* @init: initialize object from netlink attributes
* @destroy: release existing stateful object
* @dump: netlink dump stateful object
+ * @update: update stateful object
*/
struct nft_object_ops {
void (*eval)(struct nft_object *obj,
@@ -1122,6 +1142,8 @@ struct nft_object_ops {
int (*dump)(struct sk_buff *skb,
struct nft_object *obj,
bool reset);
+ void (*update)(struct nft_object *obj,
+ struct nft_object *newobj);
const struct nft_object_type *type;
};
@@ -1410,10 +1432,16 @@ struct nft_trans_elem {
struct nft_trans_obj {
struct nft_object *obj;
+ struct nft_object *newobj;
+ bool update;
};
#define nft_trans_obj(trans) \
(((struct nft_trans_obj *)trans->data)->obj)
+#define nft_trans_obj_newobj(trans) \
+ (((struct nft_trans_obj *)trans->data)->newobj)
+#define nft_trans_obj_update(trans) \
+ (((struct nft_trans_obj *)trans->data)->update)
struct nft_trans_flowtable {
struct nft_flowtable *flowtable;
diff --git a/include/net/netns/can.h b/include/net/netns/can.h
index ca9bd9f..b6ab7d1 100644
--- a/include/net/netns/can.h
+++ b/include/net/netns/can.h
@@ -9,8 +9,8 @@
#include <linux/spinlock.h>
struct can_dev_rcv_lists;
-struct s_stats;
-struct s_pstats;
+struct can_pkg_stats;
+struct can_rcv_lists_stats;
struct netns_can {
#if IS_ENABLED(CONFIG_PROC_FS)
@@ -28,11 +28,11 @@ struct netns_can {
#endif
/* receive filters subscribed for 'all' CAN devices */
- struct can_dev_rcv_lists *can_rx_alldev_list;
- spinlock_t can_rcvlists_lock;
- struct timer_list can_stattimer;/* timer for statistics update */
- struct s_stats *can_stats; /* packet statistics */
- struct s_pstats *can_pstats; /* receive list statistics */
+ struct can_dev_rcv_lists *rx_alldev_list;
+ spinlock_t rcvlists_lock;
+ struct timer_list stattimer; /* timer for statistics update */
+ struct can_pkg_stats *pkg_stats;
+ struct can_rcv_lists_stats *rcv_lists_stats;
/* CAN GW per-net gateway jobs */
struct hlist_head cgw_list;
diff --git a/include/net/netns/sctp.h b/include/net/netns/sctp.h
index 0db7fb3..bdc0f27 100644
--- a/include/net/netns/sctp.h
+++ b/include/net/netns/sctp.h
@@ -128,6 +128,9 @@ struct netns_sctp {
/* Flag to indicate if stream interleave is enabled */
int intl_enable;
+ /* Flag to indicate if ecn is enabled */
+ int ecn_enable;
+
/*
* Policy to control SCTP IPv4 address scoping
* 0 - Disable IPv4 address scoping
diff --git a/include/net/nexthop.h b/include/net/nexthop.h
index 25f1f9a..95f766c 100644
--- a/include/net/nexthop.h
+++ b/include/net/nexthop.h
@@ -141,12 +141,6 @@ static inline unsigned int nexthop_num_path(const struct nexthop *nh)
nh_grp = rcu_dereference_rtnl(nh->nh_grp);
rc = nh_grp->num_nh;
- } else {
- const struct nh_info *nhi;
-
- nhi = rcu_dereference_rtnl(nh->nh_info);
- if (nhi->reject_nh)
- rc = 0;
}
return rc;
diff --git a/include/net/pkt_cls.h b/include/net/pkt_cls.h
index 64999ff..e553fc8 100644
--- a/include/net/pkt_cls.h
+++ b/include/net/pkt_cls.h
@@ -504,9 +504,26 @@ tcf_match_indev(struct sk_buff *skb, int ifindex)
}
int tc_setup_flow_action(struct flow_action *flow_action,
- const struct tcf_exts *exts);
+ const struct tcf_exts *exts, bool rtnl_held);
+void tc_cleanup_flow_action(struct flow_action *flow_action);
+
int tc_setup_cb_call(struct tcf_block *block, enum tc_setup_type type,
- void *type_data, bool err_stop);
+ void *type_data, bool err_stop, bool rtnl_held);
+int tc_setup_cb_add(struct tcf_block *block, struct tcf_proto *tp,
+ enum tc_setup_type type, void *type_data, bool err_stop,
+ u32 *flags, unsigned int *in_hw_count, bool rtnl_held);
+int tc_setup_cb_replace(struct tcf_block *block, struct tcf_proto *tp,
+ enum tc_setup_type type, void *type_data, bool err_stop,
+ u32 *old_flags, unsigned int *old_in_hw_count,
+ u32 *new_flags, unsigned int *new_in_hw_count,
+ bool rtnl_held);
+int tc_setup_cb_destroy(struct tcf_block *block, struct tcf_proto *tp,
+ enum tc_setup_type type, void *type_data, bool err_stop,
+ u32 *flags, unsigned int *in_hw_count, bool rtnl_held);
+int tc_setup_cb_reoffload(struct tcf_block *block, struct tcf_proto *tp,
+ bool add, flow_setup_cb_t *cb,
+ enum tc_setup_type type, void *type_data,
+ void *cb_priv, u32 *flags, unsigned int *in_hw_count);
unsigned int tcf_exts_num_actions(struct tcf_exts *exts);
struct tc_cls_u32_knode {
diff --git a/include/net/psample.h b/include/net/psample.h
index 37a4df2..6b578ce 100644
--- a/include/net/psample.h
+++ b/include/net/psample.h
@@ -11,6 +11,7 @@ struct psample_group {
u32 group_num;
u32 refcount;
u32 seq;
+ struct rcu_head rcu;
};
struct psample_group *psample_group_get(struct net *net, u32 group_num);
diff --git a/include/net/route.h b/include/net/route.h
index 630a049..dfce19c 100644
--- a/include/net/route.h
+++ b/include/net/route.h
@@ -233,7 +233,7 @@ void rt_del_uncached_list(struct rtable *rt);
int fib_dump_info_fnhe(struct sk_buff *skb, struct netlink_callback *cb,
u32 table_id, struct fib_info *fi,
- int *fa_index, int fa_start);
+ int *fa_index, int fa_start, unsigned int flags);
static inline void ip_rt_put(struct rtable *rt)
{
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index d9f359a..43f5b7e 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -13,6 +13,8 @@
#include <linux/refcount.h>
#include <linux/workqueue.h>
#include <linux/mutex.h>
+#include <linux/rwsem.h>
+#include <linux/atomic.h>
#include <net/gen_stats.h>
#include <net/rtnetlink.h>
#include <net/flow_offload.h>
@@ -310,6 +312,10 @@ struct tcf_proto_ops {
int (*reoffload)(struct tcf_proto *tp, bool add,
flow_setup_cb_t *cb, void *cb_priv,
struct netlink_ext_ack *extack);
+ void (*hw_add)(struct tcf_proto *tp,
+ void *type_data);
+ void (*hw_del)(struct tcf_proto *tp,
+ void *type_data);
void (*bind_class)(void *, u32, unsigned long);
void * (*tmplt_create)(struct net *net,
struct tcf_chain *chain,
@@ -396,11 +402,13 @@ struct tcf_block {
refcount_t refcnt;
struct net *net;
struct Qdisc *q;
+ struct rw_semaphore cb_lock; /* protects cb_list and offload counters */
struct flow_block flow_block;
struct list_head owner_list;
bool keep_dst;
- unsigned int offloadcnt; /* Number of oddloaded filters */
+ atomic_t offloadcnt; /* Number of oddloaded filters */
unsigned int nooffloaddevcnt; /* Number of devs unable to do offload */
+ unsigned int lockeddevcnt; /* Number of devs that require rtnl lock. */
struct {
struct tcf_chain *chain;
struct list_head filter_chain_list;
@@ -436,37 +444,6 @@ static inline bool lockdep_tcf_proto_is_locked(struct tcf_proto *tp)
#define tcf_proto_dereference(p, tp) \
rcu_dereference_protected(p, lockdep_tcf_proto_is_locked(tp))
-static inline void tcf_block_offload_inc(struct tcf_block *block, u32 *flags)
-{
- if (*flags & TCA_CLS_FLAGS_IN_HW)
- return;
- *flags |= TCA_CLS_FLAGS_IN_HW;
- block->offloadcnt++;
-}
-
-static inline void tcf_block_offload_dec(struct tcf_block *block, u32 *flags)
-{
- if (!(*flags & TCA_CLS_FLAGS_IN_HW))
- return;
- *flags &= ~TCA_CLS_FLAGS_IN_HW;
- block->offloadcnt--;
-}
-
-static inline void
-tc_cls_offload_cnt_update(struct tcf_block *block, u32 *cnt,
- u32 *flags, bool add)
-{
- if (add) {
- if (!*cnt)
- tcf_block_offload_inc(block, flags);
- (*cnt)++;
- } else {
- (*cnt)--;
- if (!*cnt)
- tcf_block_offload_dec(block, flags);
- }
-}
-
static inline void qdisc_cb_private_validate(const struct sk_buff *skb, int sz)
{
struct qdisc_skb_cb *qcb;
diff --git a/include/net/sctp/structs.h b/include/net/sctp/structs.h
index daac1ef..503fbc3 100644
--- a/include/net/sctp/structs.h
+++ b/include/net/sctp/structs.h
@@ -1322,7 +1322,8 @@ struct sctp_endpoint {
/* SCTP-AUTH: endpoint shared keys */
struct list_head endpoint_shared_keys;
__u16 active_key_id;
- __u8 auth_enable:1,
+ __u8 ecn_enable:1,
+ auth_enable:1,
intl_enable:1,
prsctp_enable:1,
asconf_enable:1,
diff --git a/include/net/tc_act/tc_tunnel_key.h b/include/net/tc_act/tc_tunnel_key.h
index 7c3f777..0689d9b 100644
--- a/include/net/tc_act/tc_tunnel_key.h
+++ b/include/net/tc_act/tc_tunnel_key.h
@@ -59,4 +59,21 @@ static inline struct ip_tunnel_info *tcf_tunnel_info(const struct tc_action *a)
return NULL;
#endif
}
+
+static inline struct ip_tunnel_info *
+tcf_tunnel_info_copy(const struct tc_action *a)
+{
+#ifdef CONFIG_NET_CLS_ACT
+ struct ip_tunnel_info *tun = tcf_tunnel_info(a);
+
+ if (tun) {
+ size_t tun_size = sizeof(*tun) + tun->options_len;
+ struct ip_tunnel_info *tun_copy = kmemdup(tun, tun_size,
+ GFP_KERNEL);
+
+ return tun_copy;
+ }
+#endif
+ return NULL;
+}
#endif /* __NET_TC_TUNNEL_KEY_H */
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 77fe87f..c9a3f96 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2122,6 +2122,9 @@ struct tcp_ulp_ops {
void (*update)(struct sock *sk, struct proto *p);
/* cleanup ulp */
void (*release)(struct sock *sk);
+ /* diagnostic */
+ int (*get_info)(const struct sock *sk, struct sk_buff *skb);
+ size_t (*get_info_size)(const struct sock *sk);
char name[TCP_ULP_NAME_MAX];
struct module *owner;
diff --git a/include/net/tls.h b/include/net/tls.h
index 41b2d41..c664e6d 100644
--- a/include/net/tls.h
+++ b/include/net/tls.h
@@ -41,6 +41,7 @@
#include <linux/tcp.h>
#include <linux/skmsg.h>
#include <linux/netdevice.h>
+#include <linux/rcupdate.h>
#include <net/tcp.h>
#include <net/strparser.h>
@@ -274,22 +275,13 @@ struct tls_context {
struct proto *sk_proto;
void (*sk_destruct)(struct sock *sk);
- void (*sk_proto_close)(struct sock *sk, long timeout);
-
- int (*setsockopt)(struct sock *sk, int level,
- int optname, char __user *optval,
- unsigned int optlen);
- int (*getsockopt)(struct sock *sk, int level,
- int optname, char __user *optval,
- int __user *optlen);
- int (*hash)(struct sock *sk);
- void (*unhash)(struct sock *sk);
union tls_crypto_context crypto_send;
union tls_crypto_context crypto_recv;
struct list_head list;
refcount_t refcount;
+ struct rcu_head rcu;
};
enum tls_offload_ctx_dir {
@@ -348,7 +340,7 @@ struct tls_offload_context_rx {
#define TLS_OFFLOAD_CONTEXT_SIZE_RX \
(sizeof(struct tls_offload_context_rx) + TLS_DRIVER_STATE_SIZE_RX)
-void tls_ctx_free(struct tls_context *ctx);
+void tls_ctx_free(struct sock *sk, struct tls_context *ctx);
int wait_on_pending_writer(struct sock *sk, long *timeo);
int tls_sk_query(struct sock *sk, int optname, char __user *optval,
int __user *optlen);
@@ -374,13 +366,9 @@ ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos,
struct pipe_inode_info *pipe,
size_t len, unsigned int flags);
-int tls_set_device_offload(struct sock *sk, struct tls_context *ctx);
int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
int tls_device_sendpage(struct sock *sk, struct page *page,
int offset, size_t size, int flags);
-void tls_device_free_resources_tx(struct sock *sk);
-void tls_device_init(void);
-void tls_device_cleanup(void);
int tls_tx_records(struct sock *sk, int flags);
struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context,
@@ -429,6 +417,23 @@ static inline bool is_tx_ready(struct tls_sw_context_tx *ctx)
return READ_ONCE(rec->tx_ready);
}
+static inline u16 tls_user_config(struct tls_context *ctx, bool tx)
+{
+ u16 config = tx ? ctx->tx_conf : ctx->rx_conf;
+
+ switch (config) {
+ case TLS_BASE:
+ return TLS_CONF_BASE;
+ case TLS_SW:
+ return TLS_CONF_SW;
+ case TLS_HW:
+ return TLS_CONF_HW;
+ case TLS_HW_RECORD:
+ return TLS_CONF_HW_RECORD;
+ }
+ return 0;
+}
+
struct sk_buff *
tls_validate_xmit_skb(struct sock *sk, struct net_device *dev,
struct sk_buff *skb);
@@ -467,7 +472,10 @@ static inline struct tls_context *tls_get_ctx(const struct sock *sk)
{
struct inet_connection_sock *icsk = inet_csk(sk);
- return icsk->icsk_ulp_data;
+ /* Use RCU on icsk_ulp_data only for sock diag code,
+ * TLS data path doesn't need rcu_dereference().
+ */
+ return (__force void *)icsk->icsk_ulp_data;
}
static inline void tls_advance_record_sn(struct sock *sk,
@@ -637,7 +645,6 @@ int tls_proccess_cmsg(struct sock *sk, struct msghdr *msg,
unsigned char *record_type);
void tls_register_device(struct tls_device *device);
void tls_unregister_device(struct tls_device *device);
-int tls_device_decrypted(struct sock *sk, struct sk_buff *skb);
int decrypt_skb(struct sock *sk, struct sk_buff *skb,
struct scatterlist *sgout);
struct sk_buff *tls_encrypt_skb(struct sk_buff *skb);
@@ -650,9 +657,40 @@ int tls_sw_fallback_init(struct sock *sk,
struct tls_offload_context_tx *offload_ctx,
struct tls_crypto_info *crypto_info);
+#ifdef CONFIG_TLS_DEVICE
+void tls_device_init(void);
+void tls_device_cleanup(void);
+int tls_set_device_offload(struct sock *sk, struct tls_context *ctx);
+void tls_device_free_resources_tx(struct sock *sk);
int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx);
-
void tls_device_offload_cleanup_rx(struct sock *sk);
void tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq);
+int tls_device_decrypted(struct sock *sk, struct sk_buff *skb);
+#else
+static inline void tls_device_init(void) {}
+static inline void tls_device_cleanup(void) {}
+static inline int
+tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void tls_device_free_resources_tx(struct sock *sk) {}
+
+static inline int
+tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void tls_device_offload_cleanup_rx(struct sock *sk) {}
+static inline void
+tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq) {}
+
+static inline int tls_device_decrypted(struct sock *sk, struct sk_buff *skb)
+{
+ return 0;
+}
+#endif
#endif /* _TLS_OFFLOAD_H */
diff --git a/include/net/udp.h b/include/net/udp.h
index 79d141d..bad74f7 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -480,7 +480,7 @@ static inline struct sk_buff *udp_rcv_segment(struct sock *sk,
* CB fragment
*/
segs = __skb_gso_segment(skb, features, false);
- if (unlikely(IS_ERR_OR_NULL(segs))) {
+ if (IS_ERR_OR_NULL(segs)) {
int segs_nr = skb_shinfo(skb)->gso_segs;
atomic_add(segs_nr, &sk->sk_drops);
diff --git a/include/net/vxlan.h b/include/net/vxlan.h
index dc1583a..335283d 100644
--- a/include/net/vxlan.h
+++ b/include/net/vxlan.h
@@ -391,7 +391,7 @@ static inline bool vxlan_addr_multicast(const union vxlan_addr *ipa)
if (ipa->sa.sa_family == AF_INET6)
return ipv6_addr_is_multicast(&ipa->sin6.sin6_addr);
else
- return IN_MULTICAST(ntohl(ipa->sin.sin_addr.s_addr));
+ return ipv4_is_multicast(ipa->sin.sin_addr.s_addr);
}
#else /* !IS_ENABLED(CONFIG_IPV6) */
@@ -403,7 +403,7 @@ static inline bool vxlan_addr_any(const union vxlan_addr *ipa)
static inline bool vxlan_addr_multicast(const union vxlan_addr *ipa)
{
- return IN_MULTICAST(ntohl(ipa->sin.sin_addr.s_addr));
+ return ipv4_is_multicast(ipa->sin.sin_addr.s_addr);
}
#endif /* IS_ENABLED(CONFIG_IPV6) */
diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h
index 69796d2..c9398ce 100644
--- a/include/net/xdp_sock.h
+++ b/include/net/xdp_sock.h
@@ -16,6 +16,13 @@
struct net_device;
struct xsk_queue;
+/* Masks for xdp_umem_page flags.
+ * The low 12-bits of the addr will be 0 since this is the page address, so we
+ * can use them for flags.
+ */
+#define XSK_NEXT_PG_CONTIG_SHIFT 0
+#define XSK_NEXT_PG_CONTIG_MASK (1ULL << XSK_NEXT_PG_CONTIG_SHIFT)
+
struct xdp_umem_page {
void *addr;
dma_addr_t dma;
@@ -27,6 +34,13 @@ struct xdp_umem_fq_reuse {
u64 handles[];
};
+/* Flags for the umem flags field.
+ *
+ * The NEED_WAKEUP flag is 1 due to the reuse of the flags field for public
+ * flags. See inlude/uapi/include/linux/if_xdp.h.
+ */
+#define XDP_UMEM_USES_NEED_WAKEUP (1 << 1)
+
struct xdp_umem {
struct xsk_queue *fq;
struct xsk_queue *cq;
@@ -41,15 +55,27 @@ struct xdp_umem {
struct work_struct work;
struct page **pgs;
u32 npgs;
+ u16 queue_id;
+ u8 need_wakeup;
+ u8 flags;
int id;
struct net_device *dev;
struct xdp_umem_fq_reuse *fq_reuse;
- u16 queue_id;
bool zc;
spinlock_t xsk_list_lock;
struct list_head xsk_list;
};
+/* Nodes are linked in the struct xdp_sock map_list field, and used to
+ * track which maps a certain socket reside in.
+ */
+struct xsk_map;
+struct xsk_map_node {
+ struct list_head node;
+ struct xsk_map *map;
+ struct xdp_sock **map_entry;
+};
+
struct xdp_sock {
/* struct sock must be the first member of struct xdp_sock */
struct sock sk;
@@ -75,6 +101,9 @@ struct xdp_sock {
/* Protects generic receive. */
spinlock_t rx_lock;
u64 rx_dropped;
+ struct list_head map_list;
+ /* Protects map_list */
+ spinlock_t map_list_lock;
};
struct xdp_buff;
@@ -95,15 +124,47 @@ struct xdp_umem_fq_reuse *xsk_reuseq_swap(struct xdp_umem *umem,
struct xdp_umem_fq_reuse *newq);
void xsk_reuseq_free(struct xdp_umem_fq_reuse *rq);
struct xdp_umem *xdp_get_umem_from_qid(struct net_device *dev, u16 queue_id);
+void xsk_set_rx_need_wakeup(struct xdp_umem *umem);
+void xsk_set_tx_need_wakeup(struct xdp_umem *umem);
+void xsk_clear_rx_need_wakeup(struct xdp_umem *umem);
+void xsk_clear_tx_need_wakeup(struct xdp_umem *umem);
+bool xsk_umem_uses_need_wakeup(struct xdp_umem *umem);
+
+void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs,
+ struct xdp_sock **map_entry);
+int xsk_map_inc(struct xsk_map *map);
+void xsk_map_put(struct xsk_map *map);
+
+static inline u64 xsk_umem_extract_addr(u64 addr)
+{
+ return addr & XSK_UNALIGNED_BUF_ADDR_MASK;
+}
+
+static inline u64 xsk_umem_extract_offset(u64 addr)
+{
+ return addr >> XSK_UNALIGNED_BUF_OFFSET_SHIFT;
+}
+
+static inline u64 xsk_umem_add_offset_to_addr(u64 addr)
+{
+ return xsk_umem_extract_addr(addr) + xsk_umem_extract_offset(addr);
+}
static inline char *xdp_umem_get_data(struct xdp_umem *umem, u64 addr)
{
- return umem->pages[addr >> PAGE_SHIFT].addr + (addr & (PAGE_SIZE - 1));
+ unsigned long page_addr;
+
+ addr = xsk_umem_add_offset_to_addr(addr);
+ page_addr = (unsigned long)umem->pages[addr >> PAGE_SHIFT].addr;
+
+ return (char *)(page_addr & PAGE_MASK) + (addr & ~PAGE_MASK);
}
static inline dma_addr_t xdp_umem_get_dma(struct xdp_umem *umem, u64 addr)
{
- return umem->pages[addr >> PAGE_SHIFT].dma + (addr & (PAGE_SIZE - 1));
+ addr = xsk_umem_add_offset_to_addr(addr);
+
+ return umem->pages[addr >> PAGE_SHIFT].dma + (addr & ~PAGE_MASK);
}
/* Reuse-queue aware version of FILL queue helpers */
@@ -144,6 +205,19 @@ static inline void xsk_umem_fq_reuse(struct xdp_umem *umem, u64 addr)
rq->handles[rq->length++] = addr;
}
+
+/* Handle the offset appropriately depending on aligned or unaligned mode.
+ * For unaligned mode, we store the offset in the upper 16-bits of the address.
+ * For aligned mode, we simply add the offset to the address.
+ */
+static inline u64 xsk_umem_adjust_offset(struct xdp_umem *umem, u64 address,
+ u64 offset)
+{
+ if (umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG)
+ return address + (offset << XSK_UNALIGNED_BUF_OFFSET_SHIFT);
+ else
+ return address + offset;
+}
#else
static inline int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp)
{
@@ -213,6 +287,21 @@ static inline struct xdp_umem *xdp_get_umem_from_qid(struct net_device *dev,
return NULL;
}
+static inline u64 xsk_umem_extract_addr(u64 addr)
+{
+ return 0;
+}
+
+static inline u64 xsk_umem_extract_offset(u64 addr)
+{
+ return 0;
+}
+
+static inline u64 xsk_umem_add_offset_to_addr(u64 addr)
+{
+ return 0;
+}
+
static inline char *xdp_umem_get_data(struct xdp_umem *umem, u64 addr)
{
return NULL;
@@ -241,6 +330,33 @@ static inline void xsk_umem_fq_reuse(struct xdp_umem *umem, u64 addr)
{
}
+static inline void xsk_set_rx_need_wakeup(struct xdp_umem *umem)
+{
+}
+
+static inline void xsk_set_tx_need_wakeup(struct xdp_umem *umem)
+{
+}
+
+static inline void xsk_clear_rx_need_wakeup(struct xdp_umem *umem)
+{
+}
+
+static inline void xsk_clear_tx_need_wakeup(struct xdp_umem *umem)
+{
+}
+
+static inline bool xsk_umem_uses_need_wakeup(struct xdp_umem *umem)
+{
+ return false;
+}
+
+static inline u64 xsk_umem_adjust_offset(struct xdp_umem *umem, u64 handle,
+ u64 offset)
+{
+ return 0;
+}
+
#endif /* CONFIG_XDP_SOCKETS */
#endif /* _LINUX_XDP_SOCK_H */
diff --git a/include/rdma/restrack.h b/include/rdma/restrack.h
index b0fc6b2..83df1ec 100644
--- a/include/rdma/restrack.h
+++ b/include/rdma/restrack.h
@@ -105,8 +105,7 @@ struct rdma_restrack_entry {
};
int rdma_restrack_count(struct ib_device *dev,
- enum rdma_restrack_type type,
- struct pid_namespace *ns);
+ enum rdma_restrack_type type);
void rdma_restrack_kadd(struct rdma_restrack_entry *res);
void rdma_restrack_uadd(struct rdma_restrack_entry *res);
diff --git a/include/soc/arc/mcip.h b/include/soc/arc/mcip.h
index 50f49e0..d1a93c7 100644
--- a/include/soc/arc/mcip.h
+++ b/include/soc/arc/mcip.h
@@ -46,7 +46,9 @@ struct mcip_cmd {
#define CMD_IDU_ENABLE 0x71
#define CMD_IDU_DISABLE 0x72
#define CMD_IDU_SET_MODE 0x74
+#define CMD_IDU_READ_MODE 0x75
#define CMD_IDU_SET_DEST 0x76
+#define CMD_IDU_ACK_CIRQ 0x79
#define CMD_IDU_SET_MASK 0x7C
#define IDU_M_TRIG_LEVEL 0x0
@@ -119,4 +121,13 @@ static inline void __mcip_cmd_data(unsigned int cmd, unsigned int param,
__mcip_cmd(cmd, param);
}
+/*
+ * Read MCIP register
+ */
+static inline unsigned int __mcip_cmd_read(unsigned int cmd, unsigned int param)
+{
+ __mcip_cmd(cmd, param);
+ return read_aux_reg(ARC_REG_MCIP_READBACK);
+}
+
#endif
diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index fa06b52..a13a62d 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -23,20 +23,17 @@
#define __RXRPC_DECLARE_TRACE_ENUMS_ONCE_ONLY
enum rxrpc_skb_trace {
- rxrpc_skb_rx_cleaned,
- rxrpc_skb_rx_freed,
- rxrpc_skb_rx_got,
- rxrpc_skb_rx_lost,
- rxrpc_skb_rx_purged,
- rxrpc_skb_rx_received,
- rxrpc_skb_rx_rotated,
- rxrpc_skb_rx_seen,
- rxrpc_skb_tx_cleaned,
- rxrpc_skb_tx_freed,
- rxrpc_skb_tx_got,
- rxrpc_skb_tx_new,
- rxrpc_skb_tx_rotated,
- rxrpc_skb_tx_seen,
+ rxrpc_skb_cleaned,
+ rxrpc_skb_freed,
+ rxrpc_skb_got,
+ rxrpc_skb_lost,
+ rxrpc_skb_new,
+ rxrpc_skb_purged,
+ rxrpc_skb_received,
+ rxrpc_skb_rotated,
+ rxrpc_skb_seen,
+ rxrpc_skb_unshared,
+ rxrpc_skb_unshared_nomem,
};
enum rxrpc_local_trace {
@@ -228,20 +225,17 @@ enum rxrpc_tx_point {
* Declare tracing information enums and their string mappings for display.
*/
#define rxrpc_skb_traces \
- EM(rxrpc_skb_rx_cleaned, "Rx CLN") \
- EM(rxrpc_skb_rx_freed, "Rx FRE") \
- EM(rxrpc_skb_rx_got, "Rx GOT") \
- EM(rxrpc_skb_rx_lost, "Rx *L*") \
- EM(rxrpc_skb_rx_purged, "Rx PUR") \
- EM(rxrpc_skb_rx_received, "Rx RCV") \
- EM(rxrpc_skb_rx_rotated, "Rx ROT") \
- EM(rxrpc_skb_rx_seen, "Rx SEE") \
- EM(rxrpc_skb_tx_cleaned, "Tx CLN") \
- EM(rxrpc_skb_tx_freed, "Tx FRE") \
- EM(rxrpc_skb_tx_got, "Tx GOT") \
- EM(rxrpc_skb_tx_new, "Tx NEW") \
- EM(rxrpc_skb_tx_rotated, "Tx ROT") \
- E_(rxrpc_skb_tx_seen, "Tx SEE")
+ EM(rxrpc_skb_cleaned, "CLN") \
+ EM(rxrpc_skb_freed, "FRE") \
+ EM(rxrpc_skb_got, "GOT") \
+ EM(rxrpc_skb_lost, "*L*") \
+ EM(rxrpc_skb_new, "NEW") \
+ EM(rxrpc_skb_purged, "PUR") \
+ EM(rxrpc_skb_received, "RCV") \
+ EM(rxrpc_skb_rotated, "ROT") \
+ EM(rxrpc_skb_seen, "SEE") \
+ EM(rxrpc_skb_unshared, "UNS") \
+ E_(rxrpc_skb_unshared_nomem, "US0")
#define rxrpc_local_traces \
EM(rxrpc_local_got, "GOT") \
@@ -643,13 +637,14 @@ TRACE_EVENT(rxrpc_call,
TRACE_EVENT(rxrpc_skb,
TP_PROTO(struct sk_buff *skb, enum rxrpc_skb_trace op,
- int usage, int mod_count, const void *where),
+ int usage, int mod_count, u8 flags, const void *where),
- TP_ARGS(skb, op, usage, mod_count, where),
+ TP_ARGS(skb, op, usage, mod_count, flags, where),
TP_STRUCT__entry(
__field(struct sk_buff *, skb )
__field(enum rxrpc_skb_trace, op )
+ __field(u8, flags )
__field(int, usage )
__field(int, mod_count )
__field(const void *, where )
@@ -657,14 +652,16 @@ TRACE_EVENT(rxrpc_skb,
TP_fast_assign(
__entry->skb = skb;
+ __entry->flags = flags;
__entry->op = op;
__entry->usage = usage;
__entry->mod_count = mod_count;
__entry->where = where;
),
- TP_printk("s=%p %s u=%d m=%d p=%pSR",
+ TP_printk("s=%p %cx %s u=%d m=%d p=%pSR",
__entry->skb,
+ __entry->flags & RXRPC_SKB_TX_BUFFER ? 'T' : 'R',
__print_symbolic(__entry->op, rxrpc_skb_traces),
__entry->usage,
__entry->mod_count,
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 0e66371..77c6be9 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -106,6 +106,7 @@ enum bpf_cmd {
BPF_TASK_FD_QUERY,
BPF_MAP_LOOKUP_AND_DELETE_ELEM,
BPF_MAP_FREEZE,
+ BPF_BTF_GET_NEXT_ID,
};
enum bpf_map_type {
@@ -284,6 +285,9 @@ enum bpf_attach_type {
*/
#define BPF_F_TEST_RND_HI32 (1U << 2)
+/* The verifier internal test flag. Behavior is undefined */
+#define BPF_F_TEST_STATE_FREQ (1U << 3)
+
/* When BPF ldimm64's insn[0].src_reg != 0 then this can have
* two extensions:
*
@@ -337,6 +341,9 @@ enum bpf_attach_type {
#define BPF_F_RDONLY_PROG (1U << 7)
#define BPF_F_WRONLY_PROG (1U << 8)
+/* Clone map from listener for newly accepted socket */
+#define BPF_F_CLONE (1U << 9)
+
/* flags for BPF_PROG_QUERY */
#define BPF_F_QUERY_EFFECTIVE (1U << 0)
@@ -576,6 +583,8 @@ union bpf_attr {
* limited to five).
*
* Each time the helper is called, it appends a line to the trace.
+ * Lines are discarded while *\/sys/kernel/debug/tracing/trace* is
+ * open, use *\/sys/kernel/debug/tracing/trace_pipe* to avoid this.
* The format of the trace is customizable, and the exact output
* one will get depends on the options set in
* *\/sys/kernel/debug/tracing/trace_options* (see also the
@@ -1014,7 +1023,7 @@ union bpf_attr {
* The realm of the route for the packet associated to *skb*, or 0
* if none was found.
*
- * int bpf_perf_event_output(struct pt_reg *ctx, struct bpf_map *map, u64 flags, void *data, u64 size)
+ * int bpf_perf_event_output(struct pt_regs *ctx, struct bpf_map *map, u64 flags, void *data, u64 size)
* Description
* Write raw *data* blob into a special BPF perf event held by
* *map* of type **BPF_MAP_TYPE_PERF_EVENT_ARRAY**. This perf
@@ -1076,7 +1085,7 @@ union bpf_attr {
* Return
* 0 on success, or a negative error in case of failure.
*
- * int bpf_get_stackid(struct pt_reg *ctx, struct bpf_map *map, u64 flags)
+ * int bpf_get_stackid(struct pt_regs *ctx, struct bpf_map *map, u64 flags)
* Description
* Walk a user or a kernel stack and return its id. To achieve
* this, the helper needs *ctx*, which is a pointer to the context
@@ -1725,7 +1734,7 @@ union bpf_attr {
* Return
* 0 on success, or a negative error in case of failure.
*
- * int bpf_override_return(struct pt_reg *regs, u64 rc)
+ * int bpf_override_return(struct pt_regs *regs, u64 rc)
* Description
* Used for error injection, this helper uses kprobes to override
* the return value of the probed function, and to set it to *rc*.
diff --git a/include/uapi/linux/can.h b/include/uapi/linux/can.h
index 0afb7d8..1e988fd 100644
--- a/include/uapi/linux/can.h
+++ b/include/uapi/linux/can.h
@@ -157,7 +157,8 @@ struct canfd_frame {
#define CAN_TP20 4 /* VAG Transport Protocol v2.0 */
#define CAN_MCNET 5 /* Bosch MCNet */
#define CAN_ISOTP 6 /* ISO 15765-2 Transport Protocol */
-#define CAN_NPROTO 7
+#define CAN_J1939 7 /* SAE J1939 */
+#define CAN_NPROTO 8
#define SOL_CAN_BASE 100
@@ -174,6 +175,23 @@ struct sockaddr_can {
/* transport protocol class address information (e.g. ISOTP) */
struct { canid_t rx_id, tx_id; } tp;
+ /* J1939 address information */
+ struct {
+ /* 8 byte name when using dynamic addressing */
+ __u64 name;
+
+ /* pgn:
+ * 8 bit: PS in PDU2 case, else 0
+ * 8 bit: PF
+ * 1 bit: DP
+ * 1 bit: reserved
+ */
+ __u32 pgn;
+
+ /* 1 byte address */
+ __u8 addr;
+ } j1939;
+
/* reserved for future CAN protocols address information */
} can_addr;
};
diff --git a/include/uapi/linux/can/j1939.h b/include/uapi/linux/can/j1939.h
new file mode 100644
index 0000000..c323253
--- /dev/null
+++ b/include/uapi/linux/can/j1939.h
@@ -0,0 +1,99 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * j1939.h
+ *
+ * Copyright (c) 2010-2011 EIA Electronics
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef _UAPI_CAN_J1939_H_
+#define _UAPI_CAN_J1939_H_
+
+#include <linux/types.h>
+#include <linux/socket.h>
+#include <linux/can.h>
+
+#define J1939_MAX_UNICAST_ADDR 0xfd
+#define J1939_IDLE_ADDR 0xfe
+#define J1939_NO_ADDR 0xff /* == broadcast or no addr */
+#define J1939_NO_NAME 0
+#define J1939_PGN_REQUEST 0x0ea00 /* Request PG */
+#define J1939_PGN_ADDRESS_CLAIMED 0x0ee00 /* Address Claimed */
+#define J1939_PGN_ADDRESS_COMMANDED 0x0fed8 /* Commanded Address */
+#define J1939_PGN_PDU1_MAX 0x3ff00
+#define J1939_PGN_MAX 0x3ffff
+#define J1939_NO_PGN 0x40000
+
+/* J1939 Parameter Group Number
+ *
+ * bit 0-7 : PDU Specific (PS)
+ * bit 8-15 : PDU Format (PF)
+ * bit 16 : Data Page (DP)
+ * bit 17 : Reserved (R)
+ * bit 19-31 : set to zero
+ */
+typedef __u32 pgn_t;
+
+/* J1939 Priority
+ *
+ * bit 0-2 : Priority (P)
+ * bit 3-7 : set to zero
+ */
+typedef __u8 priority_t;
+
+/* J1939 NAME
+ *
+ * bit 0-20 : Identity Number
+ * bit 21-31 : Manufacturer Code
+ * bit 32-34 : ECU Instance
+ * bit 35-39 : Function Instance
+ * bit 40-47 : Function
+ * bit 48 : Reserved
+ * bit 49-55 : Vehicle System
+ * bit 56-59 : Vehicle System Instance
+ * bit 60-62 : Industry Group
+ * bit 63 : Arbitrary Address Capable
+ */
+typedef __u64 name_t;
+
+/* J1939 socket options */
+#define SOL_CAN_J1939 (SOL_CAN_BASE + CAN_J1939)
+enum {
+ SO_J1939_FILTER = 1, /* set filters */
+ SO_J1939_PROMISC = 2, /* set/clr promiscuous mode */
+ SO_J1939_SEND_PRIO = 3,
+ SO_J1939_ERRQUEUE = 4,
+};
+
+enum {
+ SCM_J1939_DEST_ADDR = 1,
+ SCM_J1939_DEST_NAME = 2,
+ SCM_J1939_PRIO = 3,
+ SCM_J1939_ERRQUEUE = 4,
+};
+
+enum {
+ J1939_NLA_PAD,
+ J1939_NLA_BYTES_ACKED,
+};
+
+enum {
+ J1939_EE_INFO_NONE,
+ J1939_EE_INFO_TX_ABORT,
+};
+
+struct j1939_filter {
+ name_t name;
+ name_t name_mask;
+ pgn_t pgn;
+ pgn_t pgn_mask;
+ __u8 addr;
+ __u8 addr_mask;
+};
+
+#define J1939_FILTER_MAX 512 /* maximum number of j1939_filter set via setsockopt() */
+
+#endif /* !_UAPI_CAN_J1939_H_ */
diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h
index 546e75d..1da3e83 100644
--- a/include/uapi/linux/devlink.h
+++ b/include/uapi/linux/devlink.h
@@ -202,6 +202,14 @@ enum devlink_param_cmode {
enum devlink_param_fw_load_policy_value {
DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER,
DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH,
+ DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK,
+};
+
+enum devlink_param_reset_dev_on_drv_probe_value {
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN,
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS,
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER,
+ DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK,
};
enum {
diff --git a/include/uapi/linux/if_xdp.h b/include/uapi/linux/if_xdp.h
index faaa5ca..be328c5 100644
--- a/include/uapi/linux/if_xdp.h
+++ b/include/uapi/linux/if_xdp.h
@@ -16,6 +16,18 @@
#define XDP_SHARED_UMEM (1 << 0)
#define XDP_COPY (1 << 1) /* Force copy-mode */
#define XDP_ZEROCOPY (1 << 2) /* Force zero-copy mode */
+/* If this option is set, the driver might go sleep and in that case
+ * the XDP_RING_NEED_WAKEUP flag in the fill and/or Tx rings will be
+ * set. If it is set, the application need to explicitly wake up the
+ * driver with a poll() (Rx and Tx) or sendto() (Tx only). If you are
+ * running the driver and the application on the same core, you should
+ * use this option so that the kernel will yield to the user space
+ * application.
+ */
+#define XDP_USE_NEED_WAKEUP (1 << 3)
+
+/* Flags for xsk_umem_config flags */
+#define XDP_UMEM_UNALIGNED_CHUNK_FLAG (1 << 0)
struct sockaddr_xdp {
__u16 sxdp_family;
@@ -25,10 +37,14 @@ struct sockaddr_xdp {
__u32 sxdp_shared_umem_fd;
};
+/* XDP_RING flags */
+#define XDP_RING_NEED_WAKEUP (1 << 0)
+
struct xdp_ring_offset {
__u64 producer;
__u64 consumer;
__u64 desc;
+ __u64 flags;
};
struct xdp_mmap_offsets {
@@ -53,6 +69,7 @@ struct xdp_umem_reg {
__u64 len; /* Length of packet data area */
__u32 chunk_size;
__u32 headroom;
+ __u32 flags;
};
struct xdp_statistics {
@@ -74,6 +91,11 @@ struct xdp_options {
#define XDP_UMEM_PGOFF_FILL_RING 0x100000000ULL
#define XDP_UMEM_PGOFF_COMPLETION_RING 0x180000000ULL
+/* Masks for unaligned chunks mode */
+#define XSK_UNALIGNED_BUF_OFFSET_SHIFT 48
+#define XSK_UNALIGNED_BUF_ADDR_MASK \
+ ((1ULL << XSK_UNALIGNED_BUF_OFFSET_SHIFT) - 1)
+
/* Rx/Tx descriptor */
struct xdp_desc {
__u64 addr;
diff --git a/include/uapi/linux/inet_diag.h b/include/uapi/linux/inet_diag.h
index e8baca8..a1ff345 100644
--- a/include/uapi/linux/inet_diag.h
+++ b/include/uapi/linux/inet_diag.h
@@ -153,11 +153,20 @@ enum {
INET_DIAG_BBRINFO, /* request as INET_DIAG_VEGASINFO */
INET_DIAG_CLASS_ID, /* request as INET_DIAG_TCLASS */
INET_DIAG_MD5SIG,
+ INET_DIAG_ULP_INFO,
__INET_DIAG_MAX,
};
#define INET_DIAG_MAX (__INET_DIAG_MAX - 1)
+enum {
+ INET_ULP_INFO_UNSPEC,
+ INET_ULP_INFO_NAME,
+ INET_ULP_INFO_TLS,
+ __INET_ULP_INFO_MAX,
+};
+#define INET_ULP_INFO_MAX (__INET_ULP_INFO_MAX - 1)
+
/* INET_DIAG_MEM */
struct inet_diag_meminfo {
diff --git a/include/uapi/linux/jffs2.h b/include/uapi/linux/jffs2.h
index a18b719..784ba0b 100644
--- a/include/uapi/linux/jffs2.h
+++ b/include/uapi/linux/jffs2.h
@@ -77,11 +77,6 @@
#define JFFS2_ACL_VERSION 0x0001
-// Maybe later...
-//#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
-//#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4)
-
-
#define JFFS2_INO_FLAG_PREREAD 1 /* Do read_inode() for this one at
mount time, don't wait for it to
happen later */
diff --git a/include/uapi/linux/net_dropmon.h b/include/uapi/linux/net_dropmon.h
index 75a35dc..8bf79a9 100644
--- a/include/uapi/linux/net_dropmon.h
+++ b/include/uapi/linux/net_dropmon.h
@@ -75,7 +75,7 @@ enum net_dm_attr {
NET_DM_ATTR_PC, /* u64 */
NET_DM_ATTR_SYMBOL, /* string */
NET_DM_ATTR_IN_PORT, /* nested */
- NET_DM_ATTR_TIMESTAMP, /* struct timespec */
+ NET_DM_ATTR_TIMESTAMP, /* u64 */
NET_DM_ATTR_PROTO, /* u16 */
NET_DM_ATTR_PAYLOAD, /* binary */
NET_DM_ATTR_PAD,
diff --git a/include/uapi/linux/netfilter/nf_tables.h b/include/uapi/linux/netfilter/nf_tables.h
index 82abaa1..0ff932d 100644
--- a/include/uapi/linux/netfilter/nf_tables.h
+++ b/include/uapi/linux/netfilter/nf_tables.h
@@ -636,6 +636,7 @@ enum nft_lookup_attributes {
enum nft_dynset_ops {
NFT_DYNSET_OP_ADD,
NFT_DYNSET_OP_UPDATE,
+ NFT_DYNSET_OP_DELETE,
};
enum nft_dynset_flags {
@@ -799,6 +800,9 @@ enum nft_exthdr_attributes {
* @NFT_META_OIFKIND: packet output interface kind name (dev->rtnl_link_ops->kind)
* @NFT_META_BRI_IIFPVID: packet input bridge port pvid
* @NFT_META_BRI_IIFVPROTO: packet input bridge vlan proto
+ * @NFT_META_TIME_NS: time since epoch (in nanoseconds)
+ * @NFT_META_TIME_DAY: day of week (from 0 = Sunday to 6 = Saturday)
+ * @NFT_META_TIME_HOUR: hour of day (in seconds)
*/
enum nft_meta_keys {
NFT_META_LEN,
@@ -831,6 +835,9 @@ enum nft_meta_keys {
NFT_META_OIFKIND,
NFT_META_BRI_IIFPVID,
NFT_META_BRI_IIFVPROTO,
+ NFT_META_TIME_NS,
+ NFT_META_TIME_DAY,
+ NFT_META_TIME_HOUR,
};
/**
diff --git a/include/uapi/linux/netfilter/nfnetlink_log.h b/include/uapi/linux/netfilter/nfnetlink_log.h
index 20983cb..45c8d3b 100644
--- a/include/uapi/linux/netfilter/nfnetlink_log.h
+++ b/include/uapi/linux/netfilter/nfnetlink_log.h
@@ -33,6 +33,15 @@ struct nfulnl_msg_packet_timestamp {
__aligned_be64 usec;
};
+enum nfulnl_vlan_attr {
+ NFULA_VLAN_UNSPEC,
+ NFULA_VLAN_PROTO, /* __be16 skb vlan_proto */
+ NFULA_VLAN_TCI, /* __be16 skb htons(vlan_tci) */
+ __NFULA_VLAN_MAX,
+};
+
+#define NFULA_VLAN_MAX (__NFULA_VLAN_MAX + 1)
+
enum nfulnl_attr_type {
NFULA_UNSPEC,
NFULA_PACKET_HDR,
@@ -54,6 +63,8 @@ enum nfulnl_attr_type {
NFULA_HWLEN, /* hardware header length */
NFULA_CT, /* nf_conntrack_netlink.h */
NFULA_CT_INFO, /* enum ip_conntrack_info */
+ NFULA_VLAN, /* nested attribute: packet vlan info */
+ NFULA_L2HDR, /* full L2 header */
__NFULA_MAX
};
diff --git a/include/uapi/linux/netfilter/xt_nfacct.h b/include/uapi/linux/netfilter/xt_nfacct.h
index 5c8a4d7..b5123ab 100644
--- a/include/uapi/linux/netfilter/xt_nfacct.h
+++ b/include/uapi/linux/netfilter/xt_nfacct.h
@@ -11,4 +11,9 @@ struct xt_nfacct_match_info {
struct nf_acct *nfacct;
};
+struct xt_nfacct_match_info_v1 {
+ char name[NFACCT_NAME_MAX];
+ struct nf_acct *nfacct __attribute__((aligned(8)));
+};
+
#endif /* _XT_NFACCT_MATCH_H */
diff --git a/include/uapi/linux/openvswitch.h b/include/uapi/linux/openvswitch.h
index f271f1e..1887a45 100644
--- a/include/uapi/linux/openvswitch.h
+++ b/include/uapi/linux/openvswitch.h
@@ -123,6 +123,9 @@ struct ovs_vport_stats {
/* Allow datapath to associate multiple Netlink PIDs to each vport */
#define OVS_DP_F_VPORT_PIDS (1 << 1)
+/* Allow tc offload recirc sharing */
+#define OVS_DP_F_TC_RECIRC_SHARING (1 << 2)
+
/* Fixed logical ports. */
#define OVSP_LOCAL ((__u32)0)
diff --git a/include/uapi/linux/pkt_cls.h b/include/uapi/linux/pkt_cls.h
index b057aee..a6aa466 100644
--- a/include/uapi/linux/pkt_cls.h
+++ b/include/uapi/linux/pkt_cls.h
@@ -160,6 +160,8 @@ enum {
TCA_POLICE_RESULT,
TCA_POLICE_TM,
TCA_POLICE_PAD,
+ TCA_POLICE_RATE64,
+ TCA_POLICE_PEAKRATE64,
__TCA_POLICE_MAX
#define TCA_POLICE_RESULT TCA_POLICE_RESULT
};
diff --git a/include/uapi/linux/rds.h b/include/uapi/linux/rds.h
index fd6b5f6..cba368e 100644
--- a/include/uapi/linux/rds.h
+++ b/include/uapi/linux/rds.h
@@ -250,6 +250,7 @@ struct rds_info_rdma_connection {
__u32 rdma_mr_max;
__u32 rdma_mr_size;
__u8 tos;
+ __u8 sl;
__u32 cache_allocs;
};
@@ -265,6 +266,7 @@ struct rds6_info_rdma_connection {
__u32 rdma_mr_max;
__u32 rdma_mr_size;
__u8 tos;
+ __u8 sl;
__u32 cache_allocs;
};
diff --git a/include/uapi/linux/sctp.h b/include/uapi/linux/sctp.h
index 62527ac..6d5b164 100644
--- a/include/uapi/linux/sctp.h
+++ b/include/uapi/linux/sctp.h
@@ -136,6 +136,7 @@ typedef __s32 sctp_assoc_t;
#define SCTP_EVENT 127
#define SCTP_ASCONF_SUPPORTED 128
#define SCTP_AUTH_SUPPORTED 129
+#define SCTP_ECN_SUPPORTED 130
/* PR-SCTP policies */
#define SCTP_PR_SCTP_NONE 0x0000
diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h
index 5b9c267..bcd2869 100644
--- a/include/uapi/linux/tls.h
+++ b/include/uapi/linux/tls.h
@@ -109,4 +109,19 @@ struct tls12_crypto_info_aes_ccm_128 {
unsigned char rec_seq[TLS_CIPHER_AES_CCM_128_REC_SEQ_SIZE];
};
+enum {
+ TLS_INFO_UNSPEC,
+ TLS_INFO_VERSION,
+ TLS_INFO_CIPHER,
+ TLS_INFO_TXCONF,
+ TLS_INFO_RXCONF,
+ __TLS_INFO_MAX,
+};
+#define TLS_INFO_MAX (__TLS_INFO_MAX - 1)
+
+#define TLS_CONF_BASE 1
+#define TLS_CONF_SW 2
+#define TLS_CONF_HW 3
+#define TLS_CONF_HW_RECORD 4
+
#endif /* _UAPI_LINUX_TLS_H */
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 5fcc7a1..adb3adc 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -195,8 +195,8 @@
i < btf_type_vlen(struct_type); \
i++, member++)
-static DEFINE_IDR(btf_idr);
-static DEFINE_SPINLOCK(btf_idr_lock);
+DEFINE_IDR(btf_idr);
+DEFINE_SPINLOCK(btf_idr_lock);
struct btf {
void *data;
@@ -3376,6 +3376,15 @@ void btf_type_seq_show(const struct btf *btf, u32 type_id, void *obj,
btf_type_ops(t)->seq_show(btf, t, type_id, obj, 0, m);
}
+#ifdef CONFIG_PROC_FS
+static void bpf_btf_show_fdinfo(struct seq_file *m, struct file *filp)
+{
+ const struct btf *btf = filp->private_data;
+
+ seq_printf(m, "btf_id:\t%u\n", btf->id);
+}
+#endif
+
static int btf_release(struct inode *inode, struct file *filp)
{
btf_put(filp->private_data);
@@ -3383,6 +3392,9 @@ static int btf_release(struct inode *inode, struct file *filp)
}
const struct file_operations btf_fops = {
+#ifdef CONFIG_PROC_FS
+ .show_fdinfo = bpf_btf_show_fdinfo,
+#endif
.release = btf_release,
};
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 8191a7d..66088a9 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -890,7 +890,8 @@ int bpf_jit_get_func_addr(const struct bpf_prog *prog,
static int bpf_jit_blind_insn(const struct bpf_insn *from,
const struct bpf_insn *aux,
- struct bpf_insn *to_buff)
+ struct bpf_insn *to_buff,
+ bool emit_zext)
{
struct bpf_insn *to = to_buff;
u32 imm_rnd = get_random_int();
@@ -1005,6 +1006,8 @@ static int bpf_jit_blind_insn(const struct bpf_insn *from,
case 0: /* Part 2 of BPF_LD | BPF_IMM | BPF_DW. */
*to++ = BPF_ALU32_IMM(BPF_MOV, BPF_REG_AX, imm_rnd ^ aux[0].imm);
*to++ = BPF_ALU32_IMM(BPF_XOR, BPF_REG_AX, imm_rnd);
+ if (emit_zext)
+ *to++ = BPF_ZEXT_REG(BPF_REG_AX);
*to++ = BPF_ALU64_REG(BPF_OR, aux[0].dst_reg, BPF_REG_AX);
break;
@@ -1088,7 +1091,8 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
insn[1].code == 0)
memcpy(aux, insn, sizeof(aux));
- rewritten = bpf_jit_blind_insn(insn, aux, insn_buff);
+ rewritten = bpf_jit_blind_insn(insn, aux, insn_buff,
+ clone->aux->verifier_zext);
if (!rewritten)
continue;
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 5d141f1..82eabd4 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -683,8 +683,8 @@ struct bpf_map *bpf_map_get_with_uref(u32 ufd)
}
/* map_idr_lock should have been held */
-static struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map,
- bool uref)
+static struct bpf_map *__bpf_map_inc_not_zero(struct bpf_map *map,
+ bool uref)
{
int refold;
@@ -704,6 +704,16 @@ static struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map,
return map;
}
+struct bpf_map *bpf_map_inc_not_zero(struct bpf_map *map, bool uref)
+{
+ spin_lock_bh(&map_idr_lock);
+ map = __bpf_map_inc_not_zero(map, uref);
+ spin_unlock_bh(&map_idr_lock);
+
+ return map;
+}
+EXPORT_SYMBOL_GPL(bpf_map_inc_not_zero);
+
int __weak bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
{
return -ENOTSUPP;
@@ -1619,6 +1629,7 @@ static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
if (attr->prog_flags & ~(BPF_F_STRICT_ALIGNMENT |
BPF_F_ANY_ALIGNMENT |
+ BPF_F_TEST_STATE_FREQ |
BPF_F_TEST_RND_HI32))
return -EINVAL;
@@ -1707,20 +1718,26 @@ static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
if (err)
goto free_used_maps;
- err = bpf_prog_new_fd(prog);
- if (err < 0) {
- /* failed to allocate fd.
- * bpf_prog_put() is needed because the above
- * bpf_prog_alloc_id() has published the prog
- * to the userspace and the userspace may
- * have refcnt-ed it through BPF_PROG_GET_FD_BY_ID.
- */
- bpf_prog_put(prog);
- return err;
- }
-
+ /* Upon success of bpf_prog_alloc_id(), the BPF prog is
+ * effectively publicly exposed. However, retrieving via
+ * bpf_prog_get_fd_by_id() will take another reference,
+ * therefore it cannot be gone underneath us.
+ *
+ * Only for the time /after/ successful bpf_prog_new_fd()
+ * and before returning to userspace, we might just hold
+ * one reference and any parallel close on that fd could
+ * rip everything out. Hence, below notifications must
+ * happen before bpf_prog_new_fd().
+ *
+ * Also, any failure handling from this point onwards must
+ * be using bpf_prog_put() given the program is exposed.
+ */
bpf_prog_kallsyms_add(prog);
perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_LOAD, 0);
+
+ err = bpf_prog_new_fd(prog);
+ if (err < 0)
+ bpf_prog_put(prog);
return err;
free_used_maps:
@@ -2177,7 +2194,7 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr)
spin_lock_bh(&map_idr_lock);
map = idr_find(&map_idr, id);
if (map)
- map = bpf_map_inc_not_zero(map, true);
+ map = __bpf_map_inc_not_zero(map, true);
else
map = ERR_PTR(-ENOENT);
spin_unlock_bh(&map_idr_lock);
@@ -2874,6 +2891,10 @@ SYSCALL_DEFINE3(bpf, int, cmd, union bpf_attr __user *, uattr, unsigned int, siz
err = bpf_obj_get_next_id(&attr, uattr,
&map_idr, &map_idr_lock);
break;
+ case BPF_BTF_GET_NEXT_ID:
+ err = bpf_obj_get_next_id(&attr, uattr,
+ &btf_idr, &btf_idr_lock);
+ break;
case BPF_PROG_GET_FD_BY_ID:
err = bpf_prog_get_fd_by_id(&attr);
break;
diff --git a/kernel/bpf/sysfs_btf.c b/kernel/bpf/sysfs_btf.c
index 4659349..7ae5ddd 100644
--- a/kernel/bpf/sysfs_btf.c
+++ b/kernel/bpf/sysfs_btf.c
@@ -30,17 +30,12 @@ static struct kobject *btf_kobj;
static int __init btf_vmlinux_init(void)
{
- int err;
-
if (!_binary__btf_vmlinux_bin_start)
return 0;
btf_kobj = kobject_create_and_add("btf", kernel_kobj);
- if (IS_ERR(btf_kobj)) {
- err = PTR_ERR(btf_kobj);
- btf_kobj = NULL;
- return err;
- }
+ if (!btf_kobj)
+ return -ENOMEM;
bin_attr_btf_vmlinux.size = _binary__btf_vmlinux_bin_end -
_binary__btf_vmlinux_bin_start;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 10c0ff9..3fb5075 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -985,9 +985,6 @@ static void __mark_reg_unbounded(struct bpf_reg_state *reg)
reg->smax_value = S64_MAX;
reg->umin_value = 0;
reg->umax_value = U64_MAX;
-
- /* constant backtracking is enabled for root only for now */
- reg->precise = capable(CAP_SYS_ADMIN) ? false : true;
}
/* Mark a register as having a completely unknown (scalar) value. */
@@ -1014,7 +1011,11 @@ static void mark_reg_unknown(struct bpf_verifier_env *env,
__mark_reg_not_init(regs + regno);
return;
}
- __mark_reg_unknown(regs + regno);
+ regs += regno;
+ __mark_reg_unknown(regs);
+ /* constant backtracking is enabled for root without bpf2bpf calls */
+ regs->precise = env->subprog_cnt > 1 || !env->allow_ptr_leaks ?
+ true : false;
}
static void __mark_reg_not_init(struct bpf_reg_state *reg)
@@ -7222,7 +7223,7 @@ static int is_state_visited(struct bpf_verifier_env *env, int insn_idx)
struct bpf_verifier_state_list *sl, **pprev;
struct bpf_verifier_state *cur = env->cur_state, *new;
int i, j, err, states_cnt = 0;
- bool add_new_state = false;
+ bool add_new_state = env->test_state_freq ? true : false;
cur->last_insn_idx = env->prev_insn_idx;
if (!env->insn_aux_data[insn_idx].prune_point)
@@ -9262,6 +9263,9 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr,
env->allow_ptr_leaks = is_priv;
+ if (is_priv)
+ env->test_state_freq = attr->prog_flags & BPF_F_TEST_STATE_FREQ;
+
ret = replace_map_fd_with_map_ptr(env);
if (ret < 0)
goto skip_full_check;
diff --git a/kernel/bpf/xskmap.c b/kernel/bpf/xskmap.c
index 9bb96ac..942c662 100644
--- a/kernel/bpf/xskmap.c
+++ b/kernel/bpf/xskmap.c
@@ -13,8 +13,71 @@ struct xsk_map {
struct bpf_map map;
struct xdp_sock **xsk_map;
struct list_head __percpu *flush_list;
+ spinlock_t lock; /* Synchronize map updates */
};
+int xsk_map_inc(struct xsk_map *map)
+{
+ struct bpf_map *m = &map->map;
+
+ m = bpf_map_inc(m, false);
+ return PTR_ERR_OR_ZERO(m);
+}
+
+void xsk_map_put(struct xsk_map *map)
+{
+ bpf_map_put(&map->map);
+}
+
+static struct xsk_map_node *xsk_map_node_alloc(struct xsk_map *map,
+ struct xdp_sock **map_entry)
+{
+ struct xsk_map_node *node;
+ int err;
+
+ node = kzalloc(sizeof(*node), GFP_ATOMIC | __GFP_NOWARN);
+ if (!node)
+ return NULL;
+
+ err = xsk_map_inc(map);
+ if (err) {
+ kfree(node);
+ return ERR_PTR(err);
+ }
+
+ node->map = map;
+ node->map_entry = map_entry;
+ return node;
+}
+
+static void xsk_map_node_free(struct xsk_map_node *node)
+{
+ xsk_map_put(node->map);
+ kfree(node);
+}
+
+static void xsk_map_sock_add(struct xdp_sock *xs, struct xsk_map_node *node)
+{
+ spin_lock_bh(&xs->map_list_lock);
+ list_add_tail(&node->node, &xs->map_list);
+ spin_unlock_bh(&xs->map_list_lock);
+}
+
+static void xsk_map_sock_delete(struct xdp_sock *xs,
+ struct xdp_sock **map_entry)
+{
+ struct xsk_map_node *n, *tmp;
+
+ spin_lock_bh(&xs->map_list_lock);
+ list_for_each_entry_safe(n, tmp, &xs->map_list, node) {
+ if (map_entry == n->map_entry) {
+ list_del(&n->node);
+ xsk_map_node_free(n);
+ }
+ }
+ spin_unlock_bh(&xs->map_list_lock);
+}
+
static struct bpf_map *xsk_map_alloc(union bpf_attr *attr)
{
struct xsk_map *m;
@@ -34,6 +97,7 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr)
return ERR_PTR(-ENOMEM);
bpf_map_init_from_attr(&m->map, attr);
+ spin_lock_init(&m->lock);
cost = (u64)m->map.max_entries * sizeof(struct xdp_sock *);
cost += sizeof(struct list_head) * num_possible_cpus();
@@ -71,21 +135,9 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr)
static void xsk_map_free(struct bpf_map *map)
{
struct xsk_map *m = container_of(map, struct xsk_map, map);
- int i;
bpf_clear_redirect_map(map);
synchronize_net();
-
- for (i = 0; i < map->max_entries; i++) {
- struct xdp_sock *xs;
-
- xs = m->xsk_map[i];
- if (!xs)
- continue;
-
- sock_put((struct sock *)xs);
- }
-
free_percpu(m->flush_list);
bpf_map_area_free(m->xsk_map);
kfree(m);
@@ -164,8 +216,9 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value,
u64 map_flags)
{
struct xsk_map *m = container_of(map, struct xsk_map, map);
+ struct xdp_sock *xs, *old_xs, **map_entry;
u32 i = *(u32 *)key, fd = *(u32 *)value;
- struct xdp_sock *xs, *old_xs;
+ struct xsk_map_node *node;
struct socket *sock;
int err;
@@ -173,8 +226,6 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value,
return -EINVAL;
if (unlikely(i >= m->map.max_entries))
return -E2BIG;
- if (unlikely(map_flags == BPF_NOEXIST))
- return -EEXIST;
sock = sockfd_lookup(fd, &err);
if (!sock)
@@ -192,32 +243,70 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value,
return -EOPNOTSUPP;
}
- sock_hold(sock->sk);
+ map_entry = &m->xsk_map[i];
+ node = xsk_map_node_alloc(m, map_entry);
+ if (IS_ERR(node)) {
+ sockfd_put(sock);
+ return PTR_ERR(node);
+ }
- old_xs = xchg(&m->xsk_map[i], xs);
+ spin_lock_bh(&m->lock);
+ old_xs = READ_ONCE(*map_entry);
+ if (old_xs == xs) {
+ err = 0;
+ goto out;
+ } else if (old_xs && map_flags == BPF_NOEXIST) {
+ err = -EEXIST;
+ goto out;
+ } else if (!old_xs && map_flags == BPF_EXIST) {
+ err = -ENOENT;
+ goto out;
+ }
+ xsk_map_sock_add(xs, node);
+ WRITE_ONCE(*map_entry, xs);
if (old_xs)
- sock_put((struct sock *)old_xs);
-
+ xsk_map_sock_delete(old_xs, map_entry);
+ spin_unlock_bh(&m->lock);
sockfd_put(sock);
return 0;
+
+out:
+ spin_unlock_bh(&m->lock);
+ sockfd_put(sock);
+ xsk_map_node_free(node);
+ return err;
}
static int xsk_map_delete_elem(struct bpf_map *map, void *key)
{
struct xsk_map *m = container_of(map, struct xsk_map, map);
- struct xdp_sock *old_xs;
+ struct xdp_sock *old_xs, **map_entry;
int k = *(u32 *)key;
if (k >= map->max_entries)
return -EINVAL;
- old_xs = xchg(&m->xsk_map[k], NULL);
+ spin_lock_bh(&m->lock);
+ map_entry = &m->xsk_map[k];
+ old_xs = xchg(map_entry, NULL);
if (old_xs)
- sock_put((struct sock *)old_xs);
+ xsk_map_sock_delete(old_xs, map_entry);
+ spin_unlock_bh(&m->lock);
return 0;
}
+void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs,
+ struct xdp_sock **map_entry)
+{
+ spin_lock_bh(&map->lock);
+ if (READ_ONCE(*map_entry) == xs) {
+ WRITE_ONCE(*map_entry, NULL);
+ xsk_map_sock_delete(xs, map_entry);
+ }
+ spin_unlock_bh(&map->lock);
+}
+
const struct bpf_map_ops xsk_map_ops = {
.map_alloc = xsk_map_alloc,
.map_free = xsk_map_free,
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index 2bd410f..69cfb43 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -230,9 +230,7 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
*/
struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
{
- int node = dev ? dev_to_node(dev) : NUMA_NO_NODE;
- size_t count = PAGE_ALIGN(size) >> PAGE_SHIFT;
- size_t align = get_order(PAGE_ALIGN(size));
+ size_t count = size >> PAGE_SHIFT;
struct page *page = NULL;
struct cma *cma = NULL;
@@ -243,14 +241,12 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
/* CMA can be used only in the context which permits sleeping */
if (cma && gfpflags_allow_blocking(gfp)) {
+ size_t align = get_order(size);
size_t cma_align = min_t(size_t, align, CONFIG_CMA_ALIGNMENT);
page = cma_alloc(cma, count, cma_align, gfp & __GFP_NOWARN);
}
- /* Fallback allocation of normal pages */
- if (!page)
- page = alloc_pages_node(node, gfp, align);
return page;
}
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 795c9b0..706113c 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -85,6 +85,8 @@ static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size)
struct page *__dma_direct_alloc_pages(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs)
{
+ size_t alloc_size = PAGE_ALIGN(size);
+ int node = dev_to_node(dev);
struct page *page = NULL;
u64 phys_mask;
@@ -95,8 +97,14 @@ struct page *__dma_direct_alloc_pages(struct device *dev, size_t size,
gfp &= ~__GFP_ZERO;
gfp |= __dma_direct_optimal_gfp_mask(dev, dev->coherent_dma_mask,
&phys_mask);
+ page = dma_alloc_contiguous(dev, alloc_size, gfp);
+ if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) {
+ dma_free_contiguous(dev, page, alloc_size);
+ page = NULL;
+ }
again:
- page = dma_alloc_contiguous(dev, size, gfp);
+ if (!page)
+ page = alloc_pages_node(node, gfp, get_order(alloc_size));
if (page && !dma_coherent_ok(dev, page_to_phys(page), size)) {
dma_free_contiguous(dev, page, size);
page = NULL;
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 9484e88..9be995f 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -295,6 +295,18 @@ static void irq_sysfs_add(int irq, struct irq_desc *desc)
}
}
+static void irq_sysfs_del(struct irq_desc *desc)
+{
+ /*
+ * If irq_sysfs_init() has not yet been invoked (early boot), then
+ * irq_kobj_base is NULL and the descriptor was never added.
+ * kobject_del() complains about a object with no parent, so make
+ * it conditional.
+ */
+ if (irq_kobj_base)
+ kobject_del(&desc->kobj);
+}
+
static int __init irq_sysfs_init(void)
{
struct irq_desc *desc;
@@ -325,6 +337,7 @@ static struct kobj_type irq_kobj_type = {
};
static void irq_sysfs_add(int irq, struct irq_desc *desc) {}
+static void irq_sysfs_del(struct irq_desc *desc) {}
#endif /* CONFIG_SYSFS */
@@ -438,7 +451,7 @@ static void free_desc(unsigned int irq)
* The sysfs entry must be serialized against a concurrent
* irq_sysfs_init() as well.
*/
- kobject_del(&desc->kobj);
+ irq_sysfs_del(desc);
delete_irq_desc(irq);
/*
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index 95a260f..136ce04 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -263,8 +263,10 @@ int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize,
{
char namebuf[KSYM_NAME_LEN];
- if (is_ksym_addr(addr))
- return !!get_symbol_pos(addr, symbolsize, offset);
+ if (is_ksym_addr(addr)) {
+ get_symbol_pos(addr, symbolsize, offset);
+ return 1;
+ }
return !!module_address_lookup(addr, symbolsize, offset, NULL, namebuf) ||
!!__bpf_address_lookup(addr, symbolsize, offset, namebuf);
}
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 9873fc6..d9770a53 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -470,6 +470,7 @@ static DECLARE_DELAYED_WORK(optimizing_work, kprobe_optimizer);
*/
static void do_optimize_kprobes(void)
{
+ lockdep_assert_held(&text_mutex);
/*
* The optimization/unoptimization refers online_cpus via
* stop_machine() and cpu-hotplug modifies online_cpus.
@@ -487,9 +488,7 @@ static void do_optimize_kprobes(void)
list_empty(&optimizing_list))
return;
- mutex_lock(&text_mutex);
arch_optimize_kprobes(&optimizing_list);
- mutex_unlock(&text_mutex);
}
/*
@@ -500,6 +499,7 @@ static void do_unoptimize_kprobes(void)
{
struct optimized_kprobe *op, *tmp;
+ lockdep_assert_held(&text_mutex);
/* See comment in do_optimize_kprobes() */
lockdep_assert_cpus_held();
@@ -507,7 +507,6 @@ static void do_unoptimize_kprobes(void)
if (list_empty(&unoptimizing_list))
return;
- mutex_lock(&text_mutex);
arch_unoptimize_kprobes(&unoptimizing_list, &freeing_list);
/* Loop free_list for disarming */
list_for_each_entry_safe(op, tmp, &freeing_list, list) {
@@ -524,7 +523,6 @@ static void do_unoptimize_kprobes(void)
} else
list_del_init(&op->list);
}
- mutex_unlock(&text_mutex);
}
/* Reclaim all kprobes on the free_list */
@@ -556,6 +554,7 @@ static void kprobe_optimizer(struct work_struct *work)
{
mutex_lock(&kprobe_mutex);
cpus_read_lock();
+ mutex_lock(&text_mutex);
/* Lock modules while optimizing kprobes */
mutex_lock(&module_mutex);
@@ -583,6 +582,7 @@ static void kprobe_optimizer(struct work_struct *work)
do_free_cleaned_kprobes();
mutex_unlock(&module_mutex);
+ mutex_unlock(&text_mutex);
cpus_read_unlock();
mutex_unlock(&kprobe_mutex);
diff --git a/kernel/module.c b/kernel/module.c
index 5933395..9ee9342 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -65,9 +65,9 @@
/*
* Modules' sections will be aligned on page boundaries
* to ensure complete separation of code and data, but
- * only when CONFIG_STRICT_MODULE_RWX=y
+ * only when CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
*/
-#ifdef CONFIG_STRICT_MODULE_RWX
+#ifdef CONFIG_ARCH_HAS_STRICT_MODULE_RWX
# define debug_align(X) ALIGN(X, PAGE_SIZE)
#else
# define debug_align(X) (X)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2b037f1..010d578 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3904,7 +3904,7 @@ void __noreturn do_task_dead(void)
static inline void sched_submit_work(struct task_struct *tsk)
{
- if (!tsk->state || tsk_is_pi_blocked(tsk))
+ if (!tsk->state)
return;
/*
@@ -3920,6 +3920,9 @@ static inline void sched_submit_work(struct task_struct *tsk)
preempt_enable_no_resched();
}
+ if (tsk_is_pi_blocked(tsk))
+ return;
+
/*
* If we are going to sleep and we have plugged IO queued,
* make sure to submit it to avoid deadlocks.
diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c
index 23fbbcc..6e52b67 100644
--- a/kernel/sched/psi.c
+++ b/kernel/sched/psi.c
@@ -1131,7 +1131,15 @@ static void psi_trigger_destroy(struct kref *ref)
* deadlock while waiting for psi_poll_work to acquire trigger_lock
*/
if (kworker_to_destroy) {
+ /*
+ * After the RCU grace period has expired, the worker
+ * can no longer be found through group->poll_kworker.
+ * But it might have been already scheduled before
+ * that - deschedule it cleanly before destroying it.
+ */
kthread_cancel_delayed_work_sync(&group->poll_work);
+ atomic_set(&group->poll_scheduled, 0);
+
kthread_destroy_worker(kworker_to_destroy);
}
kfree(t);
diff --git a/kernel/signal.c b/kernel/signal.c
index e667be6..534fec2 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -90,6 +90,11 @@ static bool sig_task_ignored(struct task_struct *t, int sig, bool force)
handler == SIG_DFL && !(force && sig_kernel_only(sig)))
return true;
+ /* Only allow kernel generated signals to this kthread */
+ if (unlikely((t->flags & PF_KTHREAD) &&
+ (handler == SIG_KTHREAD_KERNEL) && !force))
+ return true;
+
return sig_handler_ignored(handler, sig);
}
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index d911c84..ca69290 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -146,6 +146,11 @@ static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec64 wtm)
static inline void tk_update_sleep_time(struct timekeeper *tk, ktime_t delta)
{
tk->offs_boot = ktime_add(tk->offs_boot, delta);
+ /*
+ * Timespec representation for VDSO update to avoid 64bit division
+ * on every update.
+ */
+ tk->monotonic_to_boot = ktime_to_timespec64(tk->offs_boot);
}
/*
diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c
index 8cf3596a..4bc37ac 100644
--- a/kernel/time/vsyscall.c
+++ b/kernel/time/vsyscall.c
@@ -17,7 +17,7 @@ static inline void update_vdso_data(struct vdso_data *vdata,
struct timekeeper *tk)
{
struct vdso_timestamp *vdso_ts;
- u64 nsec;
+ u64 nsec, sec;
vdata[CS_HRES_COARSE].cycle_last = tk->tkr_mono.cycle_last;
vdata[CS_HRES_COARSE].mask = tk->tkr_mono.mask;
@@ -45,23 +45,27 @@ static inline void update_vdso_data(struct vdso_data *vdata,
}
vdso_ts->nsec = nsec;
- /* CLOCK_MONOTONIC_RAW */
- vdso_ts = &vdata[CS_RAW].basetime[CLOCK_MONOTONIC_RAW];
- vdso_ts->sec = tk->raw_sec;
- vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* Copy MONOTONIC time for BOOTTIME */
+ sec = vdso_ts->sec;
+ /* Add the boot offset */
+ sec += tk->monotonic_to_boot.tv_sec;
+ nsec += (u64)tk->monotonic_to_boot.tv_nsec << tk->tkr_mono.shift;
/* CLOCK_BOOTTIME */
vdso_ts = &vdata[CS_HRES_COARSE].basetime[CLOCK_BOOTTIME];
- vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec;
- nsec += ((u64)(tk->wall_to_monotonic.tv_nsec +
- ktime_to_ns(tk->offs_boot)) << tk->tkr_mono.shift);
+ vdso_ts->sec = sec;
+
while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
nsec -= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
vdso_ts->sec++;
}
vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdata[CS_RAW].basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+
/* CLOCK_TAI */
vdso_ts = &vdata[CS_HRES_COARSE].basetime[CLOCK_TAI];
vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 98da899..b09d7b1 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -520,7 +520,8 @@
bool
default y
help
- This allows the user to attach BPF programs to kprobe events.
+ This allows the user to attach BPF programs to kprobe, uprobe, and
+ tracepoint events.
config DYNAMIC_EVENTS
def_bool n
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index eca3450..f9821a3 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -3095,6 +3095,14 @@ t_probe_next(struct seq_file *m, loff_t *pos)
hnd = &iter->probe_entry->hlist;
hash = iter->probe->ops.func_hash->filter_hash;
+
+ /*
+ * A probe being registered may temporarily have an empty hash
+ * and it's at the end of the func_probes list.
+ */
+ if (!hash || hash == EMPTY_HASH)
+ return NULL;
+
size = 1 << hash->size_bits;
retry:
@@ -4320,12 +4328,21 @@ register_ftrace_function_probe(char *glob, struct trace_array *tr,
mutex_unlock(&ftrace_lock);
+ /*
+ * Note, there's a small window here that the func_hash->filter_hash
+ * may be NULL or empty. Need to be carefule when reading the loop.
+ */
mutex_lock(&probe->ops.func_hash->regex_lock);
orig_hash = &probe->ops.func_hash->filter_hash;
old_hash = *orig_hash;
hash = alloc_and_copy_ftrace_hash(FTRACE_HASH_DEFAULT_BITS, old_hash);
+ if (!hash) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
ret = ftrace_match_records(hash, glob, strlen(glob));
/* Nothing found? */
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 525a97f..563e80f 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -1567,9 +1567,9 @@ update_max_tr(struct trace_array *tr, struct task_struct *tsk, int cpu,
/**
* update_max_tr_single - only copy one trace over, and reset the rest
- * @tr - tracer
- * @tsk - task with the latency
- * @cpu - the cpu of the buffer to copy.
+ * @tr: tracer
+ * @tsk: task with the latency
+ * @cpu: the cpu of the buffer to copy.
*
* Flip the trace of a single CPU buffer between the @tr and the max_tr.
*/
@@ -1767,7 +1767,7 @@ static void __init apply_trace_boot_options(void);
/**
* register_tracer - register a tracer with the ftrace system.
- * @type - the plugin for the tracer
+ * @type: the plugin for the tracer
*
* Register a new plugin tracer.
*/
@@ -2230,9 +2230,9 @@ static bool tracing_record_taskinfo_skip(int flags)
/**
* tracing_record_taskinfo - record the task info of a task
*
- * @task - task to record
- * @flags - TRACE_RECORD_CMDLINE for recording comm
- * - TRACE_RECORD_TGID for recording tgid
+ * @task: task to record
+ * @flags: TRACE_RECORD_CMDLINE for recording comm
+ * TRACE_RECORD_TGID for recording tgid
*/
void tracing_record_taskinfo(struct task_struct *task, int flags)
{
@@ -2258,10 +2258,10 @@ void tracing_record_taskinfo(struct task_struct *task, int flags)
/**
* tracing_record_taskinfo_sched_switch - record task info for sched_switch
*
- * @prev - previous task during sched_switch
- * @next - next task during sched_switch
- * @flags - TRACE_RECORD_CMDLINE for recording comm
- * TRACE_RECORD_TGID for recording tgid
+ * @prev: previous task during sched_switch
+ * @next: next task during sched_switch
+ * @flags: TRACE_RECORD_CMDLINE for recording comm
+ * TRACE_RECORD_TGID for recording tgid
*/
void tracing_record_taskinfo_sched_switch(struct task_struct *prev,
struct task_struct *next, int flags)
@@ -3072,7 +3072,9 @@ static void trace_printk_start_stop_comm(int enabled)
/**
* trace_vbprintk - write binary msg to tracing buffer
- *
+ * @ip: The address of the caller
+ * @fmt: The string format to write to the buffer
+ * @args: Arguments for @fmt
*/
int trace_vbprintk(unsigned long ip, const char *fmt, va_list args)
{
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index c7506bc..6489308 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -787,7 +787,7 @@ static int __ftrace_set_clr_event(struct trace_array *tr, const char *match,
return ret;
}
-static int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
+int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
{
char *event = NULL, *sub = NULL, *match;
int ret;
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index dbef0d1..fb6bfbc 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -895,7 +895,8 @@ void trace_probe_cleanup(struct trace_probe *tp)
for (i = 0; i < tp->nr_args; i++)
traceprobe_free_probe_arg(&tp->args[i]);
- kfree(call->class->system);
+ if (call->class)
+ kfree(call->class->system);
kfree(call->name);
kfree(call->print_fmt);
}
diff --git a/lib/kfifo.c b/lib/kfifo.c
index 117ad0e..70dab9ac 100644
--- a/lib/kfifo.c
+++ b/lib/kfifo.c
@@ -68,7 +68,8 @@ int __kfifo_init(struct __kfifo *fifo, void *buffer,
{
size /= esize;
- size = roundup_pow_of_two(size);
+ if (!is_power_of_2(size))
+ size = rounddown_pow_of_two(size);
fifo->in = 0;
fifo->out = 0;
diff --git a/lib/logic_pio.c b/lib/logic_pio.c
index feea48f..9050275 100644
--- a/lib/logic_pio.c
+++ b/lib/logic_pio.c
@@ -35,7 +35,7 @@ int logic_pio_register_range(struct logic_pio_hwaddr *new_range)
struct logic_pio_hwaddr *range;
resource_size_t start;
resource_size_t end;
- resource_size_t mmio_sz = 0;
+ resource_size_t mmio_end = 0;
resource_size_t iio_sz = MMIO_UPPER_LIMIT;
int ret = 0;
@@ -46,7 +46,7 @@ int logic_pio_register_range(struct logic_pio_hwaddr *new_range)
end = new_range->hw_start + new_range->size;
mutex_lock(&io_range_mutex);
- list_for_each_entry_rcu(range, &io_range_list, list) {
+ list_for_each_entry(range, &io_range_list, list) {
if (range->fwnode == new_range->fwnode) {
/* range already there */
goto end_register;
@@ -56,7 +56,7 @@ int logic_pio_register_range(struct logic_pio_hwaddr *new_range)
/* for MMIO ranges we need to check for overlap */
if (start >= range->hw_start + range->size ||
end < range->hw_start) {
- mmio_sz += range->size;
+ mmio_end = range->io_start + range->size;
} else {
ret = -EFAULT;
goto end_register;
@@ -69,16 +69,16 @@ int logic_pio_register_range(struct logic_pio_hwaddr *new_range)
/* range not registered yet, check for available space */
if (new_range->flags == LOGIC_PIO_CPU_MMIO) {
- if (mmio_sz + new_range->size - 1 > MMIO_UPPER_LIMIT) {
+ if (mmio_end + new_range->size - 1 > MMIO_UPPER_LIMIT) {
/* if it's too big check if 64K space can be reserved */
- if (mmio_sz + SZ_64K - 1 > MMIO_UPPER_LIMIT) {
+ if (mmio_end + SZ_64K - 1 > MMIO_UPPER_LIMIT) {
ret = -E2BIG;
goto end_register;
}
new_range->size = SZ_64K;
pr_warn("Requested IO range too big, new size set to 64K\n");
}
- new_range->io_start = mmio_sz;
+ new_range->io_start = mmio_end;
} else if (new_range->flags == LOGIC_PIO_INDIRECT) {
if (iio_sz + new_range->size - 1 > IO_SPACE_LIMIT) {
ret = -E2BIG;
@@ -99,6 +99,20 @@ int logic_pio_register_range(struct logic_pio_hwaddr *new_range)
}
/**
+ * logic_pio_unregister_range - unregister a logical PIO range for a host
+ * @range: pointer to the IO range which has been already registered.
+ *
+ * Unregister a previously-registered IO range node.
+ */
+void logic_pio_unregister_range(struct logic_pio_hwaddr *range)
+{
+ mutex_lock(&io_range_mutex);
+ list_del_rcu(&range->list);
+ mutex_unlock(&io_range_mutex);
+ synchronize_rcu();
+}
+
+/**
* find_io_range_by_fwnode - find logical PIO range for given FW node
* @fwnode: FW node handle associated with logical PIO range
*
@@ -108,26 +122,38 @@ int logic_pio_register_range(struct logic_pio_hwaddr *new_range)
*/
struct logic_pio_hwaddr *find_io_range_by_fwnode(struct fwnode_handle *fwnode)
{
- struct logic_pio_hwaddr *range;
+ struct logic_pio_hwaddr *range, *found_range = NULL;
+ rcu_read_lock();
list_for_each_entry_rcu(range, &io_range_list, list) {
- if (range->fwnode == fwnode)
- return range;
+ if (range->fwnode == fwnode) {
+ found_range = range;
+ break;
+ }
}
- return NULL;
+ rcu_read_unlock();
+
+ return found_range;
}
/* Return a registered range given an input PIO token */
static struct logic_pio_hwaddr *find_io_range(unsigned long pio)
{
- struct logic_pio_hwaddr *range;
+ struct logic_pio_hwaddr *range, *found_range = NULL;
+ rcu_read_lock();
list_for_each_entry_rcu(range, &io_range_list, list) {
- if (in_range(pio, range->io_start, range->size))
- return range;
+ if (in_range(pio, range->io_start, range->size)) {
+ found_range = range;
+ break;
+ }
}
- pr_err("PIO entry token %lx invalid\n", pio);
- return NULL;
+ rcu_read_unlock();
+
+ if (!found_range)
+ pr_err("PIO entry token 0x%lx invalid\n", pio);
+
+ return found_range;
}
/**
@@ -180,14 +206,23 @@ unsigned long logic_pio_trans_cpuaddr(resource_size_t addr)
{
struct logic_pio_hwaddr *range;
+ rcu_read_lock();
list_for_each_entry_rcu(range, &io_range_list, list) {
if (range->flags != LOGIC_PIO_CPU_MMIO)
continue;
- if (in_range(addr, range->hw_start, range->size))
- return addr - range->hw_start + range->io_start;
+ if (in_range(addr, range->hw_start, range->size)) {
+ unsigned long cpuaddr;
+
+ cpuaddr = addr - range->hw_start + range->io_start;
+
+ rcu_read_unlock();
+ return cpuaddr;
+ }
}
- pr_err("addr %llx not registered in io_range_list\n",
- (unsigned long long) addr);
+ rcu_read_unlock();
+
+ pr_err("addr %pa not registered in io_range_list\n", &addr);
+
return ~0UL;
}
diff --git a/lib/test_bpf.c b/lib/test_bpf.c
index c41705835..5ef3ecc 100644
--- a/lib/test_bpf.c
+++ b/lib/test_bpf.c
@@ -867,7 +867,7 @@ static struct bpf_test tests[] = {
},
CLASSIC,
{ },
- { { 4, 10 ^ 300 }, { 20, 10 ^ 300 } },
+ { { 4, 0xA ^ 300 }, { 20, 0xA ^ 300 } },
},
{
"SPILL_FILL",
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 738065f..de1f159 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -32,6 +32,7 @@
#include <linux/shmem_fs.h>
#include <linux/oom.h>
#include <linux/numa.h>
+#include <linux/page_owner.h>
#include <asm/tlb.h>
#include <asm/pgalloc.h>
@@ -2516,6 +2517,9 @@ static void __split_huge_page(struct page *page, struct list_head *list,
}
ClearPageCompound(head);
+
+ split_page_owner(head, HPAGE_PMD_ORDER);
+
/* See comment in __split_huge_page_tail() */
if (PageAnon(head)) {
/* Additional pin to swap cache */
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 2277b82..95d16a4 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -407,8 +407,14 @@ static inline bool shadow_invalid(u8 tag, s8 shadow_byte)
if (IS_ENABLED(CONFIG_KASAN_GENERIC))
return shadow_byte < 0 ||
shadow_byte >= KASAN_SHADOW_SCALE_SIZE;
- else
- return tag != (u8)shadow_byte;
+
+ /* else CONFIG_KASAN_SW_TAGS: */
+ if ((u8)shadow_byte == KASAN_TAG_INVALID)
+ return true;
+ if ((tag != KASAN_TAG_KERNEL) && (tag != (u8)shadow_byte))
+ return true;
+
+ return false;
}
static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6f5c0c5..9ec5e12 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -752,15 +752,13 @@ void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx,
/* Update memcg */
__mod_memcg_state(memcg, idx, val);
+ /* Update lruvec */
+ __this_cpu_add(pn->lruvec_stat_local->count[idx], val);
+
x = val + __this_cpu_read(pn->lruvec_stat_cpu->count[idx]);
if (unlikely(abs(x) > MEMCG_CHARGE_BATCH)) {
struct mem_cgroup_per_node *pi;
- /*
- * Batch local counters to keep them in sync with
- * the hierarchical ones.
- */
- __this_cpu_add(pn->lruvec_stat_local->count[idx], x);
for (pi = pn; pi; pi = parent_nodeinfo(pi, pgdat->node_id))
atomic_long_add(x, &pi->lruvec_stat[idx]);
x = 0;
@@ -3260,6 +3258,72 @@ static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,
}
}
+static void memcg_flush_percpu_vmstats(struct mem_cgroup *memcg, bool slab_only)
+{
+ unsigned long stat[MEMCG_NR_STAT];
+ struct mem_cgroup *mi;
+ int node, cpu, i;
+ int min_idx, max_idx;
+
+ if (slab_only) {
+ min_idx = NR_SLAB_RECLAIMABLE;
+ max_idx = NR_SLAB_UNRECLAIMABLE;
+ } else {
+ min_idx = 0;
+ max_idx = MEMCG_NR_STAT;
+ }
+
+ for (i = min_idx; i < max_idx; i++)
+ stat[i] = 0;
+
+ for_each_online_cpu(cpu)
+ for (i = min_idx; i < max_idx; i++)
+ stat[i] += per_cpu(memcg->vmstats_percpu->stat[i], cpu);
+
+ for (mi = memcg; mi; mi = parent_mem_cgroup(mi))
+ for (i = min_idx; i < max_idx; i++)
+ atomic_long_add(stat[i], &mi->vmstats[i]);
+
+ if (!slab_only)
+ max_idx = NR_VM_NODE_STAT_ITEMS;
+
+ for_each_node(node) {
+ struct mem_cgroup_per_node *pn = memcg->nodeinfo[node];
+ struct mem_cgroup_per_node *pi;
+
+ for (i = min_idx; i < max_idx; i++)
+ stat[i] = 0;
+
+ for_each_online_cpu(cpu)
+ for (i = min_idx; i < max_idx; i++)
+ stat[i] += per_cpu(
+ pn->lruvec_stat_cpu->count[i], cpu);
+
+ for (pi = pn; pi; pi = parent_nodeinfo(pi, node))
+ for (i = min_idx; i < max_idx; i++)
+ atomic_long_add(stat[i], &pi->lruvec_stat[i]);
+ }
+}
+
+static void memcg_flush_percpu_vmevents(struct mem_cgroup *memcg)
+{
+ unsigned long events[NR_VM_EVENT_ITEMS];
+ struct mem_cgroup *mi;
+ int cpu, i;
+
+ for (i = 0; i < NR_VM_EVENT_ITEMS; i++)
+ events[i] = 0;
+
+ for_each_online_cpu(cpu)
+ for (i = 0; i < NR_VM_EVENT_ITEMS; i++)
+ events[i] += per_cpu(memcg->vmstats_percpu->events[i],
+ cpu);
+
+ for (mi = memcg; mi; mi = parent_mem_cgroup(mi))
+ for (i = 0; i < NR_VM_EVENT_ITEMS; i++)
+ atomic_long_add(events[i], &mi->vmevents[i]);
+}
+
#ifdef CONFIG_MEMCG_KMEM
static int memcg_online_kmem(struct mem_cgroup *memcg)
{
@@ -3309,7 +3373,14 @@ static void memcg_offline_kmem(struct mem_cgroup *memcg)
if (!parent)
parent = root_mem_cgroup;
+ /*
+ * Deactivate and reparent kmem_caches. Then flush percpu
+ * slab statistics to have precise values at the parent and
+ * all ancestor levels. It's required to keep slab stats
+ * accurate after the reparenting of kmem_caches.
+ */
memcg_deactivate_kmem_caches(memcg, parent);
+ memcg_flush_percpu_vmstats(memcg, true);
kmemcg_id = memcg->kmemcg_id;
BUG_ON(kmemcg_id < 0);
@@ -4682,6 +4753,12 @@ static void __mem_cgroup_free(struct mem_cgroup *memcg)
{
int node;
+ /*
+ * Flush percpu vmstats and vmevents to guarantee the value correctness
+ * on parent's and all ancestor levels.
+ */
+ memcg_flush_percpu_vmstats(memcg, false);
+ memcg_flush_percpu_vmevents(memcg);
for_each_node(node)
free_mem_cgroup_per_node_info(memcg, node);
free_percpu(memcg->vmstats_percpu);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 272c6de..9c91949 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2238,27 +2238,12 @@ static int move_freepages(struct zone *zone,
unsigned int order;
int pages_moved = 0;
-#ifndef CONFIG_HOLES_IN_ZONE
- /*
- * page_zone is not safe to call in this context when
- * CONFIG_HOLES_IN_ZONE is set. This bug check is probably redundant
- * anyway as we check zone boundaries in move_freepages_block().
- * Remove at a later date when no bug reports exist related to
- * grouping pages by mobility
- */
- VM_BUG_ON(pfn_valid(page_to_pfn(start_page)) &&
- pfn_valid(page_to_pfn(end_page)) &&
- page_zone(start_page) != page_zone(end_page));
-#endif
for (page = start_page; page <= end_page;) {
if (!pfn_valid_within(page_to_pfn(page))) {
page++;
continue;
}
- /* Make sure we are not inadvertently changing nodes */
- VM_BUG_ON_PAGE(page_to_nid(page) != zone_to_nid(zone), page);
-
if (!PageBuddy(page)) {
/*
* We assume that pages that could be isolated for
@@ -2273,6 +2258,10 @@ static int move_freepages(struct zone *zone,
continue;
}
+ /* Make sure we are not inadvertently changing nodes */
+ VM_BUG_ON_PAGE(page_to_nid(page) != zone_to_nid(zone), page);
+ VM_BUG_ON_PAGE(page_zone(page) != zone, page);
+
order = page_order(page);
move_to_free_area(page, &zone->free_area[order], migratetype);
page += 1 << order;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index c77d1e3..a6c5d0b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3220,6 +3220,7 @@ unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
#ifdef CONFIG_MEMCG
+/* Only used by soft limit reclaim. Do not reuse for anything else. */
unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg,
gfp_t gfp_mask, bool noswap,
pg_data_t *pgdat,
@@ -3235,7 +3236,8 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg,
};
unsigned long lru_pages;
- set_task_reclaim_state(current, &sc.reclaim_state);
+ WARN_ON_ONCE(!current->reclaim_state);
+
sc.gfp_mask = (gfp_mask & GFP_RECLAIM_MASK) |
(GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK);
@@ -3253,7 +3255,6 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg,
trace_mm_vmscan_memcg_softlimit_reclaim_end(sc.nr_reclaimed);
- set_task_reclaim_state(current, NULL);
*nr_scanned = sc.nr_scanned;
return sc.nr_reclaimed;
diff --git a/mm/z3fold.c b/mm/z3fold.c
index ed19d98..75b7962 100644
--- a/mm/z3fold.c
+++ b/mm/z3fold.c
@@ -41,6 +41,7 @@
#include <linux/workqueue.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
+#include <linux/wait.h>
#include <linux/zpool.h>
#include <linux/magic.h>
@@ -145,6 +146,8 @@ struct z3fold_header {
* @release_wq: workqueue for safe page release
* @work: work_struct for safe page release
* @inode: inode for z3fold pseudo filesystem
+ * @destroying: bool to stop migration once we start destruction
+ * @isolated: int to count the number of pages currently in isolation
*
* This structure is allocated at pool creation time and maintains metadata
* pertaining to a particular z3fold pool.
@@ -163,8 +166,11 @@ struct z3fold_pool {
const struct zpool_ops *zpool_ops;
struct workqueue_struct *compact_wq;
struct workqueue_struct *release_wq;
+ struct wait_queue_head isolate_wait;
struct work_struct work;
struct inode *inode;
+ bool destroying;
+ int isolated;
};
/*
@@ -769,6 +775,7 @@ static struct z3fold_pool *z3fold_create_pool(const char *name, gfp_t gfp,
goto out_c;
spin_lock_init(&pool->lock);
spin_lock_init(&pool->stale_lock);
+ init_waitqueue_head(&pool->isolate_wait);
pool->unbuddied = __alloc_percpu(sizeof(struct list_head)*NCHUNKS, 2);
if (!pool->unbuddied)
goto out_pool;
@@ -808,6 +815,15 @@ static struct z3fold_pool *z3fold_create_pool(const char *name, gfp_t gfp,
return NULL;
}
+static bool pool_isolated_are_drained(struct z3fold_pool *pool)
+{
+ bool ret;
+
+ spin_lock(&pool->lock);
+ ret = pool->isolated == 0;
+ spin_unlock(&pool->lock);
+ return ret;
+}
/**
* z3fold_destroy_pool() - destroys an existing z3fold pool
* @pool: the z3fold pool to be destroyed
@@ -817,6 +833,22 @@ static struct z3fold_pool *z3fold_create_pool(const char *name, gfp_t gfp,
static void z3fold_destroy_pool(struct z3fold_pool *pool)
{
kmem_cache_destroy(pool->c_handle);
+ /*
+ * We set pool-> destroying under lock to ensure that
+ * z3fold_page_isolate() sees any changes to destroying. This way we
+ * avoid the need for any memory barriers.
+ */
+
+ spin_lock(&pool->lock);
+ pool->destroying = true;
+ spin_unlock(&pool->lock);
+
+ /*
+ * We need to ensure that no pages are being migrated while we destroy
+ * these workqueues, as migration can queue work on either of the
+ * workqueues.
+ */
+ wait_event(pool->isolate_wait, !pool_isolated_are_drained(pool));
/*
* We need to destroy pool->compact_wq before pool->release_wq,
@@ -1307,6 +1339,28 @@ static u64 z3fold_get_pool_size(struct z3fold_pool *pool)
return atomic64_read(&pool->pages_nr);
}
+/*
+ * z3fold_dec_isolated() expects to be called while pool->lock is held.
+ */
+static void z3fold_dec_isolated(struct z3fold_pool *pool)
+{
+ assert_spin_locked(&pool->lock);
+ VM_BUG_ON(pool->isolated <= 0);
+ pool->isolated--;
+
+ /*
+ * If we have no more isolated pages, we have to see if
+ * z3fold_destroy_pool() is waiting for a signal.
+ */
+ if (pool->isolated == 0 && waitqueue_active(&pool->isolate_wait))
+ wake_up_all(&pool->isolate_wait);
+}
+
+static void z3fold_inc_isolated(struct z3fold_pool *pool)
+{
+ pool->isolated++;
+}
+
static bool z3fold_page_isolate(struct page *page, isolate_mode_t mode)
{
struct z3fold_header *zhdr;
@@ -1333,6 +1387,34 @@ static bool z3fold_page_isolate(struct page *page, isolate_mode_t mode)
spin_lock(&pool->lock);
if (!list_empty(&page->lru))
list_del(&page->lru);
+ /*
+ * We need to check for destruction while holding pool->lock, as
+ * otherwise destruction could see 0 isolated pages, and
+ * proceed.
+ */
+ if (unlikely(pool->destroying)) {
+ spin_unlock(&pool->lock);
+ /*
+ * If this page isn't stale, somebody else holds a
+ * reference to it. Let't drop our refcount so that they
+ * can call the release logic.
+ */
+ if (unlikely(kref_put(&zhdr->refcount,
+ release_z3fold_page_locked))) {
+ /*
+ * If we get here we have kref problems, so we
+ * should freak out.
+ */
+ WARN(1, "Z3fold is experiencing kref problems\n");
+ z3fold_page_unlock(zhdr);
+ return false;
+ }
+ z3fold_page_unlock(zhdr);
+ return false;
+ }
+
+
+ z3fold_inc_isolated(pool);
spin_unlock(&pool->lock);
z3fold_page_unlock(zhdr);
return true;
@@ -1401,6 +1483,10 @@ static int z3fold_page_migrate(struct address_space *mapping, struct page *newpa
queue_work_on(new_zhdr->cpu, pool->compact_wq, &new_zhdr->work);
+ spin_lock(&pool->lock);
+ z3fold_dec_isolated(pool);
+ spin_unlock(&pool->lock);
+
page_mapcount_reset(page);
put_page(page);
return 0;
@@ -1420,10 +1506,14 @@ static void z3fold_page_putback(struct page *page)
INIT_LIST_HEAD(&page->lru);
if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
atomic64_dec(&pool->pages_nr);
+ spin_lock(&pool->lock);
+ z3fold_dec_isolated(pool);
+ spin_unlock(&pool->lock);
return;
}
spin_lock(&pool->lock);
list_add(&page->lru, &pool->lru);
+ z3fold_dec_isolated(pool);
spin_unlock(&pool->lock);
z3fold_page_unlock(zhdr);
}
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 57fbb7c..e98bb6a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -54,6 +54,7 @@
#include <linux/mount.h>
#include <linux/pseudo_fs.h>
#include <linux/migrate.h>
+#include <linux/wait.h>
#include <linux/pagemap.h>
#include <linux/fs.h>
@@ -268,6 +269,10 @@ struct zs_pool {
#ifdef CONFIG_COMPACTION
struct inode *inode;
struct work_struct free_work;
+ /* A wait queue for when migration races with async_free_zspage() */
+ struct wait_queue_head migration_wait;
+ atomic_long_t isolated_pages;
+ bool destroying;
#endif
};
@@ -1862,6 +1867,31 @@ static void dec_zspage_isolation(struct zspage *zspage)
zspage->isolated--;
}
+static void putback_zspage_deferred(struct zs_pool *pool,
+ struct size_class *class,
+ struct zspage *zspage)
+{
+ enum fullness_group fg;
+
+ fg = putback_zspage(class, zspage);
+ if (fg == ZS_EMPTY)
+ schedule_work(&pool->free_work);
+
+}
+
+static inline void zs_pool_dec_isolated(struct zs_pool *pool)
+{
+ VM_BUG_ON(atomic_long_read(&pool->isolated_pages) <= 0);
+ atomic_long_dec(&pool->isolated_pages);
+ /*
+ * There's no possibility of racing, since wait_for_isolated_drain()
+ * checks the isolated count under &class->lock after enqueuing
+ * on migration_wait.
+ */
+ if (atomic_long_read(&pool->isolated_pages) == 0 && pool->destroying)
+ wake_up_all(&pool->migration_wait);
+}
+
static void replace_sub_page(struct size_class *class, struct zspage *zspage,
struct page *newpage, struct page *oldpage)
{
@@ -1931,6 +1961,7 @@ static bool zs_page_isolate(struct page *page, isolate_mode_t mode)
*/
if (!list_empty(&zspage->list) && !is_zspage_isolated(zspage)) {
get_zspage_mapping(zspage, &class_idx, &fullness);
+ atomic_long_inc(&pool->isolated_pages);
remove_zspage(class, zspage, fullness);
}
@@ -2030,8 +2061,16 @@ static int zs_page_migrate(struct address_space *mapping, struct page *newpage,
* Page migration is done so let's putback isolated zspage to
* the list if @page is final isolated subpage in the zspage.
*/
- if (!is_zspage_isolated(zspage))
- putback_zspage(class, zspage);
+ if (!is_zspage_isolated(zspage)) {
+ /*
+ * We cannot race with zs_destroy_pool() here because we wait
+ * for isolation to hit zero before we start destroying.
+ * Also, we ensure that everyone can see pool->destroying before
+ * we start waiting.
+ */
+ putback_zspage_deferred(pool, class, zspage);
+ zs_pool_dec_isolated(pool);
+ }
reset_page(page);
put_page(page);
@@ -2077,13 +2116,12 @@ static void zs_page_putback(struct page *page)
spin_lock(&class->lock);
dec_zspage_isolation(zspage);
if (!is_zspage_isolated(zspage)) {
- fg = putback_zspage(class, zspage);
/*
* Due to page_lock, we cannot free zspage immediately
* so let's defer.
*/
- if (fg == ZS_EMPTY)
- schedule_work(&pool->free_work);
+ putback_zspage_deferred(pool, class, zspage);
+ zs_pool_dec_isolated(pool);
}
spin_unlock(&class->lock);
}
@@ -2107,8 +2145,36 @@ static int zs_register_migration(struct zs_pool *pool)
return 0;
}
+static bool pool_isolated_are_drained(struct zs_pool *pool)
+{
+ return atomic_long_read(&pool->isolated_pages) == 0;
+}
+
+/* Function for resolving migration */
+static void wait_for_isolated_drain(struct zs_pool *pool)
+{
+
+ /*
+ * We're in the process of destroying the pool, so there are no
+ * active allocations. zs_page_isolate() fails for completely free
+ * zspages, so we need only wait for the zs_pool's isolated
+ * count to hit zero.
+ */
+ wait_event(pool->migration_wait,
+ pool_isolated_are_drained(pool));
+}
+
static void zs_unregister_migration(struct zs_pool *pool)
{
+ pool->destroying = true;
+ /*
+ * We need a memory barrier here to ensure global visibility of
+ * pool->destroying. Thus pool->isolated pages will either be 0 in which
+ * case we don't care, or it will be > 0 and pool->destroying will
+ * ensure that we wake up once isolation hits 0.
+ */
+ smp_mb();
+ wait_for_isolated_drain(pool); /* This can block */
flush_work(&pool->free_work);
iput(pool->inode);
}
@@ -2346,6 +2412,10 @@ struct zs_pool *zs_create_pool(const char *name)
if (!pool->name)
goto err;
+#ifdef CONFIG_COMPACTION
+ init_waitqueue_head(&pool->migration_wait);
+#endif
+
if (create_cache(pool))
goto err;
diff --git a/net/atm/mpoa_caches.c b/net/atm/mpoa_caches.c
index 4bb4183..3286f9d 100644
--- a/net/atm/mpoa_caches.c
+++ b/net/atm/mpoa_caches.c
@@ -180,8 +180,7 @@ static int cache_hit(in_cache_entry *entry, struct mpoa_client *mpc)
static void in_cache_put(in_cache_entry *entry)
{
if (refcount_dec_and_test(&entry->use)) {
- memset(entry, 0, sizeof(in_cache_entry));
- kfree(entry);
+ kzfree(entry);
}
}
@@ -416,8 +415,7 @@ static eg_cache_entry *eg_cache_get_by_src_ip(__be32 ipaddr,
static void eg_cache_put(eg_cache_entry *entry)
{
if (refcount_dec_and_test(&entry->use)) {
- memset(entry, 0, sizeof(eg_cache_entry));
- kfree(entry);
+ kzfree(entry);
}
}
diff --git a/net/atm/pppoatm.c b/net/atm/pppoatm.c
index bd3da9a..45d8e1d 100644
--- a/net/atm/pppoatm.c
+++ b/net/atm/pppoatm.c
@@ -216,9 +216,7 @@ static void pppoatm_push(struct atm_vcc *atmvcc, struct sk_buff *skb)
pvcc->chan.mtu += LLC_LEN;
break;
}
- pr_debug("Couldn't autodetect yet (skb: %02X %02X %02X %02X %02X %02X)\n",
- skb->data[0], skb->data[1], skb->data[2],
- skb->data[3], skb->data[4], skb->data[5]);
+ pr_debug("Couldn't autodetect yet (skb: %6ph)\n", skb->data);
goto error;
case e_vc:
break;
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index 240ed70..d78938e 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -277,17 +277,23 @@ static u8 batadv_hop_penalty(u8 tq, const struct batadv_priv *bat_priv)
* batadv_iv_ogm_aggr_packet() - checks if there is another OGM attached
* @buff_pos: current position in the skb
* @packet_len: total length of the skb
- * @tvlv_len: tvlv length of the previously considered OGM
+ * @ogm_packet: potential OGM in buffer
*
* Return: true if there is enough space for another OGM, false otherwise.
*/
-static bool batadv_iv_ogm_aggr_packet(int buff_pos, int packet_len,
- __be16 tvlv_len)
+static bool
+batadv_iv_ogm_aggr_packet(int buff_pos, int packet_len,
+ const struct batadv_ogm_packet *ogm_packet)
{
int next_buff_pos = 0;
- next_buff_pos += buff_pos + BATADV_OGM_HLEN;
- next_buff_pos += ntohs(tvlv_len);
+ /* check if there is enough space for the header */
+ next_buff_pos += buff_pos + sizeof(*ogm_packet);
+ if (next_buff_pos > packet_len)
+ return false;
+
+ /* check if there is enough space for the optional TVLV */
+ next_buff_pos += ntohs(ogm_packet->tvlv_len);
return (next_buff_pos <= packet_len) &&
(next_buff_pos <= BATADV_MAX_AGGREGATION_BYTES);
@@ -315,7 +321,7 @@ static void batadv_iv_ogm_send_to_if(struct batadv_forw_packet *forw_packet,
/* adjust all flags and log packets */
while (batadv_iv_ogm_aggr_packet(buff_pos, forw_packet->packet_len,
- batadv_ogm_packet->tvlv_len)) {
+ batadv_ogm_packet)) {
/* we might have aggregated direct link packets with an
* ordinary base packet
*/
@@ -1704,7 +1710,7 @@ static int batadv_iv_ogm_receive(struct sk_buff *skb,
/* unpack the aggregated packets and process them one by one */
while (batadv_iv_ogm_aggr_packet(ogm_offset, skb_headlen(skb),
- ogm_packet->tvlv_len)) {
+ ogm_packet)) {
batadv_iv_ogm_process(skb, ogm_offset, if_incoming);
ogm_offset += BATADV_OGM_HLEN;
diff --git a/net/batman-adv/bat_v_ogm.c b/net/batman-adv/bat_v_ogm.c
index 319249f..dc4f743 100644
--- a/net/batman-adv/bat_v_ogm.c
+++ b/net/batman-adv/bat_v_ogm.c
@@ -806,17 +806,23 @@ batadv_v_ogm_process_per_outif(struct batadv_priv *bat_priv,
* batadv_v_ogm_aggr_packet() - checks if there is another OGM aggregated
* @buff_pos: current position in the skb
* @packet_len: total length of the skb
- * @tvlv_len: tvlv length of the previously considered OGM
+ * @ogm2_packet: potential OGM2 in buffer
*
* Return: true if there is enough space for another OGM, false otherwise.
*/
-static bool batadv_v_ogm_aggr_packet(int buff_pos, int packet_len,
- __be16 tvlv_len)
+static bool
+batadv_v_ogm_aggr_packet(int buff_pos, int packet_len,
+ const struct batadv_ogm2_packet *ogm2_packet)
{
int next_buff_pos = 0;
- next_buff_pos += buff_pos + BATADV_OGM2_HLEN;
- next_buff_pos += ntohs(tvlv_len);
+ /* check if there is enough space for the header */
+ next_buff_pos += buff_pos + sizeof(*ogm2_packet);
+ if (next_buff_pos > packet_len)
+ return false;
+
+ /* check if there is enough space for the optional TVLV */
+ next_buff_pos += ntohs(ogm2_packet->tvlv_len);
return (next_buff_pos <= packet_len) &&
(next_buff_pos <= BATADV_MAX_AGGREGATION_BYTES);
@@ -993,7 +999,7 @@ int batadv_v_ogm_packet_recv(struct sk_buff *skb,
ogm_packet = (struct batadv_ogm2_packet *)skb->data;
while (batadv_v_ogm_aggr_packet(ogm_offset, skb_headlen(skb),
- ogm_packet->tvlv_len)) {
+ ogm_packet)) {
batadv_v_ogm_process(skb, ogm_offset, if_incoming);
ogm_offset += BATADV_OGM2_HLEN;
diff --git a/net/batman-adv/netlink.c b/net/batman-adv/netlink.c
index 6f08fd1..7e052d6 100644
--- a/net/batman-adv/netlink.c
+++ b/net/batman-adv/netlink.c
@@ -164,7 +164,7 @@ batadv_netlink_get_ifindex(const struct nlmsghdr *nlh, int attrtype)
{
struct nlattr *attr = nlmsg_find_attr(nlh, GENL_HDRLEN, attrtype);
- return attr ? nla_get_u32(attr) : 0;
+ return (attr && nla_len(attr) == sizeof(u32)) ? nla_get_u32(attr) : 0;
}
/**
diff --git a/net/bluetooth/6lowpan.c b/net/bluetooth/6lowpan.c
index 9d41de1..bb55d92 100644
--- a/net/bluetooth/6lowpan.c
+++ b/net/bluetooth/6lowpan.c
@@ -583,7 +583,7 @@ static const struct net_device_ops netdev_ops = {
.ndo_start_xmit = bt_xmit,
};
-static struct header_ops header_ops = {
+static const struct header_ops header_ops = {
.create = header_create,
};
diff --git a/net/bluetooth/hci_request.c b/net/bluetooth/hci_request.c
index 621f1a9..7f6a581 100644
--- a/net/bluetooth/hci_request.c
+++ b/net/bluetooth/hci_request.c
@@ -1054,6 +1054,7 @@ void __hci_req_enable_advertising(struct hci_request *req)
struct hci_cp_le_set_adv_param cp;
u8 own_addr_type, enable = 0x01;
bool connectable;
+ u16 adv_min_interval, adv_max_interval;
u32 flags;
flags = get_adv_instance_flags(hdev, hdev->cur_adv_instance);
@@ -1087,16 +1088,30 @@ void __hci_req_enable_advertising(struct hci_request *req)
return;
memset(&cp, 0, sizeof(cp));
- cp.min_interval = cpu_to_le16(hdev->le_adv_min_interval);
- cp.max_interval = cpu_to_le16(hdev->le_adv_max_interval);
- if (connectable)
+ if (connectable) {
cp.type = LE_ADV_IND;
- else if (get_cur_adv_instance_scan_rsp_len(hdev))
- cp.type = LE_ADV_SCAN_IND;
- else
- cp.type = LE_ADV_NONCONN_IND;
+ adv_min_interval = hdev->le_adv_min_interval;
+ adv_max_interval = hdev->le_adv_max_interval;
+ } else {
+ if (get_cur_adv_instance_scan_rsp_len(hdev))
+ cp.type = LE_ADV_SCAN_IND;
+ else
+ cp.type = LE_ADV_NONCONN_IND;
+
+ if (!hci_dev_test_flag(hdev, HCI_DISCOVERABLE) ||
+ hci_dev_test_flag(hdev, HCI_LIMITED_DISCOVERABLE)) {
+ adv_min_interval = DISCOV_LE_FAST_ADV_INT_MIN;
+ adv_max_interval = DISCOV_LE_FAST_ADV_INT_MAX;
+ } else {
+ adv_min_interval = hdev->le_adv_min_interval;
+ adv_max_interval = hdev->le_adv_max_interval;
+ }
+ }
+
+ cp.min_interval = cpu_to_le16(adv_min_interval);
+ cp.max_interval = cpu_to_le16(adv_max_interval);
cp.own_address_type = own_addr_type;
cp.channel_map = hdev->le_adv_channel_map;
diff --git a/net/bluetooth/hidp/core.c b/net/bluetooth/hidp/core.c
index 8d88996..bef84b9 100644
--- a/net/bluetooth/hidp/core.c
+++ b/net/bluetooth/hidp/core.c
@@ -267,7 +267,7 @@ static int hidp_get_raw_report(struct hid_device *hid,
set_bit(HIDP_WAITING_FOR_RETURN, &session->flags);
data[0] = report_number;
ret = hidp_send_ctrl_message(session, report_type, data, 1);
- if (ret)
+ if (ret < 0)
goto err;
/* Wait for the return of the report. The returned report
@@ -343,7 +343,7 @@ static int hidp_set_raw_report(struct hid_device *hid, unsigned char reportnum,
data[0] = reportnum;
set_bit(HIDP_WAITING_FOR_SEND_ACK, &session->flags);
ret = hidp_send_ctrl_message(session, report_type, data, count);
- if (ret)
+ if (ret < 0)
goto err;
/* Wait for the ACK from the device. */
diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c
index 150114e..acb7c6d 100644
--- a/net/bluetooth/mgmt.c
+++ b/net/bluetooth/mgmt.c
@@ -2588,7 +2588,6 @@ static int get_connections(struct sock *sk, struct hci_dev *hdev, void *data,
{
struct mgmt_rp_get_connections *rp;
struct hci_conn *c;
- size_t rp_len;
int err;
u16 i;
@@ -2608,8 +2607,7 @@ static int get_connections(struct sock *sk, struct hci_dev *hdev, void *data,
i++;
}
- rp_len = sizeof(*rp) + (i * sizeof(struct mgmt_addr_info));
- rp = kmalloc(rp_len, GFP_KERNEL);
+ rp = kmalloc(struct_size(rp, addr, i), GFP_KERNEL);
if (!rp) {
err = -ENOMEM;
goto unlock;
@@ -2629,10 +2627,8 @@ static int get_connections(struct sock *sk, struct hci_dev *hdev, void *data,
rp->conn_count = cpu_to_le16(i);
/* Recalculate length in case of filtered SCO connections, etc */
- rp_len = sizeof(*rp) + (i * sizeof(struct mgmt_addr_info));
-
err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_GET_CONNECTIONS, 0, rp,
- rp_len);
+ struct_size(rp, addr, i));
kfree(rp);
diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c
index f5b2aee..bb98984 100644
--- a/net/bridge/br_vlan.c
+++ b/net/bridge/br_vlan.c
@@ -1281,6 +1281,8 @@ int br_vlan_get_info(const struct net_device *dev, u16 vid,
p_vinfo->vid = vid;
p_vinfo->flags = v->flags;
+ if (vid == br_get_pvid(vg))
+ p_vinfo->flags |= BRIDGE_VLAN_INFO_PVID;
return 0;
}
EXPORT_SYMBOL_GPL(br_vlan_get_info);
diff --git a/net/bridge/netfilter/ebtables.c b/net/bridge/netfilter/ebtables.c
index c8177a8..4096d8a 100644
--- a/net/bridge/netfilter/ebtables.c
+++ b/net/bridge/netfilter/ebtables.c
@@ -221,7 +221,7 @@ unsigned int ebt_do_table(struct sk_buff *skb,
return NF_DROP;
}
- ADD_COUNTER(*(counter_base + i), 1, skb->len);
+ ADD_COUNTER(*(counter_base + i), skb->len, 1);
/* these should only watch: not modify, nor tell us
* what to do with the packet
@@ -959,8 +959,8 @@ static void get_counters(const struct ebt_counter *oldcounters,
continue;
counter_base = COUNTER_BASE(oldcounters, nentries, cpu);
for (i = 0; i < nentries; i++)
- ADD_COUNTER(counters[i], counter_base[i].pcnt,
- counter_base[i].bcnt);
+ ADD_COUNTER(counters[i], counter_base[i].bcnt,
+ counter_base[i].pcnt);
}
}
@@ -1280,7 +1280,7 @@ static int do_update_counters(struct net *net, const char *name,
/* we add to the counters of the first cpu */
for (i = 0; i < num_counters; i++)
- ADD_COUNTER(t->private->counters[i], tmp[i].pcnt, tmp[i].bcnt);
+ ADD_COUNTER(t->private->counters[i], tmp[i].bcnt, tmp[i].pcnt);
write_unlock_bh(&t->lock);
ret = 0;
diff --git a/net/bridge/netfilter/nft_meta_bridge.c b/net/bridge/netfilter/nft_meta_bridge.c
index 1804e86..7c9e92b2 100644
--- a/net/bridge/netfilter/nft_meta_bridge.c
+++ b/net/bridge/netfilter/nft_meta_bridge.c
@@ -53,7 +53,7 @@ static void nft_meta_bridge_get_eval(const struct nft_expr *expr,
goto err;
br_vlan_get_proto(br_dev, &p_proto);
- nft_reg_store16(dest, p_proto);
+ nft_reg_store16(dest, htons(p_proto));
return;
}
default:
diff --git a/net/can/Kconfig b/net/can/Kconfig
index d4319aa..d770427 100644
--- a/net/can/Kconfig
+++ b/net/can/Kconfig
@@ -53,6 +53,8 @@
They can be modified with AND/OR/XOR/SET operations as configured
by the netlink configuration interface known e.g. from iptables.
+source "net/can/j1939/Kconfig"
+
source "drivers/net/can/Kconfig"
endif
diff --git a/net/can/Makefile b/net/can/Makefile
index 1242bbb..08bd217 100644
--- a/net/can/Makefile
+++ b/net/can/Makefile
@@ -15,3 +15,5 @@
obj-$(CONFIG_CAN_GW) += can-gw.o
can-gw-y := gw.o
+
+obj-$(CONFIG_CAN_J1939) += j1939/
diff --git a/net/can/af_can.c b/net/can/af_can.c
index 9a9a518..5518a7d9 100644
--- a/net/can/af_can.c
+++ b/net/can/af_can.c
@@ -58,6 +58,7 @@
#include <linux/can.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
+#include <linux/can/can-ml.h>
#include <linux/ratelimit.h>
#include <net/net_namespace.h>
#include <net/sock.h>
@@ -198,7 +199,7 @@ int can_send(struct sk_buff *skb, int loop)
{
struct sk_buff *newskb = NULL;
struct canfd_frame *cfd = (struct canfd_frame *)skb->data;
- struct s_stats *can_stats = dev_net(skb->dev)->can.can_stats;
+ struct can_pkg_stats *pkg_stats = dev_net(skb->dev)->can.pkg_stats;
int err = -EINVAL;
if (skb->len == CAN_MTU) {
@@ -285,8 +286,8 @@ int can_send(struct sk_buff *skb, int loop)
netif_rx_ni(newskb);
/* update statistics */
- can_stats->tx_frames++;
- can_stats->tx_frames_delta++;
+ pkg_stats->tx_frames++;
+ pkg_stats->tx_frames_delta++;
return 0;
@@ -298,13 +299,15 @@ EXPORT_SYMBOL(can_send);
/* af_can rx path */
-static struct can_dev_rcv_lists *find_dev_rcv_lists(struct net *net,
- struct net_device *dev)
+static struct can_dev_rcv_lists *can_dev_rcv_lists_find(struct net *net,
+ struct net_device *dev)
{
- if (!dev)
- return net->can.can_rx_alldev_list;
- else
- return (struct can_dev_rcv_lists *)dev->ml_priv;
+ if (dev) {
+ struct can_ml_priv *ml_priv = dev->ml_priv;
+ return &ml_priv->dev_rcv_lists;
+ } else {
+ return net->can.rx_alldev_list;
+ }
}
/**
@@ -331,7 +334,7 @@ static unsigned int effhash(canid_t can_id)
}
/**
- * find_rcv_list - determine optimal filterlist inside device filter struct
+ * can_rcv_list_find - determine optimal filterlist inside device filter struct
* @can_id: pointer to CAN identifier of a given can_filter
* @mask: pointer to CAN mask of a given can_filter
* @d: pointer to the device filter struct
@@ -357,8 +360,8 @@ static unsigned int effhash(canid_t can_id)
* Constistency checked mask.
* Reduced can_id to have a preprocessed filter compare value.
*/
-static struct hlist_head *find_rcv_list(canid_t *can_id, canid_t *mask,
- struct can_dev_rcv_lists *d)
+static struct hlist_head *can_rcv_list_find(canid_t *can_id, canid_t *mask,
+ struct can_dev_rcv_lists *dev_rcv_lists)
{
canid_t inv = *can_id & CAN_INV_FILTER; /* save flag before masking */
@@ -366,7 +369,7 @@ static struct hlist_head *find_rcv_list(canid_t *can_id, canid_t *mask,
if (*mask & CAN_ERR_FLAG) {
/* clear CAN_ERR_FLAG in filter entry */
*mask &= CAN_ERR_MASK;
- return &d->rx[RX_ERR];
+ return &dev_rcv_lists->rx[RX_ERR];
}
/* with cleared CAN_ERR_FLAG we have a simple mask/value filterpair */
@@ -382,26 +385,26 @@ static struct hlist_head *find_rcv_list(canid_t *can_id, canid_t *mask,
/* inverse can_id/can_mask filter */
if (inv)
- return &d->rx[RX_INV];
+ return &dev_rcv_lists->rx[RX_INV];
/* mask == 0 => no condition testing at receive time */
if (!(*mask))
- return &d->rx[RX_ALL];
+ return &dev_rcv_lists->rx[RX_ALL];
/* extra filterlists for the subscription of a single non-RTR can_id */
if (((*mask & CAN_EFF_RTR_FLAGS) == CAN_EFF_RTR_FLAGS) &&
!(*can_id & CAN_RTR_FLAG)) {
if (*can_id & CAN_EFF_FLAG) {
if (*mask == (CAN_EFF_MASK | CAN_EFF_RTR_FLAGS))
- return &d->rx_eff[effhash(*can_id)];
+ return &dev_rcv_lists->rx_eff[effhash(*can_id)];
} else {
if (*mask == (CAN_SFF_MASK | CAN_EFF_RTR_FLAGS))
- return &d->rx_sff[*can_id];
+ return &dev_rcv_lists->rx_sff[*can_id];
}
}
/* default: filter via can_id/can_mask */
- return &d->rx[RX_FIL];
+ return &dev_rcv_lists->rx[RX_FIL];
}
/**
@@ -438,10 +441,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
canid_t mask, void (*func)(struct sk_buff *, void *),
void *data, char *ident, struct sock *sk)
{
- struct receiver *r;
- struct hlist_head *rl;
- struct can_dev_rcv_lists *d;
- struct s_pstats *can_pstats = net->can.can_pstats;
+ struct receiver *rcv;
+ struct hlist_head *rcv_list;
+ struct can_dev_rcv_lists *dev_rcv_lists;
+ struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
int err = 0;
/* insert new receiver (dev,canid,mask) -> (func,data) */
@@ -452,36 +455,30 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
if (dev && !net_eq(net, dev_net(dev)))
return -ENODEV;
- r = kmem_cache_alloc(rcv_cache, GFP_KERNEL);
- if (!r)
+ rcv = kmem_cache_alloc(rcv_cache, GFP_KERNEL);
+ if (!rcv)
return -ENOMEM;
- spin_lock(&net->can.can_rcvlists_lock);
+ spin_lock_bh(&net->can.rcvlists_lock);
- d = find_dev_rcv_lists(net, dev);
- if (d) {
- rl = find_rcv_list(&can_id, &mask, d);
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+ rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
- r->can_id = can_id;
- r->mask = mask;
- r->matches = 0;
- r->func = func;
- r->data = data;
- r->ident = ident;
- r->sk = sk;
+ rcv->can_id = can_id;
+ rcv->mask = mask;
+ rcv->matches = 0;
+ rcv->func = func;
+ rcv->data = data;
+ rcv->ident = ident;
+ rcv->sk = sk;
- hlist_add_head_rcu(&r->list, rl);
- d->entries++;
+ hlist_add_head_rcu(&rcv->list, rcv_list);
+ dev_rcv_lists->entries++;
- can_pstats->rcv_entries++;
- if (can_pstats->rcv_entries_max < can_pstats->rcv_entries)
- can_pstats->rcv_entries_max = can_pstats->rcv_entries;
- } else {
- kmem_cache_free(rcv_cache, r);
- err = -ENODEV;
- }
-
- spin_unlock(&net->can.can_rcvlists_lock);
+ rcv_lists_stats->rcv_entries++;
+ rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max,
+ rcv_lists_stats->rcv_entries);
+ spin_unlock_bh(&net->can.rcvlists_lock);
return err;
}
@@ -490,10 +487,10 @@ EXPORT_SYMBOL(can_rx_register);
/* can_rx_delete_receiver - rcu callback for single receiver entry removal */
static void can_rx_delete_receiver(struct rcu_head *rp)
{
- struct receiver *r = container_of(rp, struct receiver, rcu);
- struct sock *sk = r->sk;
+ struct receiver *rcv = container_of(rp, struct receiver, rcu);
+ struct sock *sk = rcv->sk;
- kmem_cache_free(rcv_cache, r);
+ kmem_cache_free(rcv_cache, rcv);
if (sk)
sock_put(sk);
}
@@ -513,10 +510,10 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
canid_t mask, void (*func)(struct sk_buff *, void *),
void *data)
{
- struct receiver *r = NULL;
- struct hlist_head *rl;
- struct s_pstats *can_pstats = net->can.can_pstats;
- struct can_dev_rcv_lists *d;
+ struct receiver *rcv = NULL;
+ struct hlist_head *rcv_list;
+ struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
+ struct can_dev_rcv_lists *dev_rcv_lists;
if (dev && dev->type != ARPHRD_CAN)
return;
@@ -524,83 +521,69 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
if (dev && !net_eq(net, dev_net(dev)))
return;
- spin_lock(&net->can.can_rcvlists_lock);
+ spin_lock_bh(&net->can.rcvlists_lock);
- d = find_dev_rcv_lists(net, dev);
- if (!d) {
- pr_err("BUG: receive list not found for dev %s, id %03X, mask %03X\n",
- DNAME(dev), can_id, mask);
- goto out;
- }
-
- rl = find_rcv_list(&can_id, &mask, d);
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+ rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
/* Search the receiver list for the item to delete. This should
* exist, since no receiver may be unregistered that hasn't
* been registered before.
*/
-
- hlist_for_each_entry_rcu(r, rl, list) {
- if (r->can_id == can_id && r->mask == mask &&
- r->func == func && r->data == data)
+ hlist_for_each_entry_rcu(rcv, rcv_list, list) {
+ if (rcv->can_id == can_id && rcv->mask == mask &&
+ rcv->func == func && rcv->data == data)
break;
}
/* Check for bugs in CAN protocol implementations using af_can.c:
- * 'r' will be NULL if no matching list item was found for removal.
+ * 'rcv' will be NULL if no matching list item was found for removal.
*/
-
- if (!r) {
+ if (!rcv) {
WARN(1, "BUG: receive list entry not found for dev %s, id %03X, mask %03X\n",
DNAME(dev), can_id, mask);
goto out;
}
- hlist_del_rcu(&r->list);
- d->entries--;
+ hlist_del_rcu(&rcv->list);
+ dev_rcv_lists->entries--;
- if (can_pstats->rcv_entries > 0)
- can_pstats->rcv_entries--;
-
- /* remove device structure requested by NETDEV_UNREGISTER */
- if (d->remove_on_zero_entries && !d->entries) {
- kfree(d);
- dev->ml_priv = NULL;
- }
+ if (rcv_lists_stats->rcv_entries > 0)
+ rcv_lists_stats->rcv_entries--;
out:
- spin_unlock(&net->can.can_rcvlists_lock);
+ spin_unlock_bh(&net->can.rcvlists_lock);
/* schedule the receiver item for deletion */
- if (r) {
- if (r->sk)
- sock_hold(r->sk);
- call_rcu(&r->rcu, can_rx_delete_receiver);
+ if (rcv) {
+ if (rcv->sk)
+ sock_hold(rcv->sk);
+ call_rcu(&rcv->rcu, can_rx_delete_receiver);
}
}
EXPORT_SYMBOL(can_rx_unregister);
-static inline void deliver(struct sk_buff *skb, struct receiver *r)
+static inline void deliver(struct sk_buff *skb, struct receiver *rcv)
{
- r->func(skb, r->data);
- r->matches++;
+ rcv->func(skb, rcv->data);
+ rcv->matches++;
}
-static int can_rcv_filter(struct can_dev_rcv_lists *d, struct sk_buff *skb)
+static int can_rcv_filter(struct can_dev_rcv_lists *dev_rcv_lists, struct sk_buff *skb)
{
- struct receiver *r;
+ struct receiver *rcv;
int matches = 0;
struct can_frame *cf = (struct can_frame *)skb->data;
canid_t can_id = cf->can_id;
- if (d->entries == 0)
+ if (dev_rcv_lists->entries == 0)
return 0;
if (can_id & CAN_ERR_FLAG) {
/* check for error message frame entries only */
- hlist_for_each_entry_rcu(r, &d->rx[RX_ERR], list) {
- if (can_id & r->mask) {
- deliver(skb, r);
+ hlist_for_each_entry_rcu(rcv, &dev_rcv_lists->rx[RX_ERR], list) {
+ if (can_id & rcv->mask) {
+ deliver(skb, rcv);
matches++;
}
}
@@ -608,23 +591,23 @@ static int can_rcv_filter(struct can_dev_rcv_lists *d, struct sk_buff *skb)
}
/* check for unfiltered entries */
- hlist_for_each_entry_rcu(r, &d->rx[RX_ALL], list) {
- deliver(skb, r);
+ hlist_for_each_entry_rcu(rcv, &dev_rcv_lists->rx[RX_ALL], list) {
+ deliver(skb, rcv);
matches++;
}
/* check for can_id/mask entries */
- hlist_for_each_entry_rcu(r, &d->rx[RX_FIL], list) {
- if ((can_id & r->mask) == r->can_id) {
- deliver(skb, r);
+ hlist_for_each_entry_rcu(rcv, &dev_rcv_lists->rx[RX_FIL], list) {
+ if ((can_id & rcv->mask) == rcv->can_id) {
+ deliver(skb, rcv);
matches++;
}
}
/* check for inverted can_id/mask entries */
- hlist_for_each_entry_rcu(r, &d->rx[RX_INV], list) {
- if ((can_id & r->mask) != r->can_id) {
- deliver(skb, r);
+ hlist_for_each_entry_rcu(rcv, &dev_rcv_lists->rx[RX_INV], list) {
+ if ((can_id & rcv->mask) != rcv->can_id) {
+ deliver(skb, rcv);
matches++;
}
}
@@ -634,16 +617,16 @@ static int can_rcv_filter(struct can_dev_rcv_lists *d, struct sk_buff *skb)
return matches;
if (can_id & CAN_EFF_FLAG) {
- hlist_for_each_entry_rcu(r, &d->rx_eff[effhash(can_id)], list) {
- if (r->can_id == can_id) {
- deliver(skb, r);
+ hlist_for_each_entry_rcu(rcv, &dev_rcv_lists->rx_eff[effhash(can_id)], list) {
+ if (rcv->can_id == can_id) {
+ deliver(skb, rcv);
matches++;
}
}
} else {
can_id &= CAN_SFF_MASK;
- hlist_for_each_entry_rcu(r, &d->rx_sff[can_id], list) {
- deliver(skb, r);
+ hlist_for_each_entry_rcu(rcv, &dev_rcv_lists->rx_sff[can_id], list) {
+ deliver(skb, rcv);
matches++;
}
}
@@ -653,14 +636,14 @@ static int can_rcv_filter(struct can_dev_rcv_lists *d, struct sk_buff *skb)
static void can_receive(struct sk_buff *skb, struct net_device *dev)
{
- struct can_dev_rcv_lists *d;
+ struct can_dev_rcv_lists *dev_rcv_lists;
struct net *net = dev_net(dev);
- struct s_stats *can_stats = net->can.can_stats;
+ struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
int matches;
/* update statistics */
- can_stats->rx_frames++;
- can_stats->rx_frames_delta++;
+ pkg_stats->rx_frames++;
+ pkg_stats->rx_frames_delta++;
/* create non-zero unique skb identifier together with *skb */
while (!(can_skb_prv(skb)->skbcnt))
@@ -669,12 +652,11 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev)
rcu_read_lock();
/* deliver the packet to sockets listening on all devices */
- matches = can_rcv_filter(net->can.can_rx_alldev_list, skb);
+ matches = can_rcv_filter(net->can.rx_alldev_list, skb);
/* find receive list for this device */
- d = find_dev_rcv_lists(net, dev);
- if (d)
- matches += can_rcv_filter(d, skb);
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+ matches += can_rcv_filter(dev_rcv_lists, skb);
rcu_read_unlock();
@@ -682,8 +664,8 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev)
consume_skb(skb);
if (matches > 0) {
- can_stats->matches++;
- can_stats->matches_delta++;
+ pkg_stats->matches++;
+ pkg_stats->matches_delta++;
}
}
@@ -789,41 +771,14 @@ static int can_notifier(struct notifier_block *nb, unsigned long msg,
void *ptr)
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- struct can_dev_rcv_lists *d;
if (dev->type != ARPHRD_CAN)
return NOTIFY_DONE;
switch (msg) {
case NETDEV_REGISTER:
-
- /* create new dev_rcv_lists for this device */
- d = kzalloc(sizeof(*d), GFP_KERNEL);
- if (!d)
- return NOTIFY_DONE;
- BUG_ON(dev->ml_priv);
- dev->ml_priv = d;
-
- break;
-
- case NETDEV_UNREGISTER:
- spin_lock(&dev_net(dev)->can.can_rcvlists_lock);
-
- d = dev->ml_priv;
- if (d) {
- if (d->entries) {
- d->remove_on_zero_entries = 1;
- } else {
- kfree(d);
- dev->ml_priv = NULL;
- }
- } else {
- pr_err("can: notifier: receive list not found for dev %s\n",
- dev->name);
- }
-
- spin_unlock(&dev_net(dev)->can.can_rcvlists_lock);
-
+ WARN(!dev->ml_priv,
+ "No CAN mid layer private allocated, please fix your driver and use alloc_candev()!\n");
break;
}
@@ -832,66 +787,51 @@ static int can_notifier(struct notifier_block *nb, unsigned long msg,
static int can_pernet_init(struct net *net)
{
- spin_lock_init(&net->can.can_rcvlists_lock);
- net->can.can_rx_alldev_list =
- kzalloc(sizeof(*net->can.can_rx_alldev_list), GFP_KERNEL);
- if (!net->can.can_rx_alldev_list)
+ spin_lock_init(&net->can.rcvlists_lock);
+ net->can.rx_alldev_list =
+ kzalloc(sizeof(*net->can.rx_alldev_list), GFP_KERNEL);
+ if (!net->can.rx_alldev_list)
goto out;
- net->can.can_stats = kzalloc(sizeof(*net->can.can_stats), GFP_KERNEL);
- if (!net->can.can_stats)
- goto out_free_alldev_list;
- net->can.can_pstats = kzalloc(sizeof(*net->can.can_pstats), GFP_KERNEL);
- if (!net->can.can_pstats)
- goto out_free_can_stats;
+ net->can.pkg_stats = kzalloc(sizeof(*net->can.pkg_stats), GFP_KERNEL);
+ if (!net->can.pkg_stats)
+ goto out_free_rx_alldev_list;
+ net->can.rcv_lists_stats = kzalloc(sizeof(*net->can.rcv_lists_stats), GFP_KERNEL);
+ if (!net->can.rcv_lists_stats)
+ goto out_free_pkg_stats;
if (IS_ENABLED(CONFIG_PROC_FS)) {
/* the statistics are updated every second (timer triggered) */
if (stats_timer) {
- timer_setup(&net->can.can_stattimer, can_stat_update,
+ timer_setup(&net->can.stattimer, can_stat_update,
0);
- mod_timer(&net->can.can_stattimer,
+ mod_timer(&net->can.stattimer,
round_jiffies(jiffies + HZ));
}
- net->can.can_stats->jiffies_init = jiffies;
+ net->can.pkg_stats->jiffies_init = jiffies;
can_init_proc(net);
}
return 0;
- out_free_can_stats:
- kfree(net->can.can_stats);
- out_free_alldev_list:
- kfree(net->can.can_rx_alldev_list);
+ out_free_pkg_stats:
+ kfree(net->can.pkg_stats);
+ out_free_rx_alldev_list:
+ kfree(net->can.rx_alldev_list);
out:
return -ENOMEM;
}
static void can_pernet_exit(struct net *net)
{
- struct net_device *dev;
-
if (IS_ENABLED(CONFIG_PROC_FS)) {
can_remove_proc(net);
if (stats_timer)
- del_timer_sync(&net->can.can_stattimer);
+ del_timer_sync(&net->can.stattimer);
}
- /* remove created dev_rcv_lists from still registered CAN devices */
- rcu_read_lock();
- for_each_netdev_rcu(net, dev) {
- if (dev->type == ARPHRD_CAN && dev->ml_priv) {
- struct can_dev_rcv_lists *d = dev->ml_priv;
-
- BUG_ON(d->entries);
- kfree(d);
- dev->ml_priv = NULL;
- }
- }
- rcu_read_unlock();
-
- kfree(net->can.can_rx_alldev_list);
- kfree(net->can.can_stats);
- kfree(net->can.can_pstats);
+ kfree(net->can.rx_alldev_list);
+ kfree(net->can.pkg_stats);
+ kfree(net->can.rcv_lists_stats);
}
/* af_can module init/exit functions */
diff --git a/net/can/af_can.h b/net/can/af_can.h
index 9cdb790..7c2d916 100644
--- a/net/can/af_can.h
+++ b/net/can/af_can.h
@@ -60,25 +60,10 @@ struct receiver {
struct rcu_head rcu;
};
-#define CAN_SFF_RCV_ARRAY_SZ (1 << CAN_SFF_ID_BITS)
-#define CAN_EFF_RCV_HASH_BITS 10
-#define CAN_EFF_RCV_ARRAY_SZ (1 << CAN_EFF_RCV_HASH_BITS)
-
-enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX };
-
-/* per device receive filters linked at dev->ml_priv */
-struct can_dev_rcv_lists {
- struct hlist_head rx[RX_MAX];
- struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ];
- struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ];
- int remove_on_zero_entries;
- int entries;
-};
-
/* statistic structures */
/* can be reset e.g. by can_init_stats() */
-struct s_stats {
+struct can_pkg_stats {
unsigned long jiffies_init;
unsigned long rx_frames;
@@ -103,7 +88,7 @@ struct s_stats {
};
/* persistent statistics */
-struct s_pstats {
+struct can_rcv_lists_stats {
unsigned long stats_reset;
unsigned long user_reset;
unsigned long rcv_entries;
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 28fd1a1..c96fa0f 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -1294,7 +1294,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
/* no bound device as default => check msg_name */
DECLARE_SOCKADDR(struct sockaddr_can *, addr, msg->msg_name);
- if (msg->msg_namelen < sizeof(*addr))
+ if (msg->msg_namelen < CAN_REQUIRED_SIZE(*addr, can_ifindex))
return -EINVAL;
if (addr->can_family != AF_CAN)
@@ -1536,7 +1536,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr *uaddr, int len,
struct net *net = sock_net(sk);
int ret = 0;
- if (len < sizeof(*addr))
+ if (len < CAN_REQUIRED_SIZE(*addr, can_ifindex))
return -EINVAL;
lock_sock(sk);
diff --git a/net/can/j1939/Kconfig b/net/can/j1939/Kconfig
new file mode 100644
index 0000000..2998298
--- /dev/null
+++ b/net/can/j1939/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# SAE J1939 network layer core configuration
+#
+
+config CAN_J1939
+ tristate "SAE J1939"
+ depends on CAN
+ help
+ SAE J1939
+ Say Y to have in-kernel support for j1939 socket type. This
+ allows communication according to SAE j1939.
+ The relevant parts in kernel are
+ SAE j1939-21 (datalink & transport protocol)
+ & SAE j1939-81 (network management).
diff --git a/net/can/j1939/Makefile b/net/can/j1939/Makefile
new file mode 100644
index 0000000..19181bd
--- /dev/null
+++ b/net/can/j1939/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_CAN_J1939) += can-j1939.o
+
+can-j1939-objs := \
+ address-claim.o \
+ bus.o \
+ main.o \
+ socket.o \
+ transport.o
diff --git a/net/can/j1939/address-claim.c b/net/can/j1939/address-claim.c
new file mode 100644
index 0000000..f33c473
--- /dev/null
+++ b/net/can/j1939/address-claim.c
@@ -0,0 +1,230 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2010-2011 EIA Electronics,
+// Kurt Van Dijck <kurt.van.dijck@eia.be>
+// Copyright (c) 2010-2011 EIA Electronics,
+// Pieter Beyens <pieter.beyens@eia.be>
+// Copyright (c) 2017-2019 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2017-2019 Pengutronix,
+// Oleksij Rempel <kernel@pengutronix.de>
+
+/* J1939 Address Claiming.
+ * Address Claiming in the kernel
+ * - keeps track of the AC states of ECU's,
+ * - resolves NAME<=>SA taking into account the AC states of ECU's.
+ *
+ * All Address Claim msgs (including host-originated msg) are processed
+ * at the receive path (a sent msg is always received again via CAN echo).
+ * As such, the processing of AC msgs is done in the order on which msgs
+ * are sent on the bus.
+ *
+ * This module doesn't send msgs itself (e.g. replies on Address Claims),
+ * this is the responsibility of a user space application or daemon.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+
+#include "j1939-priv.h"
+
+static inline name_t j1939_skb_to_name(const struct sk_buff *skb)
+{
+ return le64_to_cpup((__le64 *)skb->data);
+}
+
+static inline bool j1939_ac_msg_is_request(struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ int req_pgn;
+
+ if (skb->len < 3 || skcb->addr.pgn != J1939_PGN_REQUEST)
+ return false;
+
+ req_pgn = skb->data[0] | (skb->data[1] << 8) | (skb->data[2] << 16);
+
+ return req_pgn == J1939_PGN_ADDRESS_CLAIMED;
+}
+
+static int j1939_ac_verify_outgoing(struct j1939_priv *priv,
+ struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+
+ if (skb->len != 8) {
+ netdev_notice(priv->ndev, "tx address claim with dlc %i\n",
+ skb->len);
+ return -EPROTO;
+ }
+
+ if (skcb->addr.src_name != j1939_skb_to_name(skb)) {
+ netdev_notice(priv->ndev, "tx address claim with different name\n");
+ return -EPROTO;
+ }
+
+ if (skcb->addr.sa == J1939_NO_ADDR) {
+ netdev_notice(priv->ndev, "tx address claim with broadcast sa\n");
+ return -EPROTO;
+ }
+
+ /* ac must always be a broadcast */
+ if (skcb->addr.dst_name || skcb->addr.da != J1939_NO_ADDR) {
+ netdev_notice(priv->ndev, "tx address claim with dest, not broadcast\n");
+ return -EPROTO;
+ }
+ return 0;
+}
+
+int j1939_ac_fixup(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ int ret;
+ u8 addr;
+
+ /* network mgmt: address claiming msgs */
+ if (skcb->addr.pgn == J1939_PGN_ADDRESS_CLAIMED) {
+ struct j1939_ecu *ecu;
+
+ ret = j1939_ac_verify_outgoing(priv, skb);
+ /* return both when failure & when successful */
+ if (ret < 0)
+ return ret;
+ ecu = j1939_ecu_get_by_name(priv, skcb->addr.src_name);
+ if (!ecu)
+ return -ENODEV;
+
+ if (ecu->addr != skcb->addr.sa)
+ /* hold further traffic for ecu, remove from parent */
+ j1939_ecu_unmap(ecu);
+ j1939_ecu_put(ecu);
+ } else if (skcb->addr.src_name) {
+ /* assign source address */
+ addr = j1939_name_to_addr(priv, skcb->addr.src_name);
+ if (!j1939_address_is_unicast(addr) &&
+ !j1939_ac_msg_is_request(skb)) {
+ netdev_notice(priv->ndev, "tx drop: invalid sa for name 0x%016llx\n",
+ skcb->addr.src_name);
+ return -EADDRNOTAVAIL;
+ }
+ skcb->addr.sa = addr;
+ }
+
+ /* assign destination address */
+ if (skcb->addr.dst_name) {
+ addr = j1939_name_to_addr(priv, skcb->addr.dst_name);
+ if (!j1939_address_is_unicast(addr)) {
+ netdev_notice(priv->ndev, "tx drop: invalid da for name 0x%016llx\n",
+ skcb->addr.dst_name);
+ return -EADDRNOTAVAIL;
+ }
+ skcb->addr.da = addr;
+ }
+ return 0;
+}
+
+static void j1939_ac_process(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_ecu *ecu, *prev;
+ name_t name;
+
+ if (skb->len != 8) {
+ netdev_notice(priv->ndev, "rx address claim with wrong dlc %i\n",
+ skb->len);
+ return;
+ }
+
+ name = j1939_skb_to_name(skb);
+ skcb->addr.src_name = name;
+ if (!name) {
+ netdev_notice(priv->ndev, "rx address claim without name\n");
+ return;
+ }
+
+ if (!j1939_address_is_valid(skcb->addr.sa)) {
+ netdev_notice(priv->ndev, "rx address claim with broadcast sa\n");
+ return;
+ }
+
+ write_lock_bh(&priv->lock);
+
+ /* Few words on the ECU ref counting:
+ *
+ * First we get an ECU handle, either with
+ * j1939_ecu_get_by_name_locked() (increments the ref counter)
+ * or j1939_ecu_create_locked() (initializes an ECU object
+ * with a ref counter of 1).
+ *
+ * j1939_ecu_unmap_locked() will decrement the ref counter,
+ * but only if the ECU was mapped before. So "ecu" still
+ * belongs to us.
+ *
+ * j1939_ecu_timer_start() will increment the ref counter
+ * before it starts the timer, so we can put the ecu when
+ * leaving this function.
+ */
+ ecu = j1939_ecu_get_by_name_locked(priv, name);
+ if (!ecu && j1939_address_is_unicast(skcb->addr.sa))
+ ecu = j1939_ecu_create_locked(priv, name);
+
+ if (IS_ERR_OR_NULL(ecu))
+ goto out_unlock_bh;
+
+ /* cancel pending (previous) address claim */
+ j1939_ecu_timer_cancel(ecu);
+
+ if (j1939_address_is_idle(skcb->addr.sa)) {
+ j1939_ecu_unmap_locked(ecu);
+ goto out_ecu_put;
+ }
+
+ /* save new addr */
+ if (ecu->addr != skcb->addr.sa)
+ j1939_ecu_unmap_locked(ecu);
+ ecu->addr = skcb->addr.sa;
+
+ prev = j1939_ecu_get_by_addr_locked(priv, skcb->addr.sa);
+ if (prev) {
+ if (ecu->name > prev->name) {
+ j1939_ecu_unmap_locked(ecu);
+ j1939_ecu_put(prev);
+ goto out_ecu_put;
+ } else {
+ /* kick prev if less or equal */
+ j1939_ecu_unmap_locked(prev);
+ j1939_ecu_put(prev);
+ }
+ }
+
+ j1939_ecu_timer_start(ecu);
+ out_ecu_put:
+ j1939_ecu_put(ecu);
+ out_unlock_bh:
+ write_unlock_bh(&priv->lock);
+}
+
+void j1939_ac_recv(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_ecu *ecu;
+
+ /* network mgmt */
+ if (skcb->addr.pgn == J1939_PGN_ADDRESS_CLAIMED) {
+ j1939_ac_process(priv, skb);
+ } else if (j1939_address_is_unicast(skcb->addr.sa)) {
+ /* assign source name */
+ ecu = j1939_ecu_get_by_addr(priv, skcb->addr.sa);
+ if (ecu) {
+ skcb->addr.src_name = ecu->name;
+ j1939_ecu_put(ecu);
+ }
+ }
+
+ /* assign destination name */
+ ecu = j1939_ecu_get_by_addr(priv, skcb->addr.da);
+ if (ecu) {
+ skcb->addr.dst_name = ecu->name;
+ j1939_ecu_put(ecu);
+ }
+}
diff --git a/net/can/j1939/bus.c b/net/can/j1939/bus.c
new file mode 100644
index 0000000..4866879
--- /dev/null
+++ b/net/can/j1939/bus.c
@@ -0,0 +1,333 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2010-2011 EIA Electronics,
+// Kurt Van Dijck <kurt.van.dijck@eia.be>
+// Copyright (c) 2017-2019 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2017-2019 Pengutronix,
+// Oleksij Rempel <kernel@pengutronix.de>
+
+/* bus for j1939 remote devices
+ * Since rtnetlink, no real bus is used.
+ */
+
+#include <net/sock.h>
+
+#include "j1939-priv.h"
+
+static void __j1939_ecu_release(struct kref *kref)
+{
+ struct j1939_ecu *ecu = container_of(kref, struct j1939_ecu, kref);
+ struct j1939_priv *priv = ecu->priv;
+
+ list_del(&ecu->list);
+ kfree(ecu);
+ j1939_priv_put(priv);
+}
+
+void j1939_ecu_put(struct j1939_ecu *ecu)
+{
+ kref_put(&ecu->kref, __j1939_ecu_release);
+}
+
+static void j1939_ecu_get(struct j1939_ecu *ecu)
+{
+ kref_get(&ecu->kref);
+}
+
+static bool j1939_ecu_is_mapped_locked(struct j1939_ecu *ecu)
+{
+ struct j1939_priv *priv = ecu->priv;
+
+ lockdep_assert_held(&priv->lock);
+
+ return j1939_ecu_find_by_addr_locked(priv, ecu->addr) == ecu;
+}
+
+/* ECU device interface */
+/* map ECU to a bus address space */
+static void j1939_ecu_map_locked(struct j1939_ecu *ecu)
+{
+ struct j1939_priv *priv = ecu->priv;
+ struct j1939_addr_ent *ent;
+
+ lockdep_assert_held(&priv->lock);
+
+ if (!j1939_address_is_unicast(ecu->addr))
+ return;
+
+ ent = &priv->ents[ecu->addr];
+
+ if (ent->ecu) {
+ netdev_warn(priv->ndev, "Trying to map already mapped ECU, addr: 0x%02x, name: 0x%016llx. Skip it.\n",
+ ecu->addr, ecu->name);
+ return;
+ }
+
+ j1939_ecu_get(ecu);
+ ent->ecu = ecu;
+ ent->nusers += ecu->nusers;
+}
+
+/* unmap ECU from a bus address space */
+void j1939_ecu_unmap_locked(struct j1939_ecu *ecu)
+{
+ struct j1939_priv *priv = ecu->priv;
+ struct j1939_addr_ent *ent;
+
+ lockdep_assert_held(&priv->lock);
+
+ if (!j1939_address_is_unicast(ecu->addr))
+ return;
+
+ if (!j1939_ecu_is_mapped_locked(ecu))
+ return;
+
+ ent = &priv->ents[ecu->addr];
+ ent->ecu = NULL;
+ ent->nusers -= ecu->nusers;
+ j1939_ecu_put(ecu);
+}
+
+void j1939_ecu_unmap(struct j1939_ecu *ecu)
+{
+ write_lock_bh(&ecu->priv->lock);
+ j1939_ecu_unmap_locked(ecu);
+ write_unlock_bh(&ecu->priv->lock);
+}
+
+void j1939_ecu_unmap_all(struct j1939_priv *priv)
+{
+ int i;
+
+ write_lock_bh(&priv->lock);
+ for (i = 0; i < ARRAY_SIZE(priv->ents); i++)
+ if (priv->ents[i].ecu)
+ j1939_ecu_unmap_locked(priv->ents[i].ecu);
+ write_unlock_bh(&priv->lock);
+}
+
+void j1939_ecu_timer_start(struct j1939_ecu *ecu)
+{
+ /* The ECU is held here and released in the
+ * j1939_ecu_timer_handler() or j1939_ecu_timer_cancel().
+ */
+ j1939_ecu_get(ecu);
+
+ /* Schedule timer in 250 msec to commit address change. */
+ hrtimer_start(&ecu->ac_timer, ms_to_ktime(250),
+ HRTIMER_MODE_REL_SOFT);
+}
+
+void j1939_ecu_timer_cancel(struct j1939_ecu *ecu)
+{
+ if (hrtimer_cancel(&ecu->ac_timer))
+ j1939_ecu_put(ecu);
+}
+
+static enum hrtimer_restart j1939_ecu_timer_handler(struct hrtimer *hrtimer)
+{
+ struct j1939_ecu *ecu =
+ container_of(hrtimer, struct j1939_ecu, ac_timer);
+ struct j1939_priv *priv = ecu->priv;
+
+ write_lock_bh(&priv->lock);
+ /* TODO: can we test if ecu->addr is unicast before starting
+ * the timer?
+ */
+ j1939_ecu_map_locked(ecu);
+
+ /* The corresponding j1939_ecu_get() is in
+ * j1939_ecu_timer_start().
+ */
+ j1939_ecu_put(ecu);
+ write_unlock_bh(&priv->lock);
+
+ return HRTIMER_NORESTART;
+}
+
+struct j1939_ecu *j1939_ecu_create_locked(struct j1939_priv *priv, name_t name)
+{
+ struct j1939_ecu *ecu;
+
+ lockdep_assert_held(&priv->lock);
+
+ ecu = kzalloc(sizeof(*ecu), gfp_any());
+ if (!ecu)
+ return ERR_PTR(-ENOMEM);
+ kref_init(&ecu->kref);
+ ecu->addr = J1939_IDLE_ADDR;
+ ecu->name = name;
+
+ hrtimer_init(&ecu->ac_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_SOFT);
+ ecu->ac_timer.function = j1939_ecu_timer_handler;
+ INIT_LIST_HEAD(&ecu->list);
+
+ j1939_priv_get(priv);
+ ecu->priv = priv;
+ list_add_tail(&ecu->list, &priv->ecus);
+
+ return ecu;
+}
+
+struct j1939_ecu *j1939_ecu_find_by_addr_locked(struct j1939_priv *priv,
+ u8 addr)
+{
+ lockdep_assert_held(&priv->lock);
+
+ return priv->ents[addr].ecu;
+}
+
+struct j1939_ecu *j1939_ecu_get_by_addr_locked(struct j1939_priv *priv, u8 addr)
+{
+ struct j1939_ecu *ecu;
+
+ lockdep_assert_held(&priv->lock);
+
+ if (!j1939_address_is_unicast(addr))
+ return NULL;
+
+ ecu = j1939_ecu_find_by_addr_locked(priv, addr);
+ if (ecu)
+ j1939_ecu_get(ecu);
+
+ return ecu;
+}
+
+struct j1939_ecu *j1939_ecu_get_by_addr(struct j1939_priv *priv, u8 addr)
+{
+ struct j1939_ecu *ecu;
+
+ read_lock_bh(&priv->lock);
+ ecu = j1939_ecu_get_by_addr_locked(priv, addr);
+ read_unlock_bh(&priv->lock);
+
+ return ecu;
+}
+
+/* get pointer to ecu without increasing ref counter */
+static struct j1939_ecu *j1939_ecu_find_by_name_locked(struct j1939_priv *priv,
+ name_t name)
+{
+ struct j1939_ecu *ecu;
+
+ lockdep_assert_held(&priv->lock);
+
+ list_for_each_entry(ecu, &priv->ecus, list) {
+ if (ecu->name == name)
+ return ecu;
+ }
+
+ return NULL;
+}
+
+struct j1939_ecu *j1939_ecu_get_by_name_locked(struct j1939_priv *priv,
+ name_t name)
+{
+ struct j1939_ecu *ecu;
+
+ lockdep_assert_held(&priv->lock);
+
+ if (!name)
+ return NULL;
+
+ ecu = j1939_ecu_find_by_name_locked(priv, name);
+ if (ecu)
+ j1939_ecu_get(ecu);
+
+ return ecu;
+}
+
+struct j1939_ecu *j1939_ecu_get_by_name(struct j1939_priv *priv, name_t name)
+{
+ struct j1939_ecu *ecu;
+
+ read_lock_bh(&priv->lock);
+ ecu = j1939_ecu_get_by_name_locked(priv, name);
+ read_unlock_bh(&priv->lock);
+
+ return ecu;
+}
+
+u8 j1939_name_to_addr(struct j1939_priv *priv, name_t name)
+{
+ struct j1939_ecu *ecu;
+ int addr = J1939_IDLE_ADDR;
+
+ if (!name)
+ return J1939_NO_ADDR;
+
+ read_lock_bh(&priv->lock);
+ ecu = j1939_ecu_find_by_name_locked(priv, name);
+ if (ecu && j1939_ecu_is_mapped_locked(ecu))
+ /* ecu's SA is registered */
+ addr = ecu->addr;
+
+ read_unlock_bh(&priv->lock);
+
+ return addr;
+}
+
+/* TX addr/name accounting
+ * Transport protocol needs to know if a SA is local or not
+ * These functions originate from userspace manipulating sockets,
+ * so locking is straigforward
+ */
+
+int j1939_local_ecu_get(struct j1939_priv *priv, name_t name, u8 sa)
+{
+ struct j1939_ecu *ecu;
+ int err = 0;
+
+ write_lock_bh(&priv->lock);
+
+ if (j1939_address_is_unicast(sa))
+ priv->ents[sa].nusers++;
+
+ if (!name)
+ goto done;
+
+ ecu = j1939_ecu_get_by_name_locked(priv, name);
+ if (!ecu)
+ ecu = j1939_ecu_create_locked(priv, name);
+ err = PTR_ERR_OR_ZERO(ecu);
+ if (err)
+ goto done;
+
+ ecu->nusers++;
+ /* TODO: do we care if ecu->addr != sa? */
+ if (j1939_ecu_is_mapped_locked(ecu))
+ /* ecu's sa is active already */
+ priv->ents[ecu->addr].nusers++;
+
+ done:
+ write_unlock_bh(&priv->lock);
+
+ return err;
+}
+
+void j1939_local_ecu_put(struct j1939_priv *priv, name_t name, u8 sa)
+{
+ struct j1939_ecu *ecu;
+
+ write_lock_bh(&priv->lock);
+
+ if (j1939_address_is_unicast(sa))
+ priv->ents[sa].nusers--;
+
+ if (!name)
+ goto done;
+
+ ecu = j1939_ecu_find_by_name_locked(priv, name);
+ if (WARN_ON_ONCE(!ecu))
+ goto done;
+
+ ecu->nusers--;
+ /* TODO: do we care if ecu->addr != sa? */
+ if (j1939_ecu_is_mapped_locked(ecu))
+ /* ecu's sa is active already */
+ priv->ents[ecu->addr].nusers--;
+ j1939_ecu_put(ecu);
+
+ done:
+ write_unlock_bh(&priv->lock);
+}
diff --git a/net/can/j1939/j1939-priv.h b/net/can/j1939/j1939-priv.h
new file mode 100644
index 0000000..12369b6
--- /dev/null
+++ b/net/can/j1939/j1939-priv.h
@@ -0,0 +1,338 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+// Copyright (c) 2010-2011 EIA Electronics,
+// Kurt Van Dijck <kurt.van.dijck@eia.be>
+// Copyright (c) 2017-2019 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2017-2019 Pengutronix,
+// Oleksij Rempel <kernel@pengutronix.de>
+
+#ifndef _J1939_PRIV_H_
+#define _J1939_PRIV_H_
+
+#include <linux/can/j1939.h>
+#include <net/sock.h>
+
+/* Timeout to receive the abort signal over loop back. In case CAN
+ * bus is open, the timeout should be triggered.
+ */
+#define J1939_XTP_ABORT_TIMEOUT_MS 500
+#define J1939_SIMPLE_ECHO_TIMEOUT_MS (10 * 1000)
+
+struct j1939_session;
+enum j1939_sk_errqueue_type {
+ J1939_ERRQUEUE_ACK,
+ J1939_ERRQUEUE_SCHED,
+ J1939_ERRQUEUE_ABORT,
+};
+
+/* j1939 devices */
+struct j1939_ecu {
+ struct list_head list;
+ name_t name;
+ u8 addr;
+
+ /* indicates that this ecu successfully claimed @sa as its address */
+ struct hrtimer ac_timer;
+ struct kref kref;
+ struct j1939_priv *priv;
+
+ /* count users, to help transport protocol decide for interaction */
+ int nusers;
+};
+
+struct j1939_priv {
+ struct list_head ecus;
+ /* local list entry in priv
+ * These allow irq (& softirq) context lookups on j1939 devices
+ * This approach (separate lists) is done as the other 2 alternatives
+ * are not easier or even wrong
+ * 1) using the pure kobject methods involves mutexes, which are not
+ * allowed in irq context.
+ * 2) duplicating data structures would require a lot of synchronization
+ * code
+ * usage:
+ */
+
+ /* segments need a lock to protect the above list */
+ rwlock_t lock;
+
+ struct net_device *ndev;
+
+ /* list of 256 ecu ptrs, that cache the claimed addresses.
+ * also protected by the above lock
+ */
+ struct j1939_addr_ent {
+ struct j1939_ecu *ecu;
+ /* count users, to help transport protocol */
+ int nusers;
+ } ents[256];
+
+ struct kref kref;
+
+ /* List of active sessions to prevent start of conflicting
+ * one.
+ *
+ * Do not start two sessions of same type, addresses and
+ * direction.
+ */
+ struct list_head active_session_list;
+
+ /* protects active_session_list */
+ spinlock_t active_session_list_lock;
+
+ unsigned int tp_max_packet_size;
+
+ /* lock for j1939_socks list */
+ spinlock_t j1939_socks_lock;
+ struct list_head j1939_socks;
+
+ struct kref rx_kref;
+};
+
+void j1939_ecu_put(struct j1939_ecu *ecu);
+
+/* keep the cache of what is local */
+int j1939_local_ecu_get(struct j1939_priv *priv, name_t name, u8 sa);
+void j1939_local_ecu_put(struct j1939_priv *priv, name_t name, u8 sa);
+
+static inline bool j1939_address_is_unicast(u8 addr)
+{
+ return addr <= J1939_MAX_UNICAST_ADDR;
+}
+
+static inline bool j1939_address_is_idle(u8 addr)
+{
+ return addr == J1939_IDLE_ADDR;
+}
+
+static inline bool j1939_address_is_valid(u8 addr)
+{
+ return addr != J1939_NO_ADDR;
+}
+
+static inline bool j1939_pgn_is_pdu1(pgn_t pgn)
+{
+ /* ignore dp & res bits for this */
+ return (pgn & 0xff00) < 0xf000;
+}
+
+/* utility to correctly unmap an ECU */
+void j1939_ecu_unmap_locked(struct j1939_ecu *ecu);
+void j1939_ecu_unmap(struct j1939_ecu *ecu);
+
+u8 j1939_name_to_addr(struct j1939_priv *priv, name_t name);
+struct j1939_ecu *j1939_ecu_find_by_addr_locked(struct j1939_priv *priv,
+ u8 addr);
+struct j1939_ecu *j1939_ecu_get_by_addr(struct j1939_priv *priv, u8 addr);
+struct j1939_ecu *j1939_ecu_get_by_addr_locked(struct j1939_priv *priv,
+ u8 addr);
+struct j1939_ecu *j1939_ecu_get_by_name(struct j1939_priv *priv, name_t name);
+struct j1939_ecu *j1939_ecu_get_by_name_locked(struct j1939_priv *priv,
+ name_t name);
+
+enum j1939_transfer_type {
+ J1939_TP,
+ J1939_ETP,
+ J1939_SIMPLE,
+};
+
+struct j1939_addr {
+ name_t src_name;
+ name_t dst_name;
+ pgn_t pgn;
+
+ u8 sa;
+ u8 da;
+
+ u8 type;
+};
+
+/* control buffer of the sk_buff */
+struct j1939_sk_buff_cb {
+ /* Offset in bytes within one ETP session */
+ u32 offset;
+
+ /* for tx, MSG_SYN will be used to sync on sockets */
+ u32 msg_flags;
+ u32 tskey;
+
+ struct j1939_addr addr;
+
+ /* Flags for quick lookups during skb processing.
+ * These are set in the receive path only.
+ */
+#define J1939_ECU_LOCAL_SRC BIT(0)
+#define J1939_ECU_LOCAL_DST BIT(1)
+ u8 flags;
+
+ priority_t priority;
+};
+
+static inline
+struct j1939_sk_buff_cb *j1939_skb_to_cb(const struct sk_buff *skb)
+{
+ BUILD_BUG_ON(sizeof(struct j1939_sk_buff_cb) > sizeof(skb->cb));
+
+ return (struct j1939_sk_buff_cb *)skb->cb;
+}
+
+int j1939_send_one(struct j1939_priv *priv, struct sk_buff *skb);
+void j1939_sk_recv(struct j1939_priv *priv, struct sk_buff *skb);
+bool j1939_sk_recv_match(struct j1939_priv *priv,
+ struct j1939_sk_buff_cb *skcb);
+void j1939_sk_send_loop_abort(struct sock *sk, int err);
+void j1939_sk_errqueue(struct j1939_session *session,
+ enum j1939_sk_errqueue_type type);
+void j1939_sk_queue_activate_next(struct j1939_session *session);
+
+/* stack entries */
+struct j1939_session *j1939_tp_send(struct j1939_priv *priv,
+ struct sk_buff *skb, size_t size);
+int j1939_tp_recv(struct j1939_priv *priv, struct sk_buff *skb);
+int j1939_ac_fixup(struct j1939_priv *priv, struct sk_buff *skb);
+void j1939_ac_recv(struct j1939_priv *priv, struct sk_buff *skb);
+void j1939_simple_recv(struct j1939_priv *priv, struct sk_buff *skb);
+
+/* network management */
+struct j1939_ecu *j1939_ecu_create_locked(struct j1939_priv *priv, name_t name);
+
+void j1939_ecu_timer_start(struct j1939_ecu *ecu);
+void j1939_ecu_timer_cancel(struct j1939_ecu *ecu);
+void j1939_ecu_unmap_all(struct j1939_priv *priv);
+
+struct j1939_priv *j1939_netdev_start(struct net_device *ndev);
+void j1939_netdev_stop(struct j1939_priv *priv);
+
+void j1939_priv_put(struct j1939_priv *priv);
+void j1939_priv_get(struct j1939_priv *priv);
+
+/* notify/alert all j1939 sockets bound to ifindex */
+void j1939_sk_netdev_event_netdown(struct j1939_priv *priv);
+int j1939_cancel_active_session(struct j1939_priv *priv, struct sock *sk);
+void j1939_tp_init(struct j1939_priv *priv);
+
+/* decrement pending skb for a j1939 socket */
+void j1939_sock_pending_del(struct sock *sk);
+
+enum j1939_session_state {
+ J1939_SESSION_NEW,
+ J1939_SESSION_ACTIVE,
+ /* waiting for abort signal on the bus */
+ J1939_SESSION_WAITING_ABORT,
+ J1939_SESSION_ACTIVE_MAX,
+ J1939_SESSION_DONE,
+};
+
+struct j1939_session {
+ struct j1939_priv *priv;
+ struct list_head active_session_list_entry;
+ struct list_head sk_session_queue_entry;
+ struct kref kref;
+ struct sock *sk;
+
+ /* ifindex, src, dst, pgn define the session block
+ * the are _never_ modified after insertion in the list
+ * this decreases locking problems a _lot_
+ */
+ struct j1939_sk_buff_cb skcb;
+ struct sk_buff_head skb_queue;
+
+ /* all tx related stuff (last_txcmd, pkt.tx)
+ * is protected (modified only) with the txtimer hrtimer
+ * 'total' & 'block' are never changed,
+ * last_cmd, last & block are protected by ->lock
+ * this means that the tx may run after cts is received that should
+ * have stopped tx, but this time discrepancy is never avoided anyhow
+ */
+ u8 last_cmd, last_txcmd;
+ bool transmission;
+ bool extd;
+ /* Total message size, number of bytes */
+ unsigned int total_message_size;
+ /* Total number of bytes queue from socket to the session */
+ unsigned int total_queued_size;
+ unsigned int tx_retry;
+
+ int err;
+ u32 tskey;
+ enum j1939_session_state state;
+
+ /* Packets counters for a (extended) transfer session. The packet is
+ * maximal of 7 bytes.
+ */
+ struct {
+ /* total - total number of packets for this session */
+ unsigned int total;
+ /* last - last packet of a transfer block after which
+ * responder should send ETP.CM_CTS and originator
+ * ETP.CM_DPO
+ */
+ unsigned int last;
+ /* tx - number of packets send by originator node.
+ * this counter can be set back if responder node
+ * didn't received all packets send by originator.
+ */
+ unsigned int tx;
+ unsigned int tx_acked;
+ /* rx - number of packets received */
+ unsigned int rx;
+ /* block - amount of packets expected in one block */
+ unsigned int block;
+ /* dpo - ETP.CM_DPO, Data Packet Offset */
+ unsigned int dpo;
+ } pkt;
+ struct hrtimer txtimer, rxtimer;
+};
+
+struct j1939_sock {
+ struct sock sk; /* must be first to skip with memset */
+ struct j1939_priv *priv;
+ struct list_head list;
+
+#define J1939_SOCK_BOUND BIT(0)
+#define J1939_SOCK_CONNECTED BIT(1)
+#define J1939_SOCK_PROMISC BIT(2)
+#define J1939_SOCK_ERRQUEUE BIT(3)
+ int state;
+
+ int ifindex;
+ struct j1939_addr addr;
+ struct j1939_filter *filters;
+ int nfilters;
+ pgn_t pgn_rx_filter;
+
+ /* j1939 may emit equal PGN (!= equal CAN-id's) out of order
+ * when transport protocol comes in.
+ * To allow emitting in order, keep a 'pending' nr. of packets
+ */
+ atomic_t skb_pending;
+ wait_queue_head_t waitq;
+
+ /* lock for the sk_session_queue list */
+ spinlock_t sk_session_queue_lock;
+ struct list_head sk_session_queue;
+};
+
+static inline struct j1939_sock *j1939_sk(const struct sock *sk)
+{
+ return container_of(sk, struct j1939_sock, sk);
+}
+
+void j1939_session_get(struct j1939_session *session);
+void j1939_session_put(struct j1939_session *session);
+void j1939_session_skb_queue(struct j1939_session *session,
+ struct sk_buff *skb);
+int j1939_session_activate(struct j1939_session *session);
+void j1939_tp_schedule_txtimer(struct j1939_session *session, int msec);
+void j1939_session_timers_cancel(struct j1939_session *session);
+
+#define J1939_MAX_TP_PACKET_SIZE (7 * 0xff)
+#define J1939_MAX_ETP_PACKET_SIZE (7 * 0x00ffffff)
+
+#define J1939_REGULAR 0
+#define J1939_EXTENDED 1
+
+/* CAN protocol */
+extern const struct can_proto j1939_can_proto;
+
+#endif /* _J1939_PRIV_H_ */
diff --git a/net/can/j1939/main.c b/net/can/j1939/main.c
new file mode 100644
index 0000000..def2f81
--- /dev/null
+++ b/net/can/j1939/main.c
@@ -0,0 +1,403 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2010-2011 EIA Electronics,
+// Pieter Beyens <pieter.beyens@eia.be>
+// Copyright (c) 2010-2011 EIA Electronics,
+// Kurt Van Dijck <kurt.van.dijck@eia.be>
+// Copyright (c) 2018 Protonic,
+// Robin van der Gracht <robin@protonic.nl>
+// Copyright (c) 2017-2019 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2017-2019 Pengutronix,
+// Oleksij Rempel <kernel@pengutronix.de>
+
+/* Core of can-j1939 that links j1939 to CAN. */
+
+#include <linux/can/can-ml.h>
+#include <linux/can/core.h>
+#include <linux/can/skb.h>
+#include <linux/if_arp.h>
+#include <linux/module.h>
+
+#include "j1939-priv.h"
+
+MODULE_DESCRIPTION("PF_CAN SAE J1939");
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("EIA Electronics (Kurt Van Dijck & Pieter Beyens)");
+MODULE_ALIAS("can-proto-" __stringify(CAN_J1939));
+
+/* LOWLEVEL CAN interface */
+
+/* CAN_HDR: #bytes before can_frame data part */
+#define J1939_CAN_HDR (offsetof(struct can_frame, data))
+
+/* CAN_FTR: #bytes beyond data part */
+#define J1939_CAN_FTR (sizeof(struct can_frame) - J1939_CAN_HDR - \
+ sizeof(((struct can_frame *)0)->data))
+
+/* lowest layer */
+static void j1939_can_recv(struct sk_buff *iskb, void *data)
+{
+ struct j1939_priv *priv = data;
+ struct sk_buff *skb;
+ struct j1939_sk_buff_cb *skcb, *iskcb;
+ struct can_frame *cf;
+
+ /* create a copy of the skb
+ * j1939 only delivers the real data bytes,
+ * the header goes into sockaddr.
+ * j1939 may not touch the incoming skb in such way
+ */
+ skb = skb_clone(iskb, GFP_ATOMIC);
+ if (!skb)
+ return;
+
+ can_skb_set_owner(skb, iskb->sk);
+
+ /* get a pointer to the header of the skb
+ * the skb payload (pointer) is moved, so that the next skb_data
+ * returns the actual payload
+ */
+ cf = (void *)skb->data;
+ skb_pull(skb, J1939_CAN_HDR);
+
+ /* fix length, set to dlc, with 8 maximum */
+ skb_trim(skb, min_t(uint8_t, cf->can_dlc, 8));
+
+ /* set addr */
+ skcb = j1939_skb_to_cb(skb);
+ memset(skcb, 0, sizeof(*skcb));
+
+ iskcb = j1939_skb_to_cb(iskb);
+ skcb->tskey = iskcb->tskey;
+ skcb->priority = (cf->can_id >> 26) & 0x7;
+ skcb->addr.sa = cf->can_id;
+ skcb->addr.pgn = (cf->can_id >> 8) & J1939_PGN_MAX;
+ /* set default message type */
+ skcb->addr.type = J1939_TP;
+ if (j1939_pgn_is_pdu1(skcb->addr.pgn)) {
+ /* Type 1: with destination address */
+ skcb->addr.da = skcb->addr.pgn;
+ /* normalize pgn: strip dst address */
+ skcb->addr.pgn &= 0x3ff00;
+ } else {
+ /* set broadcast address */
+ skcb->addr.da = J1939_NO_ADDR;
+ }
+
+ /* update localflags */
+ read_lock_bh(&priv->lock);
+ if (j1939_address_is_unicast(skcb->addr.sa) &&
+ priv->ents[skcb->addr.sa].nusers)
+ skcb->flags |= J1939_ECU_LOCAL_SRC;
+ if (j1939_address_is_unicast(skcb->addr.da) &&
+ priv->ents[skcb->addr.da].nusers)
+ skcb->flags |= J1939_ECU_LOCAL_DST;
+ read_unlock_bh(&priv->lock);
+
+ /* deliver into the j1939 stack ... */
+ j1939_ac_recv(priv, skb);
+
+ if (j1939_tp_recv(priv, skb))
+ /* this means the transport layer processed the message */
+ goto done;
+
+ j1939_simple_recv(priv, skb);
+ j1939_sk_recv(priv, skb);
+ done:
+ kfree_skb(skb);
+}
+
+/* NETDEV MANAGEMENT */
+
+/* values for can_rx_(un)register */
+#define J1939_CAN_ID CAN_EFF_FLAG
+#define J1939_CAN_MASK (CAN_EFF_FLAG | CAN_RTR_FLAG)
+
+static DEFINE_SPINLOCK(j1939_netdev_lock);
+
+static struct j1939_priv *j1939_priv_create(struct net_device *ndev)
+{
+ struct j1939_priv *priv;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return NULL;
+
+ rwlock_init(&priv->lock);
+ INIT_LIST_HEAD(&priv->ecus);
+ priv->ndev = ndev;
+ kref_init(&priv->kref);
+ kref_init(&priv->rx_kref);
+ dev_hold(ndev);
+
+ netdev_dbg(priv->ndev, "%s : 0x%p\n", __func__, priv);
+
+ return priv;
+}
+
+static inline void j1939_priv_set(struct net_device *ndev,
+ struct j1939_priv *priv)
+{
+ struct can_ml_priv *can_ml_priv = ndev->ml_priv;
+
+ can_ml_priv->j1939_priv = priv;
+}
+
+static void __j1939_priv_release(struct kref *kref)
+{
+ struct j1939_priv *priv = container_of(kref, struct j1939_priv, kref);
+ struct net_device *ndev = priv->ndev;
+
+ netdev_dbg(priv->ndev, "%s: 0x%p\n", __func__, priv);
+
+ dev_put(ndev);
+ kfree(priv);
+}
+
+void j1939_priv_put(struct j1939_priv *priv)
+{
+ kref_put(&priv->kref, __j1939_priv_release);
+}
+
+void j1939_priv_get(struct j1939_priv *priv)
+{
+ kref_get(&priv->kref);
+}
+
+static int j1939_can_rx_register(struct j1939_priv *priv)
+{
+ struct net_device *ndev = priv->ndev;
+ int ret;
+
+ j1939_priv_get(priv);
+ ret = can_rx_register(dev_net(ndev), ndev, J1939_CAN_ID, J1939_CAN_MASK,
+ j1939_can_recv, priv, "j1939", NULL);
+ if (ret < 0) {
+ j1939_priv_put(priv);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void j1939_can_rx_unregister(struct j1939_priv *priv)
+{
+ struct net_device *ndev = priv->ndev;
+
+ can_rx_unregister(dev_net(ndev), ndev, J1939_CAN_ID, J1939_CAN_MASK,
+ j1939_can_recv, priv);
+
+ j1939_priv_put(priv);
+}
+
+static void __j1939_rx_release(struct kref *kref)
+ __releases(&j1939_netdev_lock)
+{
+ struct j1939_priv *priv = container_of(kref, struct j1939_priv,
+ rx_kref);
+
+ j1939_can_rx_unregister(priv);
+ j1939_ecu_unmap_all(priv);
+ j1939_priv_set(priv->ndev, NULL);
+ spin_unlock(&j1939_netdev_lock);
+}
+
+/* get pointer to priv without increasing ref counter */
+static inline struct j1939_priv *j1939_ndev_to_priv(struct net_device *ndev)
+{
+ struct can_ml_priv *can_ml_priv = ndev->ml_priv;
+
+ return can_ml_priv->j1939_priv;
+}
+
+static struct j1939_priv *j1939_priv_get_by_ndev_locked(struct net_device *ndev)
+{
+ struct j1939_priv *priv;
+
+ lockdep_assert_held(&j1939_netdev_lock);
+
+ if (ndev->type != ARPHRD_CAN)
+ return NULL;
+
+ priv = j1939_ndev_to_priv(ndev);
+ if (priv)
+ j1939_priv_get(priv);
+
+ return priv;
+}
+
+static struct j1939_priv *j1939_priv_get_by_ndev(struct net_device *ndev)
+{
+ struct j1939_priv *priv;
+
+ spin_lock(&j1939_netdev_lock);
+ priv = j1939_priv_get_by_ndev_locked(ndev);
+ spin_unlock(&j1939_netdev_lock);
+
+ return priv;
+}
+
+struct j1939_priv *j1939_netdev_start(struct net_device *ndev)
+{
+ struct j1939_priv *priv, *priv_new;
+ int ret;
+
+ priv = j1939_priv_get_by_ndev(ndev);
+ if (priv) {
+ kref_get(&priv->rx_kref);
+ return priv;
+ }
+
+ priv = j1939_priv_create(ndev);
+ if (!priv)
+ return ERR_PTR(-ENOMEM);
+
+ j1939_tp_init(priv);
+ spin_lock_init(&priv->j1939_socks_lock);
+ INIT_LIST_HEAD(&priv->j1939_socks);
+
+ spin_lock(&j1939_netdev_lock);
+ priv_new = j1939_priv_get_by_ndev_locked(ndev);
+ if (priv_new) {
+ /* Someone was faster than us, use their priv and roll
+ * back our's.
+ */
+ spin_unlock(&j1939_netdev_lock);
+ dev_put(ndev);
+ kfree(priv);
+ kref_get(&priv_new->rx_kref);
+ return priv_new;
+ }
+ j1939_priv_set(ndev, priv);
+ spin_unlock(&j1939_netdev_lock);
+
+ ret = j1939_can_rx_register(priv);
+ if (ret < 0)
+ goto out_priv_put;
+
+ return priv;
+
+ out_priv_put:
+ j1939_priv_set(ndev, NULL);
+ dev_put(ndev);
+ kfree(priv);
+
+ return ERR_PTR(ret);
+}
+
+void j1939_netdev_stop(struct j1939_priv *priv)
+{
+ kref_put_lock(&priv->rx_kref, __j1939_rx_release, &j1939_netdev_lock);
+ j1939_priv_put(priv);
+}
+
+int j1939_send_one(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ int ret, dlc;
+ canid_t canid;
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct can_frame *cf;
+
+ /* apply sanity checks */
+ if (j1939_pgn_is_pdu1(skcb->addr.pgn))
+ skcb->addr.pgn &= J1939_PGN_PDU1_MAX;
+ else
+ skcb->addr.pgn &= J1939_PGN_MAX;
+
+ if (skcb->priority > 7)
+ skcb->priority = 6;
+
+ ret = j1939_ac_fixup(priv, skb);
+ if (unlikely(ret))
+ goto failed;
+ dlc = skb->len;
+
+ /* re-claim the CAN_HDR from the SKB */
+ cf = skb_push(skb, J1939_CAN_HDR);
+
+ /* make it a full can frame again */
+ skb_put(skb, J1939_CAN_FTR + (8 - dlc));
+
+ canid = CAN_EFF_FLAG |
+ (skcb->priority << 26) |
+ (skcb->addr.pgn << 8) |
+ skcb->addr.sa;
+ if (j1939_pgn_is_pdu1(skcb->addr.pgn))
+ canid |= skcb->addr.da << 8;
+
+ cf->can_id = canid;
+ cf->can_dlc = dlc;
+
+ return can_send(skb, 1);
+
+ failed:
+ kfree_skb(skb);
+ return ret;
+}
+
+static int j1939_netdev_notify(struct notifier_block *nb,
+ unsigned long msg, void *data)
+{
+ struct net_device *ndev = netdev_notifier_info_to_dev(data);
+ struct j1939_priv *priv;
+
+ priv = j1939_priv_get_by_ndev(ndev);
+ if (!priv)
+ goto notify_done;
+
+ if (ndev->type != ARPHRD_CAN)
+ goto notify_put;
+
+ switch (msg) {
+ case NETDEV_DOWN:
+ j1939_cancel_active_session(priv, NULL);
+ j1939_sk_netdev_event_netdown(priv);
+ j1939_ecu_unmap_all(priv);
+ break;
+ }
+
+notify_put:
+ j1939_priv_put(priv);
+
+notify_done:
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block j1939_netdev_notifier = {
+ .notifier_call = j1939_netdev_notify,
+};
+
+/* MODULE interface */
+static __init int j1939_module_init(void)
+{
+ int ret;
+
+ pr_info("can: SAE J1939\n");
+
+ ret = register_netdevice_notifier(&j1939_netdev_notifier);
+ if (ret)
+ goto fail_notifier;
+
+ ret = can_proto_register(&j1939_can_proto);
+ if (ret < 0) {
+ pr_err("can: registration of j1939 protocol failed\n");
+ goto fail_sk;
+ }
+
+ return 0;
+
+ fail_sk:
+ unregister_netdevice_notifier(&j1939_netdev_notifier);
+ fail_notifier:
+ return ret;
+}
+
+static __exit void j1939_module_exit(void)
+{
+ can_proto_unregister(&j1939_can_proto);
+
+ unregister_netdevice_notifier(&j1939_netdev_notifier);
+}
+
+module_init(j1939_module_init);
+module_exit(j1939_module_exit);
diff --git a/net/can/j1939/socket.c b/net/can/j1939/socket.c
new file mode 100644
index 0000000..37c1040
--- /dev/null
+++ b/net/can/j1939/socket.c
@@ -0,0 +1,1160 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2010-2011 EIA Electronics,
+// Pieter Beyens <pieter.beyens@eia.be>
+// Copyright (c) 2010-2011 EIA Electronics,
+// Kurt Van Dijck <kurt.van.dijck@eia.be>
+// Copyright (c) 2018 Protonic,
+// Robin van der Gracht <robin@protonic.nl>
+// Copyright (c) 2017-2019 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2017-2019 Pengutronix,
+// Oleksij Rempel <kernel@pengutronix.de>
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/can/core.h>
+#include <linux/can/skb.h>
+#include <linux/errqueue.h>
+#include <linux/if_arp.h>
+
+#include "j1939-priv.h"
+
+#define J1939_MIN_NAMELEN CAN_REQUIRED_SIZE(struct sockaddr_can, can_addr.j1939)
+
+/* conversion function between struct sock::sk_priority from linux and
+ * j1939 priority field
+ */
+static inline priority_t j1939_prio(u32 sk_priority)
+{
+ sk_priority = min(sk_priority, 7U);
+
+ return 7 - sk_priority;
+}
+
+static inline u32 j1939_to_sk_priority(priority_t prio)
+{
+ return 7 - prio;
+}
+
+/* function to see if pgn is to be evaluated */
+static inline bool j1939_pgn_is_valid(pgn_t pgn)
+{
+ return pgn <= J1939_PGN_MAX;
+}
+
+/* test function to avoid non-zero DA placeholder for pdu1 pgn's */
+static inline bool j1939_pgn_is_clean_pdu(pgn_t pgn)
+{
+ if (j1939_pgn_is_pdu1(pgn))
+ return !(pgn & 0xff);
+ else
+ return true;
+}
+
+static inline void j1939_sock_pending_add(struct sock *sk)
+{
+ struct j1939_sock *jsk = j1939_sk(sk);
+
+ atomic_inc(&jsk->skb_pending);
+}
+
+static int j1939_sock_pending_get(struct sock *sk)
+{
+ struct j1939_sock *jsk = j1939_sk(sk);
+
+ return atomic_read(&jsk->skb_pending);
+}
+
+void j1939_sock_pending_del(struct sock *sk)
+{
+ struct j1939_sock *jsk = j1939_sk(sk);
+
+ /* atomic_dec_return returns the new value */
+ if (!atomic_dec_return(&jsk->skb_pending))
+ wake_up(&jsk->waitq); /* no pending SKB's */
+}
+
+static void j1939_jsk_add(struct j1939_priv *priv, struct j1939_sock *jsk)
+{
+ jsk->state |= J1939_SOCK_BOUND;
+ j1939_priv_get(priv);
+ jsk->priv = priv;
+
+ spin_lock_bh(&priv->j1939_socks_lock);
+ list_add_tail(&jsk->list, &priv->j1939_socks);
+ spin_unlock_bh(&priv->j1939_socks_lock);
+}
+
+static void j1939_jsk_del(struct j1939_priv *priv, struct j1939_sock *jsk)
+{
+ spin_lock_bh(&priv->j1939_socks_lock);
+ list_del_init(&jsk->list);
+ spin_unlock_bh(&priv->j1939_socks_lock);
+
+ jsk->priv = NULL;
+ j1939_priv_put(priv);
+ jsk->state &= ~J1939_SOCK_BOUND;
+}
+
+static bool j1939_sk_queue_session(struct j1939_session *session)
+{
+ struct j1939_sock *jsk = j1939_sk(session->sk);
+ bool empty;
+
+ spin_lock_bh(&jsk->sk_session_queue_lock);
+ empty = list_empty(&jsk->sk_session_queue);
+ j1939_session_get(session);
+ list_add_tail(&session->sk_session_queue_entry, &jsk->sk_session_queue);
+ spin_unlock_bh(&jsk->sk_session_queue_lock);
+ j1939_sock_pending_add(&jsk->sk);
+
+ return empty;
+}
+
+static struct
+j1939_session *j1939_sk_get_incomplete_session(struct j1939_sock *jsk)
+{
+ struct j1939_session *session = NULL;
+
+ spin_lock_bh(&jsk->sk_session_queue_lock);
+ if (!list_empty(&jsk->sk_session_queue)) {
+ session = list_last_entry(&jsk->sk_session_queue,
+ struct j1939_session,
+ sk_session_queue_entry);
+ if (session->total_queued_size == session->total_message_size)
+ session = NULL;
+ else
+ j1939_session_get(session);
+ }
+ spin_unlock_bh(&jsk->sk_session_queue_lock);
+
+ return session;
+}
+
+static void j1939_sk_queue_drop_all(struct j1939_priv *priv,
+ struct j1939_sock *jsk, int err)
+{
+ struct j1939_session *session, *tmp;
+
+ netdev_dbg(priv->ndev, "%s: err: %i\n", __func__, err);
+ spin_lock_bh(&jsk->sk_session_queue_lock);
+ list_for_each_entry_safe(session, tmp, &jsk->sk_session_queue,
+ sk_session_queue_entry) {
+ list_del_init(&session->sk_session_queue_entry);
+ session->err = err;
+ j1939_session_put(session);
+ }
+ spin_unlock_bh(&jsk->sk_session_queue_lock);
+}
+
+static void j1939_sk_queue_activate_next_locked(struct j1939_session *session)
+{
+ struct j1939_sock *jsk;
+ struct j1939_session *first;
+ int err;
+
+ /* RX-Session don't have a socket (yet) */
+ if (!session->sk)
+ return;
+
+ jsk = j1939_sk(session->sk);
+ lockdep_assert_held(&jsk->sk_session_queue_lock);
+
+ err = session->err;
+
+ first = list_first_entry_or_null(&jsk->sk_session_queue,
+ struct j1939_session,
+ sk_session_queue_entry);
+
+ /* Some else has already activated the next session */
+ if (first != session)
+ return;
+
+activate_next:
+ list_del_init(&first->sk_session_queue_entry);
+ j1939_session_put(first);
+ first = list_first_entry_or_null(&jsk->sk_session_queue,
+ struct j1939_session,
+ sk_session_queue_entry);
+ if (!first)
+ return;
+
+ if (WARN_ON_ONCE(j1939_session_activate(first))) {
+ first->err = -EBUSY;
+ goto activate_next;
+ } else {
+ /* Give receiver some time (arbitrary chosen) to recover */
+ int time_ms = 0;
+
+ if (err)
+ time_ms = 10 + prandom_u32_max(16);
+
+ j1939_tp_schedule_txtimer(first, time_ms);
+ }
+}
+
+void j1939_sk_queue_activate_next(struct j1939_session *session)
+{
+ struct j1939_sock *jsk;
+
+ if (!session->sk)
+ return;
+
+ jsk = j1939_sk(session->sk);
+
+ spin_lock_bh(&jsk->sk_session_queue_lock);
+ j1939_sk_queue_activate_next_locked(session);
+ spin_unlock_bh(&jsk->sk_session_queue_lock);
+}
+
+static bool j1939_sk_match_dst(struct j1939_sock *jsk,
+ const struct j1939_sk_buff_cb *skcb)
+{
+ if ((jsk->state & J1939_SOCK_PROMISC))
+ return true;
+
+ /* Destination address filter */
+ if (jsk->addr.src_name && skcb->addr.dst_name) {
+ if (jsk->addr.src_name != skcb->addr.dst_name)
+ return false;
+ } else {
+ /* receive (all sockets) if
+ * - all packages that match our bind() address
+ * - all broadcast on a socket if SO_BROADCAST
+ * is set
+ */
+ if (j1939_address_is_unicast(skcb->addr.da)) {
+ if (jsk->addr.sa != skcb->addr.da)
+ return false;
+ } else if (!sock_flag(&jsk->sk, SOCK_BROADCAST)) {
+ /* receiving broadcast without SO_BROADCAST
+ * flag is not allowed
+ */
+ return false;
+ }
+ }
+
+ /* Source address filter */
+ if (jsk->state & J1939_SOCK_CONNECTED) {
+ /* receive (all sockets) if
+ * - all packages that match our connect() name or address
+ */
+ if (jsk->addr.dst_name && skcb->addr.src_name) {
+ if (jsk->addr.dst_name != skcb->addr.src_name)
+ return false;
+ } else {
+ if (jsk->addr.da != skcb->addr.sa)
+ return false;
+ }
+ }
+
+ /* PGN filter */
+ if (j1939_pgn_is_valid(jsk->pgn_rx_filter) &&
+ jsk->pgn_rx_filter != skcb->addr.pgn)
+ return false;
+
+ return true;
+}
+
+/* matches skb control buffer (addr) with a j1939 filter */
+static bool j1939_sk_match_filter(struct j1939_sock *jsk,
+ const struct j1939_sk_buff_cb *skcb)
+{
+ const struct j1939_filter *f = jsk->filters;
+ int nfilter = jsk->nfilters;
+
+ if (!nfilter)
+ /* receive all when no filters are assigned */
+ return true;
+
+ for (; nfilter; ++f, --nfilter) {
+ if ((skcb->addr.pgn & f->pgn_mask) != f->pgn)
+ continue;
+ if ((skcb->addr.sa & f->addr_mask) != f->addr)
+ continue;
+ if ((skcb->addr.src_name & f->name_mask) != f->name)
+ continue;
+ return true;
+ }
+ return false;
+}
+
+static bool j1939_sk_recv_match_one(struct j1939_sock *jsk,
+ const struct j1939_sk_buff_cb *skcb)
+{
+ if (!(jsk->state & J1939_SOCK_BOUND))
+ return false;
+
+ if (!j1939_sk_match_dst(jsk, skcb))
+ return false;
+
+ if (!j1939_sk_match_filter(jsk, skcb))
+ return false;
+
+ return true;
+}
+
+static void j1939_sk_recv_one(struct j1939_sock *jsk, struct sk_buff *oskb)
+{
+ const struct j1939_sk_buff_cb *oskcb = j1939_skb_to_cb(oskb);
+ struct j1939_sk_buff_cb *skcb;
+ struct sk_buff *skb;
+
+ if (oskb->sk == &jsk->sk)
+ return;
+
+ if (!j1939_sk_recv_match_one(jsk, oskcb))
+ return;
+
+ skb = skb_clone(oskb, GFP_ATOMIC);
+ if (!skb) {
+ pr_warn("skb clone failed\n");
+ return;
+ }
+ can_skb_set_owner(skb, oskb->sk);
+
+ skcb = j1939_skb_to_cb(skb);
+ skcb->msg_flags &= ~(MSG_DONTROUTE);
+ if (skb->sk)
+ skcb->msg_flags |= MSG_DONTROUTE;
+
+ if (sock_queue_rcv_skb(&jsk->sk, skb) < 0)
+ kfree_skb(skb);
+}
+
+bool j1939_sk_recv_match(struct j1939_priv *priv, struct j1939_sk_buff_cb *skcb)
+{
+ struct j1939_sock *jsk;
+ bool match = false;
+
+ spin_lock_bh(&priv->j1939_socks_lock);
+ list_for_each_entry(jsk, &priv->j1939_socks, list) {
+ match = j1939_sk_recv_match_one(jsk, skcb);
+ if (match)
+ break;
+ }
+ spin_unlock_bh(&priv->j1939_socks_lock);
+
+ return match;
+}
+
+void j1939_sk_recv(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sock *jsk;
+
+ spin_lock_bh(&priv->j1939_socks_lock);
+ list_for_each_entry(jsk, &priv->j1939_socks, list) {
+ j1939_sk_recv_one(jsk, skb);
+ }
+ spin_unlock_bh(&priv->j1939_socks_lock);
+}
+
+static int j1939_sk_init(struct sock *sk)
+{
+ struct j1939_sock *jsk = j1939_sk(sk);
+
+ /* Ensure that "sk" is first member in "struct j1939_sock", so that we
+ * can skip it during memset().
+ */
+ BUILD_BUG_ON(offsetof(struct j1939_sock, sk) != 0);
+ memset((void *)jsk + sizeof(jsk->sk), 0x0,
+ sizeof(*jsk) - sizeof(jsk->sk));
+
+ INIT_LIST_HEAD(&jsk->list);
+ init_waitqueue_head(&jsk->waitq);
+ jsk->sk.sk_priority = j1939_to_sk_priority(6);
+ jsk->sk.sk_reuse = 1; /* per default */
+ jsk->addr.sa = J1939_NO_ADDR;
+ jsk->addr.da = J1939_NO_ADDR;
+ jsk->addr.pgn = J1939_NO_PGN;
+ jsk->pgn_rx_filter = J1939_NO_PGN;
+ atomic_set(&jsk->skb_pending, 0);
+ spin_lock_init(&jsk->sk_session_queue_lock);
+ INIT_LIST_HEAD(&jsk->sk_session_queue);
+
+ return 0;
+}
+
+static int j1939_sk_sanity_check(struct sockaddr_can *addr, int len)
+{
+ if (!addr)
+ return -EDESTADDRREQ;
+ if (len < J1939_MIN_NAMELEN)
+ return -EINVAL;
+ if (addr->can_family != AF_CAN)
+ return -EINVAL;
+ if (!addr->can_ifindex)
+ return -ENODEV;
+ if (j1939_pgn_is_valid(addr->can_addr.j1939.pgn) &&
+ !j1939_pgn_is_clean_pdu(addr->can_addr.j1939.pgn))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int j1939_sk_bind(struct socket *sock, struct sockaddr *uaddr, int len)
+{
+ struct sockaddr_can *addr = (struct sockaddr_can *)uaddr;
+ struct j1939_sock *jsk = j1939_sk(sock->sk);
+ struct j1939_priv *priv = jsk->priv;
+ struct sock *sk = sock->sk;
+ struct net *net = sock_net(sk);
+ int ret = 0;
+
+ ret = j1939_sk_sanity_check(addr, len);
+ if (ret)
+ return ret;
+
+ lock_sock(sock->sk);
+
+ /* Already bound to an interface? */
+ if (jsk->state & J1939_SOCK_BOUND) {
+ /* A re-bind() to a different interface is not
+ * supported.
+ */
+ if (jsk->ifindex != addr->can_ifindex) {
+ ret = -EINVAL;
+ goto out_release_sock;
+ }
+
+ /* drop old references */
+ j1939_jsk_del(priv, jsk);
+ j1939_local_ecu_put(priv, jsk->addr.src_name, jsk->addr.sa);
+ } else {
+ struct net_device *ndev;
+
+ ndev = dev_get_by_index(net, addr->can_ifindex);
+ if (!ndev) {
+ ret = -ENODEV;
+ goto out_release_sock;
+ }
+
+ if (ndev->type != ARPHRD_CAN) {
+ dev_put(ndev);
+ ret = -ENODEV;
+ goto out_release_sock;
+ }
+
+ priv = j1939_netdev_start(ndev);
+ dev_put(ndev);
+ if (IS_ERR(priv)) {
+ ret = PTR_ERR(priv);
+ goto out_release_sock;
+ }
+
+ jsk->ifindex = addr->can_ifindex;
+ }
+
+ /* set default transmit pgn */
+ if (j1939_pgn_is_valid(addr->can_addr.j1939.pgn))
+ jsk->pgn_rx_filter = addr->can_addr.j1939.pgn;
+ jsk->addr.src_name = addr->can_addr.j1939.name;
+ jsk->addr.sa = addr->can_addr.j1939.addr;
+
+ /* get new references */
+ ret = j1939_local_ecu_get(priv, jsk->addr.src_name, jsk->addr.sa);
+ if (ret) {
+ j1939_netdev_stop(priv);
+ goto out_release_sock;
+ }
+
+ j1939_jsk_add(priv, jsk);
+
+ out_release_sock: /* fall through */
+ release_sock(sock->sk);
+
+ return ret;
+}
+
+static int j1939_sk_connect(struct socket *sock, struct sockaddr *uaddr,
+ int len, int flags)
+{
+ struct sockaddr_can *addr = (struct sockaddr_can *)uaddr;
+ struct j1939_sock *jsk = j1939_sk(sock->sk);
+ int ret = 0;
+
+ ret = j1939_sk_sanity_check(addr, len);
+ if (ret)
+ return ret;
+
+ lock_sock(sock->sk);
+
+ /* bind() before connect() is mandatory */
+ if (!(jsk->state & J1939_SOCK_BOUND)) {
+ ret = -EINVAL;
+ goto out_release_sock;
+ }
+
+ /* A connect() to a different interface is not supported. */
+ if (jsk->ifindex != addr->can_ifindex) {
+ ret = -EINVAL;
+ goto out_release_sock;
+ }
+
+ if (!addr->can_addr.j1939.name &&
+ addr->can_addr.j1939.addr == J1939_NO_ADDR &&
+ !sock_flag(&jsk->sk, SOCK_BROADCAST)) {
+ /* broadcast, but SO_BROADCAST not set */
+ ret = -EACCES;
+ goto out_release_sock;
+ }
+
+ jsk->addr.dst_name = addr->can_addr.j1939.name;
+ jsk->addr.da = addr->can_addr.j1939.addr;
+
+ if (j1939_pgn_is_valid(addr->can_addr.j1939.pgn))
+ jsk->addr.pgn = addr->can_addr.j1939.pgn;
+
+ jsk->state |= J1939_SOCK_CONNECTED;
+
+ out_release_sock: /* fall through */
+ release_sock(sock->sk);
+
+ return ret;
+}
+
+static void j1939_sk_sock2sockaddr_can(struct sockaddr_can *addr,
+ const struct j1939_sock *jsk, int peer)
+{
+ addr->can_family = AF_CAN;
+ addr->can_ifindex = jsk->ifindex;
+ addr->can_addr.j1939.pgn = jsk->addr.pgn;
+ if (peer) {
+ addr->can_addr.j1939.name = jsk->addr.dst_name;
+ addr->can_addr.j1939.addr = jsk->addr.da;
+ } else {
+ addr->can_addr.j1939.name = jsk->addr.src_name;
+ addr->can_addr.j1939.addr = jsk->addr.sa;
+ }
+}
+
+static int j1939_sk_getname(struct socket *sock, struct sockaddr *uaddr,
+ int peer)
+{
+ struct sockaddr_can *addr = (struct sockaddr_can *)uaddr;
+ struct sock *sk = sock->sk;
+ struct j1939_sock *jsk = j1939_sk(sk);
+ int ret = 0;
+
+ lock_sock(sk);
+
+ if (peer && !(jsk->state & J1939_SOCK_CONNECTED)) {
+ ret = -EADDRNOTAVAIL;
+ goto failure;
+ }
+
+ j1939_sk_sock2sockaddr_can(addr, jsk, peer);
+ ret = J1939_MIN_NAMELEN;
+
+ failure:
+ release_sock(sk);
+
+ return ret;
+}
+
+static int j1939_sk_release(struct socket *sock)
+{
+ struct sock *sk = sock->sk;
+ struct j1939_sock *jsk;
+
+ if (!sk)
+ return 0;
+
+ jsk = j1939_sk(sk);
+ lock_sock(sk);
+
+ if (jsk->state & J1939_SOCK_BOUND) {
+ struct j1939_priv *priv = jsk->priv;
+
+ if (wait_event_interruptible(jsk->waitq,
+ !j1939_sock_pending_get(&jsk->sk))) {
+ j1939_cancel_active_session(priv, sk);
+ j1939_sk_queue_drop_all(priv, jsk, ESHUTDOWN);
+ }
+
+ j1939_jsk_del(priv, jsk);
+
+ j1939_local_ecu_put(priv, jsk->addr.src_name,
+ jsk->addr.sa);
+
+ j1939_netdev_stop(priv);
+ }
+
+ sock_orphan(sk);
+ sock->sk = NULL;
+
+ release_sock(sk);
+ sock_put(sk);
+
+ return 0;
+}
+
+static int j1939_sk_setsockopt_flag(struct j1939_sock *jsk, char __user *optval,
+ unsigned int optlen, int flag)
+{
+ int tmp;
+
+ if (optlen != sizeof(tmp))
+ return -EINVAL;
+ if (copy_from_user(&tmp, optval, optlen))
+ return -EFAULT;
+ lock_sock(&jsk->sk);
+ if (tmp)
+ jsk->state |= flag;
+ else
+ jsk->state &= ~flag;
+ release_sock(&jsk->sk);
+ return tmp;
+}
+
+static int j1939_sk_setsockopt(struct socket *sock, int level, int optname,
+ char __user *optval, unsigned int optlen)
+{
+ struct sock *sk = sock->sk;
+ struct j1939_sock *jsk = j1939_sk(sk);
+ int tmp, count = 0, ret = 0;
+ struct j1939_filter *filters = NULL, *ofilters;
+
+ if (level != SOL_CAN_J1939)
+ return -EINVAL;
+
+ switch (optname) {
+ case SO_J1939_FILTER:
+ if (optval) {
+ struct j1939_filter *f;
+ int c;
+
+ if (optlen % sizeof(*filters) != 0)
+ return -EINVAL;
+
+ if (optlen > J1939_FILTER_MAX *
+ sizeof(struct j1939_filter))
+ return -EINVAL;
+
+ count = optlen / sizeof(*filters);
+ filters = memdup_user(optval, optlen);
+ if (IS_ERR(filters))
+ return PTR_ERR(filters);
+
+ for (f = filters, c = count; c; f++, c--) {
+ f->name &= f->name_mask;
+ f->pgn &= f->pgn_mask;
+ f->addr &= f->addr_mask;
+ }
+ }
+
+ lock_sock(&jsk->sk);
+ ofilters = jsk->filters;
+ jsk->filters = filters;
+ jsk->nfilters = count;
+ release_sock(&jsk->sk);
+ kfree(ofilters);
+ return 0;
+ case SO_J1939_PROMISC:
+ return j1939_sk_setsockopt_flag(jsk, optval, optlen,
+ J1939_SOCK_PROMISC);
+ case SO_J1939_ERRQUEUE:
+ ret = j1939_sk_setsockopt_flag(jsk, optval, optlen,
+ J1939_SOCK_ERRQUEUE);
+ if (ret < 0)
+ return ret;
+
+ if (!(jsk->state & J1939_SOCK_ERRQUEUE))
+ skb_queue_purge(&sk->sk_error_queue);
+ return ret;
+ case SO_J1939_SEND_PRIO:
+ if (optlen != sizeof(tmp))
+ return -EINVAL;
+ if (copy_from_user(&tmp, optval, optlen))
+ return -EFAULT;
+ if (tmp < 0 || tmp > 7)
+ return -EDOM;
+ if (tmp < 2 && !capable(CAP_NET_ADMIN))
+ return -EPERM;
+ lock_sock(&jsk->sk);
+ jsk->sk.sk_priority = j1939_to_sk_priority(tmp);
+ release_sock(&jsk->sk);
+ return 0;
+ default:
+ return -ENOPROTOOPT;
+ }
+}
+
+static int j1939_sk_getsockopt(struct socket *sock, int level, int optname,
+ char __user *optval, int __user *optlen)
+{
+ struct sock *sk = sock->sk;
+ struct j1939_sock *jsk = j1939_sk(sk);
+ int ret, ulen;
+ /* set defaults for using 'int' properties */
+ int tmp = 0;
+ int len = sizeof(tmp);
+ void *val = &tmp;
+
+ if (level != SOL_CAN_J1939)
+ return -EINVAL;
+ if (get_user(ulen, optlen))
+ return -EFAULT;
+ if (ulen < 0)
+ return -EINVAL;
+
+ lock_sock(&jsk->sk);
+ switch (optname) {
+ case SO_J1939_PROMISC:
+ tmp = (jsk->state & J1939_SOCK_PROMISC) ? 1 : 0;
+ break;
+ case SO_J1939_ERRQUEUE:
+ tmp = (jsk->state & J1939_SOCK_ERRQUEUE) ? 1 : 0;
+ break;
+ case SO_J1939_SEND_PRIO:
+ tmp = j1939_prio(jsk->sk.sk_priority);
+ break;
+ default:
+ ret = -ENOPROTOOPT;
+ goto no_copy;
+ }
+
+ /* copy to user, based on 'len' & 'val'
+ * but most sockopt's are 'int' properties, and have 'len' & 'val'
+ * left unchanged, but instead modified 'tmp'
+ */
+ if (len > ulen)
+ ret = -EFAULT;
+ else if (put_user(len, optlen))
+ ret = -EFAULT;
+ else if (copy_to_user(optval, val, len))
+ ret = -EFAULT;
+ else
+ ret = 0;
+ no_copy:
+ release_sock(&jsk->sk);
+ return ret;
+}
+
+static int j1939_sk_recvmsg(struct socket *sock, struct msghdr *msg,
+ size_t size, int flags)
+{
+ struct sock *sk = sock->sk;
+ struct sk_buff *skb;
+ struct j1939_sk_buff_cb *skcb;
+ int ret = 0;
+
+ if (flags & ~(MSG_DONTWAIT | MSG_ERRQUEUE))
+ return -EINVAL;
+
+ if (flags & MSG_ERRQUEUE)
+ return sock_recv_errqueue(sock->sk, msg, size, SOL_CAN_J1939,
+ SCM_J1939_ERRQUEUE);
+
+ skb = skb_recv_datagram(sk, flags, 0, &ret);
+ if (!skb)
+ return ret;
+
+ if (size < skb->len)
+ msg->msg_flags |= MSG_TRUNC;
+ else
+ size = skb->len;
+
+ ret = memcpy_to_msg(msg, skb->data, size);
+ if (ret < 0) {
+ skb_free_datagram(sk, skb);
+ return ret;
+ }
+
+ skcb = j1939_skb_to_cb(skb);
+ if (j1939_address_is_valid(skcb->addr.da))
+ put_cmsg(msg, SOL_CAN_J1939, SCM_J1939_DEST_ADDR,
+ sizeof(skcb->addr.da), &skcb->addr.da);
+
+ if (skcb->addr.dst_name)
+ put_cmsg(msg, SOL_CAN_J1939, SCM_J1939_DEST_NAME,
+ sizeof(skcb->addr.dst_name), &skcb->addr.dst_name);
+
+ put_cmsg(msg, SOL_CAN_J1939, SCM_J1939_PRIO,
+ sizeof(skcb->priority), &skcb->priority);
+
+ if (msg->msg_name) {
+ struct sockaddr_can *paddr = msg->msg_name;
+
+ msg->msg_namelen = J1939_MIN_NAMELEN;
+ memset(msg->msg_name, 0, msg->msg_namelen);
+ paddr->can_family = AF_CAN;
+ paddr->can_ifindex = skb->skb_iif;
+ paddr->can_addr.j1939.name = skcb->addr.src_name;
+ paddr->can_addr.j1939.addr = skcb->addr.sa;
+ paddr->can_addr.j1939.pgn = skcb->addr.pgn;
+ }
+
+ sock_recv_ts_and_drops(msg, sk, skb);
+ msg->msg_flags |= skcb->msg_flags;
+ skb_free_datagram(sk, skb);
+
+ return size;
+}
+
+static struct sk_buff *j1939_sk_alloc_skb(struct net_device *ndev,
+ struct sock *sk,
+ struct msghdr *msg, size_t size,
+ int *errcode)
+{
+ struct j1939_sock *jsk = j1939_sk(sk);
+ struct j1939_sk_buff_cb *skcb;
+ struct sk_buff *skb;
+ int ret;
+
+ skb = sock_alloc_send_skb(sk,
+ size +
+ sizeof(struct can_frame) -
+ sizeof(((struct can_frame *)NULL)->data) +
+ sizeof(struct can_skb_priv),
+ msg->msg_flags & MSG_DONTWAIT, &ret);
+ if (!skb)
+ goto failure;
+
+ can_skb_reserve(skb);
+ can_skb_prv(skb)->ifindex = ndev->ifindex;
+ can_skb_prv(skb)->skbcnt = 0;
+ skb_reserve(skb, offsetof(struct can_frame, data));
+
+ ret = memcpy_from_msg(skb_put(skb, size), msg, size);
+ if (ret < 0)
+ goto free_skb;
+
+ skb->dev = ndev;
+
+ skcb = j1939_skb_to_cb(skb);
+ memset(skcb, 0, sizeof(*skcb));
+ skcb->addr = jsk->addr;
+ skcb->priority = j1939_prio(sk->sk_priority);
+
+ if (msg->msg_name) {
+ struct sockaddr_can *addr = msg->msg_name;
+
+ if (addr->can_addr.j1939.name ||
+ addr->can_addr.j1939.addr != J1939_NO_ADDR) {
+ skcb->addr.dst_name = addr->can_addr.j1939.name;
+ skcb->addr.da = addr->can_addr.j1939.addr;
+ }
+ if (j1939_pgn_is_valid(addr->can_addr.j1939.pgn))
+ skcb->addr.pgn = addr->can_addr.j1939.pgn;
+ }
+
+ *errcode = ret;
+ return skb;
+
+free_skb:
+ kfree_skb(skb);
+failure:
+ *errcode = ret;
+ return NULL;
+}
+
+static size_t j1939_sk_opt_stats_get_size(void)
+{
+ return
+ nla_total_size(sizeof(u32)) + /* J1939_NLA_BYTES_ACKED */
+ 0;
+}
+
+static struct sk_buff *
+j1939_sk_get_timestamping_opt_stats(struct j1939_session *session)
+{
+ struct sk_buff *stats;
+ u32 size;
+
+ stats = alloc_skb(j1939_sk_opt_stats_get_size(), GFP_ATOMIC);
+ if (!stats)
+ return NULL;
+
+ if (session->skcb.addr.type == J1939_SIMPLE)
+ size = session->total_message_size;
+ else
+ size = min(session->pkt.tx_acked * 7,
+ session->total_message_size);
+
+ nla_put_u32(stats, J1939_NLA_BYTES_ACKED, size);
+
+ return stats;
+}
+
+void j1939_sk_errqueue(struct j1939_session *session,
+ enum j1939_sk_errqueue_type type)
+{
+ struct j1939_priv *priv = session->priv;
+ struct sock *sk = session->sk;
+ struct j1939_sock *jsk;
+ struct sock_exterr_skb *serr;
+ struct sk_buff *skb;
+ char *state = "UNK";
+ int err;
+
+ /* currently we have no sk for the RX session */
+ if (!sk)
+ return;
+
+ jsk = j1939_sk(sk);
+
+ if (!(jsk->state & J1939_SOCK_ERRQUEUE))
+ return;
+
+ skb = j1939_sk_get_timestamping_opt_stats(session);
+ if (!skb)
+ return;
+
+ skb->tstamp = ktime_get_real();
+
+ BUILD_BUG_ON(sizeof(struct sock_exterr_skb) > sizeof(skb->cb));
+
+ serr = SKB_EXT_ERR(skb);
+ memset(serr, 0, sizeof(*serr));
+ switch (type) {
+ case J1939_ERRQUEUE_ACK:
+ if (!(sk->sk_tsflags & SOF_TIMESTAMPING_TX_ACK))
+ return;
+
+ serr->ee.ee_errno = ENOMSG;
+ serr->ee.ee_origin = SO_EE_ORIGIN_TIMESTAMPING;
+ serr->ee.ee_info = SCM_TSTAMP_ACK;
+ state = "ACK";
+ break;
+ case J1939_ERRQUEUE_SCHED:
+ if (!(sk->sk_tsflags & SOF_TIMESTAMPING_TX_SCHED))
+ return;
+
+ serr->ee.ee_errno = ENOMSG;
+ serr->ee.ee_origin = SO_EE_ORIGIN_TIMESTAMPING;
+ serr->ee.ee_info = SCM_TSTAMP_SCHED;
+ state = "SCH";
+ break;
+ case J1939_ERRQUEUE_ABORT:
+ serr->ee.ee_errno = session->err;
+ serr->ee.ee_origin = SO_EE_ORIGIN_LOCAL;
+ serr->ee.ee_info = J1939_EE_INFO_TX_ABORT;
+ state = "ABT";
+ break;
+ default:
+ netdev_err(priv->ndev, "Unknown errqueue type %i\n", type);
+ }
+
+ serr->opt_stats = true;
+ if (sk->sk_tsflags & SOF_TIMESTAMPING_OPT_ID)
+ serr->ee.ee_data = session->tskey;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p tskey: %i, state: %s\n",
+ __func__, session, session->tskey, state);
+ err = sock_queue_err_skb(sk, skb);
+
+ if (err)
+ kfree_skb(skb);
+};
+
+void j1939_sk_send_loop_abort(struct sock *sk, int err)
+{
+ sk->sk_err = err;
+
+ sk->sk_error_report(sk);
+}
+
+static int j1939_sk_send_loop(struct j1939_priv *priv, struct sock *sk,
+ struct msghdr *msg, size_t size)
+
+{
+ struct j1939_sock *jsk = j1939_sk(sk);
+ struct j1939_session *session = j1939_sk_get_incomplete_session(jsk);
+ struct sk_buff *skb;
+ size_t segment_size, todo_size;
+ int ret = 0;
+
+ if (session &&
+ session->total_message_size != session->total_queued_size + size) {
+ j1939_session_put(session);
+ return -EIO;
+ }
+
+ todo_size = size;
+
+ while (todo_size) {
+ struct j1939_sk_buff_cb *skcb;
+
+ segment_size = min_t(size_t, J1939_MAX_TP_PACKET_SIZE,
+ todo_size);
+
+ /* Allocate skb for one segment */
+ skb = j1939_sk_alloc_skb(priv->ndev, sk, msg, segment_size,
+ &ret);
+ if (ret)
+ break;
+
+ skcb = j1939_skb_to_cb(skb);
+
+ if (!session) {
+ /* at this point the size should be full size
+ * of the session
+ */
+ skcb->offset = 0;
+ session = j1939_tp_send(priv, skb, size);
+ if (IS_ERR(session)) {
+ ret = PTR_ERR(session);
+ goto kfree_skb;
+ }
+ if (j1939_sk_queue_session(session)) {
+ /* try to activate session if we a
+ * fist in the queue
+ */
+ if (!j1939_session_activate(session)) {
+ j1939_tp_schedule_txtimer(session, 0);
+ } else {
+ ret = -EBUSY;
+ session->err = ret;
+ j1939_sk_queue_drop_all(priv, jsk,
+ EBUSY);
+ break;
+ }
+ }
+ } else {
+ skcb->offset = session->total_queued_size;
+ j1939_session_skb_queue(session, skb);
+ }
+
+ todo_size -= segment_size;
+ session->total_queued_size += segment_size;
+ }
+
+ switch (ret) {
+ case 0: /* OK */
+ if (todo_size)
+ netdev_warn(priv->ndev,
+ "no error found and not completely queued?! %zu\n",
+ todo_size);
+ ret = size;
+ break;
+ case -ERESTARTSYS:
+ ret = -EINTR;
+ /* fall through */
+ case -EAGAIN: /* OK */
+ if (todo_size != size)
+ ret = size - todo_size;
+ break;
+ default: /* ERROR */
+ break;
+ }
+
+ if (session)
+ j1939_session_put(session);
+
+ return ret;
+
+ kfree_skb:
+ kfree_skb(skb);
+ return ret;
+}
+
+static int j1939_sk_sendmsg(struct socket *sock, struct msghdr *msg,
+ size_t size)
+{
+ struct sock *sk = sock->sk;
+ struct j1939_sock *jsk = j1939_sk(sk);
+ struct j1939_priv *priv = jsk->priv;
+ int ifindex;
+ int ret;
+
+ /* various socket state tests */
+ if (!(jsk->state & J1939_SOCK_BOUND))
+ return -EBADFD;
+
+ ifindex = jsk->ifindex;
+
+ if (!jsk->addr.src_name && jsk->addr.sa == J1939_NO_ADDR)
+ /* no source address assigned yet */
+ return -EBADFD;
+
+ /* deal with provided destination address info */
+ if (msg->msg_name) {
+ struct sockaddr_can *addr = msg->msg_name;
+
+ if (msg->msg_namelen < J1939_MIN_NAMELEN)
+ return -EINVAL;
+
+ if (addr->can_family != AF_CAN)
+ return -EINVAL;
+
+ if (addr->can_ifindex && addr->can_ifindex != ifindex)
+ return -EBADFD;
+
+ if (j1939_pgn_is_valid(addr->can_addr.j1939.pgn) &&
+ !j1939_pgn_is_clean_pdu(addr->can_addr.j1939.pgn))
+ return -EINVAL;
+
+ if (!addr->can_addr.j1939.name &&
+ addr->can_addr.j1939.addr == J1939_NO_ADDR &&
+ !sock_flag(sk, SOCK_BROADCAST))
+ /* broadcast, but SO_BROADCAST not set */
+ return -EACCES;
+ } else {
+ if (!jsk->addr.dst_name && jsk->addr.da == J1939_NO_ADDR &&
+ !sock_flag(sk, SOCK_BROADCAST))
+ /* broadcast, but SO_BROADCAST not set */
+ return -EACCES;
+ }
+
+ ret = j1939_sk_send_loop(priv, sk, msg, size);
+
+ return ret;
+}
+
+void j1939_sk_netdev_event_netdown(struct j1939_priv *priv)
+{
+ struct j1939_sock *jsk;
+ int error_code = ENETDOWN;
+
+ spin_lock_bh(&priv->j1939_socks_lock);
+ list_for_each_entry(jsk, &priv->j1939_socks, list) {
+ jsk->sk.sk_err = error_code;
+ if (!sock_flag(&jsk->sk, SOCK_DEAD))
+ jsk->sk.sk_error_report(&jsk->sk);
+
+ j1939_sk_queue_drop_all(priv, jsk, error_code);
+ }
+ spin_unlock_bh(&priv->j1939_socks_lock);
+}
+
+static int j1939_sk_no_ioctlcmd(struct socket *sock, unsigned int cmd,
+ unsigned long arg)
+{
+ /* no ioctls for socket layer -> hand it down to NIC layer */
+ return -ENOIOCTLCMD;
+}
+
+static const struct proto_ops j1939_ops = {
+ .family = PF_CAN,
+ .release = j1939_sk_release,
+ .bind = j1939_sk_bind,
+ .connect = j1939_sk_connect,
+ .socketpair = sock_no_socketpair,
+ .accept = sock_no_accept,
+ .getname = j1939_sk_getname,
+ .poll = datagram_poll,
+ .ioctl = j1939_sk_no_ioctlcmd,
+ .listen = sock_no_listen,
+ .shutdown = sock_no_shutdown,
+ .setsockopt = j1939_sk_setsockopt,
+ .getsockopt = j1939_sk_getsockopt,
+ .sendmsg = j1939_sk_sendmsg,
+ .recvmsg = j1939_sk_recvmsg,
+ .mmap = sock_no_mmap,
+ .sendpage = sock_no_sendpage,
+};
+
+static struct proto j1939_proto __read_mostly = {
+ .name = "CAN_J1939",
+ .owner = THIS_MODULE,
+ .obj_size = sizeof(struct j1939_sock),
+ .init = j1939_sk_init,
+};
+
+const struct can_proto j1939_can_proto = {
+ .type = SOCK_DGRAM,
+ .protocol = CAN_J1939,
+ .ops = &j1939_ops,
+ .prot = &j1939_proto,
+};
diff --git a/net/can/j1939/transport.c b/net/can/j1939/transport.c
new file mode 100644
index 0000000..fe000ea
--- /dev/null
+++ b/net/can/j1939/transport.c
@@ -0,0 +1,2027 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2010-2011 EIA Electronics,
+// Kurt Van Dijck <kurt.van.dijck@eia.be>
+// Copyright (c) 2018 Protonic,
+// Robin van der Gracht <robin@protonic.nl>
+// Copyright (c) 2017-2019 Pengutronix,
+// Marc Kleine-Budde <kernel@pengutronix.de>
+// Copyright (c) 2017-2019 Pengutronix,
+// Oleksij Rempel <kernel@pengutronix.de>
+
+#include <linux/can/skb.h>
+
+#include "j1939-priv.h"
+
+#define J1939_XTP_TX_RETRY_LIMIT 100
+
+#define J1939_ETP_PGN_CTL 0xc800
+#define J1939_ETP_PGN_DAT 0xc700
+#define J1939_TP_PGN_CTL 0xec00
+#define J1939_TP_PGN_DAT 0xeb00
+
+#define J1939_TP_CMD_RTS 0x10
+#define J1939_TP_CMD_CTS 0x11
+#define J1939_TP_CMD_EOMA 0x13
+#define J1939_TP_CMD_BAM 0x20
+#define J1939_TP_CMD_ABORT 0xff
+
+#define J1939_ETP_CMD_RTS 0x14
+#define J1939_ETP_CMD_CTS 0x15
+#define J1939_ETP_CMD_DPO 0x16
+#define J1939_ETP_CMD_EOMA 0x17
+#define J1939_ETP_CMD_ABORT 0xff
+
+enum j1939_xtp_abort {
+ J1939_XTP_NO_ABORT = 0,
+ J1939_XTP_ABORT_BUSY = 1,
+ /* Already in one or more connection managed sessions and
+ * cannot support another.
+ *
+ * EALREADY:
+ * Operation already in progress
+ */
+
+ J1939_XTP_ABORT_RESOURCE = 2,
+ /* System resources were needed for another task so this
+ * connection managed session was terminated.
+ *
+ * EMSGSIZE:
+ * The socket type requires that message be sent atomically,
+ * and the size of the message to be sent made this
+ * impossible.
+ */
+
+ J1939_XTP_ABORT_TIMEOUT = 3,
+ /* A timeout occurred and this is the connection abort to
+ * close the session.
+ *
+ * EHOSTUNREACH:
+ * The destination host cannot be reached (probably because
+ * the host is down or a remote router cannot reach it).
+ */
+
+ J1939_XTP_ABORT_GENERIC = 4,
+ /* CTS messages received when data transfer is in progress
+ *
+ * EBADMSG:
+ * Not a data message
+ */
+
+ J1939_XTP_ABORT_FAULT = 5,
+ /* Maximal retransmit request limit reached
+ *
+ * ENOTRECOVERABLE:
+ * State not recoverable
+ */
+
+ J1939_XTP_ABORT_UNEXPECTED_DATA = 6,
+ /* Unexpected data transfer packet
+ *
+ * ENOTCONN:
+ * Transport endpoint is not connected
+ */
+
+ J1939_XTP_ABORT_BAD_SEQ = 7,
+ /* Bad sequence number (and software is not able to recover)
+ *
+ * EILSEQ:
+ * Illegal byte sequence
+ */
+
+ J1939_XTP_ABORT_DUP_SEQ = 8,
+ /* Duplicate sequence number (and software is not able to
+ * recover)
+ */
+
+ J1939_XTP_ABORT_EDPO_UNEXPECTED = 9,
+ /* Unexpected EDPO packet (ETP) or Message size > 1785 bytes
+ * (TP)
+ */
+
+ J1939_XTP_ABORT_BAD_EDPO_PGN = 10,
+ /* Unexpected EDPO PGN (PGN in EDPO is bad) */
+
+ J1939_XTP_ABORT_EDPO_OUTOF_CTS = 11,
+ /* EDPO number of packets is greater than CTS */
+
+ J1939_XTP_ABORT_BAD_EDPO_OFFSET = 12,
+ /* Bad EDPO offset */
+
+ J1939_XTP_ABORT_OTHER_DEPRECATED = 13,
+ /* Deprecated. Use 250 instead (Any other reason) */
+
+ J1939_XTP_ABORT_ECTS_UNXPECTED_PGN = 14,
+ /* Unexpected ECTS PGN (PGN in ECTS is bad) */
+
+ J1939_XTP_ABORT_ECTS_TOO_BIG = 15,
+ /* ECTS requested packets exceeds message size */
+
+ J1939_XTP_ABORT_OTHER = 250,
+ /* Any other reason (if a Connection Abort reason is
+ * identified that is not listed in the table use code 250)
+ */
+};
+
+static unsigned int j1939_tp_block = 255;
+static unsigned int j1939_tp_packet_delay;
+static unsigned int j1939_tp_padding = 1;
+
+/* helpers */
+static const char *j1939_xtp_abort_to_str(enum j1939_xtp_abort abort)
+{
+ switch (abort) {
+ case J1939_XTP_ABORT_BUSY:
+ return "Already in one or more connection managed sessions and cannot support another.";
+ case J1939_XTP_ABORT_RESOURCE:
+ return "System resources were needed for another task so this connection managed session was terminated.";
+ case J1939_XTP_ABORT_TIMEOUT:
+ return "A timeout occurred and this is the connection abort to close the session.";
+ case J1939_XTP_ABORT_GENERIC:
+ return "CTS messages received when data transfer is in progress";
+ case J1939_XTP_ABORT_FAULT:
+ return "Maximal retransmit request limit reached";
+ case J1939_XTP_ABORT_UNEXPECTED_DATA:
+ return "Unexpected data transfer packet";
+ case J1939_XTP_ABORT_BAD_SEQ:
+ return "Bad sequence number (and software is not able to recover)";
+ case J1939_XTP_ABORT_DUP_SEQ:
+ return "Duplicate sequence number (and software is not able to recover)";
+ case J1939_XTP_ABORT_EDPO_UNEXPECTED:
+ return "Unexpected EDPO packet (ETP) or Message size > 1785 bytes (TP)";
+ case J1939_XTP_ABORT_BAD_EDPO_PGN:
+ return "Unexpected EDPO PGN (PGN in EDPO is bad)";
+ case J1939_XTP_ABORT_EDPO_OUTOF_CTS:
+ return "EDPO number of packets is greater than CTS";
+ case J1939_XTP_ABORT_BAD_EDPO_OFFSET:
+ return "Bad EDPO offset";
+ case J1939_XTP_ABORT_OTHER_DEPRECATED:
+ return "Deprecated. Use 250 instead (Any other reason)";
+ case J1939_XTP_ABORT_ECTS_UNXPECTED_PGN:
+ return "Unexpected ECTS PGN (PGN in ECTS is bad)";
+ case J1939_XTP_ABORT_ECTS_TOO_BIG:
+ return "ECTS requested packets exceeds message size";
+ case J1939_XTP_ABORT_OTHER:
+ return "Any other reason (if a Connection Abort reason is identified that is not listed in the table use code 250)";
+ default:
+ return "<unknown>";
+ }
+}
+
+static int j1939_xtp_abort_to_errno(struct j1939_priv *priv,
+ enum j1939_xtp_abort abort)
+{
+ int err;
+
+ switch (abort) {
+ case J1939_XTP_NO_ABORT:
+ WARN_ON_ONCE(abort == J1939_XTP_NO_ABORT);
+ err = 0;
+ break;
+ case J1939_XTP_ABORT_BUSY:
+ err = EALREADY;
+ break;
+ case J1939_XTP_ABORT_RESOURCE:
+ err = EMSGSIZE;
+ break;
+ case J1939_XTP_ABORT_TIMEOUT:
+ err = EHOSTUNREACH;
+ break;
+ case J1939_XTP_ABORT_GENERIC:
+ err = EBADMSG;
+ break;
+ case J1939_XTP_ABORT_FAULT:
+ err = ENOTRECOVERABLE;
+ break;
+ case J1939_XTP_ABORT_UNEXPECTED_DATA:
+ err = ENOTCONN;
+ break;
+ case J1939_XTP_ABORT_BAD_SEQ:
+ err = EILSEQ;
+ break;
+ case J1939_XTP_ABORT_DUP_SEQ:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_EDPO_UNEXPECTED:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_BAD_EDPO_PGN:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_EDPO_OUTOF_CTS:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_BAD_EDPO_OFFSET:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_OTHER_DEPRECATED:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_ECTS_UNXPECTED_PGN:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_ECTS_TOO_BIG:
+ err = EPROTO;
+ break;
+ case J1939_XTP_ABORT_OTHER:
+ err = EPROTO;
+ break;
+ default:
+ netdev_warn(priv->ndev, "Unknown abort code %i", abort);
+ err = EPROTO;
+ }
+
+ return err;
+}
+
+static inline void j1939_session_list_lock(struct j1939_priv *priv)
+{
+ spin_lock_bh(&priv->active_session_list_lock);
+}
+
+static inline void j1939_session_list_unlock(struct j1939_priv *priv)
+{
+ spin_unlock_bh(&priv->active_session_list_lock);
+}
+
+void j1939_session_get(struct j1939_session *session)
+{
+ kref_get(&session->kref);
+}
+
+/* session completion functions */
+static void __j1939_session_drop(struct j1939_session *session)
+{
+ if (!session->transmission)
+ return;
+
+ j1939_sock_pending_del(session->sk);
+}
+
+static void j1939_session_destroy(struct j1939_session *session)
+{
+ if (session->err)
+ j1939_sk_errqueue(session, J1939_ERRQUEUE_ABORT);
+ else
+ j1939_sk_errqueue(session, J1939_ERRQUEUE_ACK);
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ skb_queue_purge(&session->skb_queue);
+ __j1939_session_drop(session);
+ j1939_priv_put(session->priv);
+ kfree(session);
+}
+
+static void __j1939_session_release(struct kref *kref)
+{
+ struct j1939_session *session = container_of(kref, struct j1939_session,
+ kref);
+
+ j1939_session_destroy(session);
+}
+
+void j1939_session_put(struct j1939_session *session)
+{
+ kref_put(&session->kref, __j1939_session_release);
+}
+
+static void j1939_session_txtimer_cancel(struct j1939_session *session)
+{
+ if (hrtimer_cancel(&session->txtimer))
+ j1939_session_put(session);
+}
+
+static void j1939_session_rxtimer_cancel(struct j1939_session *session)
+{
+ if (hrtimer_cancel(&session->rxtimer))
+ j1939_session_put(session);
+}
+
+void j1939_session_timers_cancel(struct j1939_session *session)
+{
+ j1939_session_txtimer_cancel(session);
+ j1939_session_rxtimer_cancel(session);
+}
+
+static inline bool j1939_cb_is_broadcast(const struct j1939_sk_buff_cb *skcb)
+{
+ return (!skcb->addr.dst_name && (skcb->addr.da == 0xff));
+}
+
+static void j1939_session_skb_drop_old(struct j1939_session *session)
+{
+ struct sk_buff *do_skb;
+ struct j1939_sk_buff_cb *do_skcb;
+ unsigned int offset_start;
+ unsigned long flags;
+
+ if (skb_queue_len(&session->skb_queue) < 2)
+ return;
+
+ offset_start = session->pkt.tx_acked * 7;
+
+ spin_lock_irqsave(&session->skb_queue.lock, flags);
+ do_skb = skb_peek(&session->skb_queue);
+ do_skcb = j1939_skb_to_cb(do_skb);
+
+ if ((do_skcb->offset + do_skb->len) < offset_start) {
+ __skb_unlink(do_skb, &session->skb_queue);
+ kfree_skb(do_skb);
+ }
+ spin_unlock_irqrestore(&session->skb_queue.lock, flags);
+}
+
+void j1939_session_skb_queue(struct j1939_session *session,
+ struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_priv *priv = session->priv;
+
+ j1939_ac_fixup(priv, skb);
+
+ if (j1939_address_is_unicast(skcb->addr.da) &&
+ priv->ents[skcb->addr.da].nusers)
+ skcb->flags |= J1939_ECU_LOCAL_DST;
+
+ skcb->flags |= J1939_ECU_LOCAL_SRC;
+
+ skb_queue_tail(&session->skb_queue, skb);
+}
+
+static struct sk_buff *j1939_session_skb_find(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ struct sk_buff *skb = NULL;
+ struct sk_buff *do_skb;
+ struct j1939_sk_buff_cb *do_skcb;
+ unsigned int offset_start;
+ unsigned long flags;
+
+ offset_start = session->pkt.dpo * 7;
+
+ spin_lock_irqsave(&session->skb_queue.lock, flags);
+ skb_queue_walk(&session->skb_queue, do_skb) {
+ do_skcb = j1939_skb_to_cb(do_skb);
+
+ if (offset_start >= do_skcb->offset &&
+ offset_start < (do_skcb->offset + do_skb->len)) {
+ skb = do_skb;
+ }
+ }
+ spin_unlock_irqrestore(&session->skb_queue.lock, flags);
+
+ if (!skb)
+ netdev_dbg(priv->ndev, "%s: 0x%p: no skb found for start: %i, queue size: %i\n",
+ __func__, session, offset_start,
+ skb_queue_len(&session->skb_queue));
+
+ return skb;
+}
+
+/* see if we are receiver
+ * returns 0 for broadcasts, although we will receive them
+ */
+static inline int j1939_tp_im_receiver(const struct j1939_sk_buff_cb *skcb)
+{
+ return skcb->flags & J1939_ECU_LOCAL_DST;
+}
+
+/* see if we are sender */
+static inline int j1939_tp_im_transmitter(const struct j1939_sk_buff_cb *skcb)
+{
+ return skcb->flags & J1939_ECU_LOCAL_SRC;
+}
+
+/* see if we are involved as either receiver or transmitter */
+static int j1939_tp_im_involved(const struct j1939_sk_buff_cb *skcb, bool swap)
+{
+ if (swap)
+ return j1939_tp_im_receiver(skcb);
+ else
+ return j1939_tp_im_transmitter(skcb);
+}
+
+static int j1939_tp_im_involved_anydir(struct j1939_sk_buff_cb *skcb)
+{
+ return skcb->flags & (J1939_ECU_LOCAL_SRC | J1939_ECU_LOCAL_DST);
+}
+
+/* extract pgn from flow-ctl message */
+static inline pgn_t j1939_xtp_ctl_to_pgn(const u8 *dat)
+{
+ pgn_t pgn;
+
+ pgn = (dat[7] << 16) | (dat[6] << 8) | (dat[5] << 0);
+ if (j1939_pgn_is_pdu1(pgn))
+ pgn &= 0xffff00;
+ return pgn;
+}
+
+static inline unsigned int j1939_tp_ctl_to_size(const u8 *dat)
+{
+ return (dat[2] << 8) + (dat[1] << 0);
+}
+
+static inline unsigned int j1939_etp_ctl_to_packet(const u8 *dat)
+{
+ return (dat[4] << 16) | (dat[3] << 8) | (dat[2] << 0);
+}
+
+static inline unsigned int j1939_etp_ctl_to_size(const u8 *dat)
+{
+ return (dat[4] << 24) | (dat[3] << 16) |
+ (dat[2] << 8) | (dat[1] << 0);
+}
+
+/* find existing session:
+ * reverse: swap cb's src & dst
+ * there is no problem with matching broadcasts, since
+ * broadcasts (no dst, no da) would never call this
+ * with reverse == true
+ */
+static bool j1939_session_match(struct j1939_addr *se_addr,
+ struct j1939_addr *sk_addr, bool reverse)
+{
+ if (se_addr->type != sk_addr->type)
+ return false;
+
+ if (reverse) {
+ if (se_addr->src_name) {
+ if (se_addr->src_name != sk_addr->dst_name)
+ return false;
+ } else if (se_addr->sa != sk_addr->da) {
+ return false;
+ }
+
+ if (se_addr->dst_name) {
+ if (se_addr->dst_name != sk_addr->src_name)
+ return false;
+ } else if (se_addr->da != sk_addr->sa) {
+ return false;
+ }
+ } else {
+ if (se_addr->src_name) {
+ if (se_addr->src_name != sk_addr->src_name)
+ return false;
+ } else if (se_addr->sa != sk_addr->sa) {
+ return false;
+ }
+
+ if (se_addr->dst_name) {
+ if (se_addr->dst_name != sk_addr->dst_name)
+ return false;
+ } else if (se_addr->da != sk_addr->da) {
+ return false;
+ }
+ }
+
+ return true;
+}
+
+static struct
+j1939_session *j1939_session_get_by_addr_locked(struct j1939_priv *priv,
+ struct list_head *root,
+ struct j1939_addr *addr,
+ bool reverse, bool transmitter)
+{
+ struct j1939_session *session;
+
+ lockdep_assert_held(&priv->active_session_list_lock);
+
+ list_for_each_entry(session, root, active_session_list_entry) {
+ j1939_session_get(session);
+ if (j1939_session_match(&session->skcb.addr, addr, reverse) &&
+ session->transmission == transmitter)
+ return session;
+ j1939_session_put(session);
+ }
+
+ return NULL;
+}
+
+static struct
+j1939_session *j1939_session_get_simple(struct j1939_priv *priv,
+ struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+
+ lockdep_assert_held(&priv->active_session_list_lock);
+
+ list_for_each_entry(session, &priv->active_session_list,
+ active_session_list_entry) {
+ j1939_session_get(session);
+ if (session->skcb.addr.type == J1939_SIMPLE &&
+ session->tskey == skcb->tskey && session->sk == skb->sk)
+ return session;
+ j1939_session_put(session);
+ }
+
+ return NULL;
+}
+
+static struct
+j1939_session *j1939_session_get_by_addr(struct j1939_priv *priv,
+ struct j1939_addr *addr,
+ bool reverse, bool transmitter)
+{
+ struct j1939_session *session;
+
+ j1939_session_list_lock(priv);
+ session = j1939_session_get_by_addr_locked(priv,
+ &priv->active_session_list,
+ addr, reverse, transmitter);
+ j1939_session_list_unlock(priv);
+
+ return session;
+}
+
+static void j1939_skbcb_swap(struct j1939_sk_buff_cb *skcb)
+{
+ u8 tmp = 0;
+
+ swap(skcb->addr.dst_name, skcb->addr.src_name);
+ swap(skcb->addr.da, skcb->addr.sa);
+
+ /* swap SRC and DST flags, leave other untouched */
+ if (skcb->flags & J1939_ECU_LOCAL_SRC)
+ tmp |= J1939_ECU_LOCAL_DST;
+ if (skcb->flags & J1939_ECU_LOCAL_DST)
+ tmp |= J1939_ECU_LOCAL_SRC;
+ skcb->flags &= ~(J1939_ECU_LOCAL_SRC | J1939_ECU_LOCAL_DST);
+ skcb->flags |= tmp;
+}
+
+static struct
+sk_buff *j1939_tp_tx_dat_new(struct j1939_priv *priv,
+ const struct j1939_sk_buff_cb *re_skcb,
+ bool ctl,
+ bool swap_src_dst)
+{
+ struct sk_buff *skb;
+ struct j1939_sk_buff_cb *skcb;
+
+ skb = alloc_skb(sizeof(struct can_frame) + sizeof(struct can_skb_priv),
+ GFP_ATOMIC);
+ if (unlikely(!skb))
+ return ERR_PTR(-ENOMEM);
+
+ skb->dev = priv->ndev;
+ can_skb_reserve(skb);
+ can_skb_prv(skb)->ifindex = priv->ndev->ifindex;
+ /* reserve CAN header */
+ skb_reserve(skb, offsetof(struct can_frame, data));
+
+ memcpy(skb->cb, re_skcb, sizeof(skb->cb));
+ skcb = j1939_skb_to_cb(skb);
+ if (swap_src_dst)
+ j1939_skbcb_swap(skcb);
+
+ if (ctl) {
+ if (skcb->addr.type == J1939_ETP)
+ skcb->addr.pgn = J1939_ETP_PGN_CTL;
+ else
+ skcb->addr.pgn = J1939_TP_PGN_CTL;
+ } else {
+ if (skcb->addr.type == J1939_ETP)
+ skcb->addr.pgn = J1939_ETP_PGN_DAT;
+ else
+ skcb->addr.pgn = J1939_TP_PGN_DAT;
+ }
+
+ return skb;
+}
+
+/* TP transmit packet functions */
+static int j1939_tp_tx_dat(struct j1939_session *session,
+ const u8 *dat, int len)
+{
+ struct j1939_priv *priv = session->priv;
+ struct sk_buff *skb;
+
+ skb = j1939_tp_tx_dat_new(priv, &session->skcb,
+ false, false);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ skb_put_data(skb, dat, len);
+ if (j1939_tp_padding && len < 8)
+ memset(skb_put(skb, 8 - len), 0xff, 8 - len);
+
+ return j1939_send_one(priv, skb);
+}
+
+static int j1939_xtp_do_tx_ctl(struct j1939_priv *priv,
+ const struct j1939_sk_buff_cb *re_skcb,
+ bool swap_src_dst, pgn_t pgn, const u8 *dat)
+{
+ struct sk_buff *skb;
+ u8 *skdat;
+
+ if (!j1939_tp_im_involved(re_skcb, swap_src_dst))
+ return 0;
+
+ skb = j1939_tp_tx_dat_new(priv, re_skcb, true, swap_src_dst);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ skdat = skb_put(skb, 8);
+ memcpy(skdat, dat, 5);
+ skdat[5] = (pgn >> 0);
+ skdat[6] = (pgn >> 8);
+ skdat[7] = (pgn >> 16);
+
+ return j1939_send_one(priv, skb);
+}
+
+static inline int j1939_tp_tx_ctl(struct j1939_session *session,
+ bool swap_src_dst, const u8 *dat)
+{
+ struct j1939_priv *priv = session->priv;
+
+ return j1939_xtp_do_tx_ctl(priv, &session->skcb,
+ swap_src_dst,
+ session->skcb.addr.pgn, dat);
+}
+
+static int j1939_xtp_tx_abort(struct j1939_priv *priv,
+ const struct j1939_sk_buff_cb *re_skcb,
+ bool swap_src_dst,
+ enum j1939_xtp_abort err,
+ pgn_t pgn)
+{
+ u8 dat[5];
+
+ if (!j1939_tp_im_involved(re_skcb, swap_src_dst))
+ return 0;
+
+ memset(dat, 0xff, sizeof(dat));
+ dat[0] = J1939_TP_CMD_ABORT;
+ dat[1] = err;
+ return j1939_xtp_do_tx_ctl(priv, re_skcb, swap_src_dst, pgn, dat);
+}
+
+void j1939_tp_schedule_txtimer(struct j1939_session *session, int msec)
+{
+ j1939_session_get(session);
+ hrtimer_start(&session->txtimer, ms_to_ktime(msec),
+ HRTIMER_MODE_REL_SOFT);
+}
+
+static inline void j1939_tp_set_rxtimeout(struct j1939_session *session,
+ int msec)
+{
+ j1939_session_rxtimer_cancel(session);
+ j1939_session_get(session);
+ hrtimer_start(&session->rxtimer, ms_to_ktime(msec),
+ HRTIMER_MODE_REL_SOFT);
+}
+
+static int j1939_session_tx_rts(struct j1939_session *session)
+{
+ u8 dat[8];
+ int ret;
+
+ memset(dat, 0xff, sizeof(dat));
+
+ dat[1] = (session->total_message_size >> 0);
+ dat[2] = (session->total_message_size >> 8);
+ dat[3] = session->pkt.total;
+
+ if (session->skcb.addr.type == J1939_ETP) {
+ dat[0] = J1939_ETP_CMD_RTS;
+ dat[1] = (session->total_message_size >> 0);
+ dat[2] = (session->total_message_size >> 8);
+ dat[3] = (session->total_message_size >> 16);
+ dat[4] = (session->total_message_size >> 24);
+ } else if (j1939_cb_is_broadcast(&session->skcb)) {
+ dat[0] = J1939_TP_CMD_BAM;
+ /* fake cts for broadcast */
+ session->pkt.tx = 0;
+ } else {
+ dat[0] = J1939_TP_CMD_RTS;
+ dat[4] = dat[3];
+ }
+
+ if (dat[0] == session->last_txcmd)
+ /* done already */
+ return 0;
+
+ ret = j1939_tp_tx_ctl(session, false, dat);
+ if (ret < 0)
+ return ret;
+
+ session->last_txcmd = dat[0];
+ if (dat[0] == J1939_TP_CMD_BAM)
+ j1939_tp_schedule_txtimer(session, 50);
+
+ j1939_tp_set_rxtimeout(session, 1250);
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ return 0;
+}
+
+static int j1939_session_tx_dpo(struct j1939_session *session)
+{
+ unsigned int pkt;
+ u8 dat[8];
+ int ret;
+
+ memset(dat, 0xff, sizeof(dat));
+
+ dat[0] = J1939_ETP_CMD_DPO;
+ session->pkt.dpo = session->pkt.tx_acked;
+ pkt = session->pkt.dpo;
+ dat[1] = session->pkt.last - session->pkt.tx_acked;
+ dat[2] = (pkt >> 0);
+ dat[3] = (pkt >> 8);
+ dat[4] = (pkt >> 16);
+
+ ret = j1939_tp_tx_ctl(session, false, dat);
+ if (ret < 0)
+ return ret;
+
+ session->last_txcmd = dat[0];
+ j1939_tp_set_rxtimeout(session, 1250);
+ session->pkt.tx = session->pkt.tx_acked;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ return 0;
+}
+
+static int j1939_session_tx_dat(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ struct j1939_sk_buff_cb *skcb;
+ int offset, pkt_done, pkt_end;
+ unsigned int len, pdelay;
+ struct sk_buff *se_skb;
+ const u8 *tpdat;
+ int ret = 0;
+ u8 dat[8];
+
+ se_skb = j1939_session_skb_find(session);
+ if (!se_skb)
+ return -ENOBUFS;
+
+ skcb = j1939_skb_to_cb(se_skb);
+ tpdat = se_skb->data;
+ ret = 0;
+ pkt_done = 0;
+ if (session->skcb.addr.type != J1939_ETP &&
+ j1939_cb_is_broadcast(&session->skcb))
+ pkt_end = session->pkt.total;
+ else
+ pkt_end = session->pkt.last;
+
+ while (session->pkt.tx < pkt_end) {
+ dat[0] = session->pkt.tx - session->pkt.dpo + 1;
+ offset = (session->pkt.tx * 7) - skcb->offset;
+ len = se_skb->len - offset;
+ if (len > 7)
+ len = 7;
+
+ memcpy(&dat[1], &tpdat[offset], len);
+ ret = j1939_tp_tx_dat(session, dat, len + 1);
+ if (ret < 0) {
+ /* ENOBUS == CAN interface TX queue is full */
+ if (ret != -ENOBUFS)
+ netdev_alert(priv->ndev,
+ "%s: 0x%p: queue data error: %i\n",
+ __func__, session, ret);
+ break;
+ }
+
+ session->last_txcmd = 0xff;
+ pkt_done++;
+ session->pkt.tx++;
+ pdelay = j1939_cb_is_broadcast(&session->skcb) ? 50 :
+ j1939_tp_packet_delay;
+
+ if (session->pkt.tx < session->pkt.total && pdelay) {
+ j1939_tp_schedule_txtimer(session, pdelay);
+ break;
+ }
+ }
+
+ if (pkt_done)
+ j1939_tp_set_rxtimeout(session, 250);
+
+ return ret;
+}
+
+static int j1939_xtp_txnext_transmiter(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ int ret = 0;
+
+ if (!j1939_tp_im_transmitter(&session->skcb)) {
+ netdev_alert(priv->ndev, "%s: 0x%p: called by not transmitter!\n",
+ __func__, session);
+ return -EINVAL;
+ }
+
+ switch (session->last_cmd) {
+ case 0:
+ ret = j1939_session_tx_rts(session);
+ break;
+
+ case J1939_ETP_CMD_CTS:
+ if (session->last_txcmd != J1939_ETP_CMD_DPO) {
+ ret = j1939_session_tx_dpo(session);
+ if (ret)
+ return ret;
+ }
+
+ /* fall through */
+ case J1939_TP_CMD_CTS:
+ case 0xff: /* did some data */
+ case J1939_ETP_CMD_DPO:
+ case J1939_TP_CMD_BAM:
+ ret = j1939_session_tx_dat(session);
+
+ break;
+ default:
+ netdev_alert(priv->ndev, "%s: 0x%p: unexpected last_cmd: %x\n",
+ __func__, session, session->last_cmd);
+ }
+
+ return ret;
+}
+
+static int j1939_session_tx_cts(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ unsigned int pkt, len;
+ int ret;
+ u8 dat[8];
+
+ if (!j1939_sk_recv_match(priv, &session->skcb))
+ return -ENOENT;
+
+ len = session->pkt.total - session->pkt.rx;
+ len = min3(len, session->pkt.block, j1939_tp_block ?: 255);
+ memset(dat, 0xff, sizeof(dat));
+
+ if (session->skcb.addr.type == J1939_ETP) {
+ pkt = session->pkt.rx + 1;
+ dat[0] = J1939_ETP_CMD_CTS;
+ dat[1] = len;
+ dat[2] = (pkt >> 0);
+ dat[3] = (pkt >> 8);
+ dat[4] = (pkt >> 16);
+ } else {
+ dat[0] = J1939_TP_CMD_CTS;
+ dat[1] = len;
+ dat[2] = session->pkt.rx + 1;
+ }
+
+ if (dat[0] == session->last_txcmd)
+ /* done already */
+ return 0;
+
+ ret = j1939_tp_tx_ctl(session, true, dat);
+ if (ret < 0)
+ return ret;
+
+ if (len)
+ /* only mark cts done when len is set */
+ session->last_txcmd = dat[0];
+ j1939_tp_set_rxtimeout(session, 1250);
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ return 0;
+}
+
+static int j1939_session_tx_eoma(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ u8 dat[8];
+ int ret;
+
+ if (!j1939_sk_recv_match(priv, &session->skcb))
+ return -ENOENT;
+
+ memset(dat, 0xff, sizeof(dat));
+
+ if (session->skcb.addr.type == J1939_ETP) {
+ dat[0] = J1939_ETP_CMD_EOMA;
+ dat[1] = session->total_message_size >> 0;
+ dat[2] = session->total_message_size >> 8;
+ dat[3] = session->total_message_size >> 16;
+ dat[4] = session->total_message_size >> 24;
+ } else {
+ dat[0] = J1939_TP_CMD_EOMA;
+ dat[1] = session->total_message_size;
+ dat[2] = session->total_message_size >> 8;
+ dat[3] = session->pkt.total;
+ }
+
+ if (dat[0] == session->last_txcmd)
+ /* done already */
+ return 0;
+
+ ret = j1939_tp_tx_ctl(session, true, dat);
+ if (ret < 0)
+ return ret;
+
+ session->last_txcmd = dat[0];
+
+ /* wait for the EOMA packet to come in */
+ j1939_tp_set_rxtimeout(session, 1250);
+
+ netdev_dbg(session->priv->ndev, "%p: 0x%p\n", __func__, session);
+
+ return 0;
+}
+
+static int j1939_xtp_txnext_receiver(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ int ret = 0;
+
+ if (!j1939_tp_im_receiver(&session->skcb)) {
+ netdev_alert(priv->ndev, "%s: 0x%p: called by not receiver!\n",
+ __func__, session);
+ return -EINVAL;
+ }
+
+ switch (session->last_cmd) {
+ case J1939_TP_CMD_RTS:
+ case J1939_ETP_CMD_RTS:
+ ret = j1939_session_tx_cts(session);
+ break;
+
+ case J1939_ETP_CMD_CTS:
+ case J1939_TP_CMD_CTS:
+ case 0xff: /* did some data */
+ case J1939_ETP_CMD_DPO:
+ if ((session->skcb.addr.type == J1939_TP &&
+ j1939_cb_is_broadcast(&session->skcb)))
+ break;
+
+ if (session->pkt.rx >= session->pkt.total) {
+ ret = j1939_session_tx_eoma(session);
+ } else if (session->pkt.rx >= session->pkt.last) {
+ session->last_txcmd = 0;
+ ret = j1939_session_tx_cts(session);
+ }
+ break;
+ default:
+ netdev_alert(priv->ndev, "%s: 0x%p: unexpected last_cmd: %x\n",
+ __func__, session, session->last_cmd);
+ }
+
+ return ret;
+}
+
+static int j1939_simple_txnext(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ struct sk_buff *se_skb = j1939_session_skb_find(session);
+ struct sk_buff *skb;
+ int ret;
+
+ if (!se_skb)
+ return 0;
+
+ skb = skb_clone(se_skb, GFP_ATOMIC);
+ if (!skb)
+ return -ENOMEM;
+
+ can_skb_set_owner(skb, se_skb->sk);
+
+ j1939_tp_set_rxtimeout(session, J1939_SIMPLE_ECHO_TIMEOUT_MS);
+
+ ret = j1939_send_one(priv, skb);
+ if (ret)
+ return ret;
+
+ j1939_sk_errqueue(session, J1939_ERRQUEUE_SCHED);
+ j1939_sk_queue_activate_next(session);
+
+ return 0;
+}
+
+static bool j1939_session_deactivate_locked(struct j1939_session *session)
+{
+ bool active = false;
+
+ lockdep_assert_held(&session->priv->active_session_list_lock);
+
+ if (session->state >= J1939_SESSION_ACTIVE &&
+ session->state < J1939_SESSION_ACTIVE_MAX) {
+ active = true;
+
+ list_del_init(&session->active_session_list_entry);
+ session->state = J1939_SESSION_DONE;
+ j1939_session_put(session);
+ }
+
+ return active;
+}
+
+static bool j1939_session_deactivate(struct j1939_session *session)
+{
+ bool active;
+
+ j1939_session_list_lock(session->priv);
+ active = j1939_session_deactivate_locked(session);
+ j1939_session_list_unlock(session->priv);
+
+ return active;
+}
+
+static void
+j1939_session_deactivate_activate_next(struct j1939_session *session)
+{
+ if (j1939_session_deactivate(session))
+ j1939_sk_queue_activate_next(session);
+}
+
+static void j1939_session_cancel(struct j1939_session *session,
+ enum j1939_xtp_abort err)
+{
+ struct j1939_priv *priv = session->priv;
+
+ WARN_ON_ONCE(!err);
+
+ session->err = j1939_xtp_abort_to_errno(priv, err);
+ /* do not send aborts on incoming broadcasts */
+ if (!j1939_cb_is_broadcast(&session->skcb)) {
+ session->state = J1939_SESSION_WAITING_ABORT;
+ j1939_xtp_tx_abort(priv, &session->skcb,
+ !session->transmission,
+ err, session->skcb.addr.pgn);
+ }
+
+ if (session->sk)
+ j1939_sk_send_loop_abort(session->sk, session->err);
+}
+
+static enum hrtimer_restart j1939_tp_txtimer(struct hrtimer *hrtimer)
+{
+ struct j1939_session *session =
+ container_of(hrtimer, struct j1939_session, txtimer);
+ struct j1939_priv *priv = session->priv;
+ int ret = 0;
+
+ if (session->skcb.addr.type == J1939_SIMPLE) {
+ ret = j1939_simple_txnext(session);
+ } else {
+ if (session->transmission)
+ ret = j1939_xtp_txnext_transmiter(session);
+ else
+ ret = j1939_xtp_txnext_receiver(session);
+ }
+
+ switch (ret) {
+ case -ENOBUFS:
+ /* Retry limit is currently arbitrary chosen */
+ if (session->tx_retry < J1939_XTP_TX_RETRY_LIMIT) {
+ session->tx_retry++;
+ j1939_tp_schedule_txtimer(session,
+ 10 + prandom_u32_max(16));
+ } else {
+ netdev_alert(priv->ndev, "%s: 0x%p: tx retry count reached\n",
+ __func__, session);
+ session->err = -ENETUNREACH;
+ j1939_session_rxtimer_cancel(session);
+ j1939_session_deactivate_activate_next(session);
+ }
+ break;
+ case -ENETDOWN:
+ /* In this case we should get a netdev_event(), all active
+ * sessions will be cleared by
+ * j1939_cancel_all_active_sessions(). So handle this as an
+ * error, but let j1939_cancel_all_active_sessions() do the
+ * cleanup including propagation of the error to user space.
+ */
+ break;
+ case 0:
+ session->tx_retry = 0;
+ break;
+ default:
+ netdev_alert(priv->ndev, "%s: 0x%p: tx aborted with unknown reason: %i\n",
+ __func__, session, ret);
+ if (session->skcb.addr.type != J1939_SIMPLE) {
+ j1939_tp_set_rxtimeout(session,
+ J1939_XTP_ABORT_TIMEOUT_MS);
+ j1939_session_cancel(session, J1939_XTP_ABORT_OTHER);
+ } else {
+ session->err = ret;
+ j1939_session_rxtimer_cancel(session);
+ j1939_session_deactivate_activate_next(session);
+ }
+ }
+
+ j1939_session_put(session);
+
+ return HRTIMER_NORESTART;
+}
+
+static void j1939_session_completed(struct j1939_session *session)
+{
+ struct sk_buff *skb;
+
+ if (!session->transmission) {
+ skb = j1939_session_skb_find(session);
+ /* distribute among j1939 receivers */
+ j1939_sk_recv(session->priv, skb);
+ }
+
+ j1939_session_deactivate_activate_next(session);
+}
+
+static enum hrtimer_restart j1939_tp_rxtimer(struct hrtimer *hrtimer)
+{
+ struct j1939_session *session = container_of(hrtimer,
+ struct j1939_session,
+ rxtimer);
+ struct j1939_priv *priv = session->priv;
+
+ if (session->state == J1939_SESSION_WAITING_ABORT) {
+ netdev_alert(priv->ndev, "%s: 0x%p: abort rx timeout. Force session deactivation\n",
+ __func__, session);
+
+ j1939_session_deactivate_activate_next(session);
+
+ } else if (session->skcb.addr.type == J1939_SIMPLE) {
+ netdev_alert(priv->ndev, "%s: 0x%p: Timeout. Failed to send simple message.\n",
+ __func__, session);
+
+ /* The message is probably stuck in the CAN controller and can
+ * be send as soon as CAN bus is in working state again.
+ */
+ session->err = -ETIME;
+ j1939_session_deactivate(session);
+ } else {
+ netdev_alert(priv->ndev, "%s: 0x%p: rx timeout, send abort\n",
+ __func__, session);
+
+ j1939_session_list_lock(session->priv);
+ if (session->state >= J1939_SESSION_ACTIVE &&
+ session->state < J1939_SESSION_ACTIVE_MAX) {
+ j1939_session_get(session);
+ hrtimer_start(&session->rxtimer,
+ ms_to_ktime(J1939_XTP_ABORT_TIMEOUT_MS),
+ HRTIMER_MODE_REL_SOFT);
+ j1939_session_cancel(session, J1939_XTP_ABORT_TIMEOUT);
+ }
+ j1939_session_list_unlock(session->priv);
+ }
+
+ j1939_session_put(session);
+
+ return HRTIMER_NORESTART;
+}
+
+static bool j1939_xtp_rx_cmd_bad_pgn(struct j1939_session *session,
+ const struct sk_buff *skb)
+{
+ const struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ pgn_t pgn = j1939_xtp_ctl_to_pgn(skb->data);
+ struct j1939_priv *priv = session->priv;
+ enum j1939_xtp_abort abort = J1939_XTP_NO_ABORT;
+ u8 cmd = skb->data[0];
+
+ if (session->skcb.addr.pgn == pgn)
+ return false;
+
+ switch (cmd) {
+ case J1939_TP_CMD_BAM:
+ abort = J1939_XTP_NO_ABORT;
+ break;
+
+ case J1939_ETP_CMD_RTS:
+ case J1939_TP_CMD_RTS: /* fall through */
+ abort = J1939_XTP_ABORT_BUSY;
+ break;
+
+ case J1939_ETP_CMD_CTS:
+ case J1939_TP_CMD_CTS: /* fall through */
+ abort = J1939_XTP_ABORT_ECTS_UNXPECTED_PGN;
+ break;
+
+ case J1939_ETP_CMD_DPO:
+ abort = J1939_XTP_ABORT_BAD_EDPO_PGN;
+ break;
+
+ case J1939_ETP_CMD_EOMA:
+ case J1939_TP_CMD_EOMA: /* fall through */
+ abort = J1939_XTP_ABORT_OTHER;
+ break;
+
+ case J1939_ETP_CMD_ABORT: /* && J1939_TP_CMD_ABORT */
+ abort = J1939_XTP_NO_ABORT;
+ break;
+
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+
+ netdev_warn(priv->ndev, "%s: 0x%p: CMD 0x%02x with PGN 0x%05x for running session with different PGN 0x%05x.\n",
+ __func__, session, cmd, pgn, session->skcb.addr.pgn);
+ if (abort != J1939_XTP_NO_ABORT)
+ j1939_xtp_tx_abort(priv, skcb, true, abort, pgn);
+
+ return true;
+}
+
+static void j1939_xtp_rx_abort_one(struct j1939_priv *priv, struct sk_buff *skb,
+ bool reverse, bool transmitter)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+ u8 abort = skb->data[1];
+
+ session = j1939_session_get_by_addr(priv, &skcb->addr, reverse,
+ transmitter);
+ if (!session)
+ return;
+
+ if (j1939_xtp_rx_cmd_bad_pgn(session, skb))
+ goto abort_put;
+
+ netdev_info(priv->ndev, "%s: 0x%p: 0x%05x: (%u) %s\n", __func__,
+ session, j1939_xtp_ctl_to_pgn(skb->data), abort,
+ j1939_xtp_abort_to_str(abort));
+
+ j1939_session_timers_cancel(session);
+ session->err = j1939_xtp_abort_to_errno(priv, abort);
+ if (session->sk)
+ j1939_sk_send_loop_abort(session->sk, session->err);
+ j1939_session_deactivate_activate_next(session);
+
+abort_put:
+ j1939_session_put(session);
+}
+
+/* abort packets may come in 2 directions */
+static void
+j1939_xtp_rx_abort(struct j1939_priv *priv, struct sk_buff *skb,
+ bool transmitter)
+{
+ j1939_xtp_rx_abort_one(priv, skb, false, transmitter);
+ j1939_xtp_rx_abort_one(priv, skb, true, transmitter);
+}
+
+static void
+j1939_xtp_rx_eoma_one(struct j1939_session *session, struct sk_buff *skb)
+{
+ if (j1939_xtp_rx_cmd_bad_pgn(session, skb))
+ return;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ session->pkt.tx_acked = session->pkt.total;
+ j1939_session_timers_cancel(session);
+ /* transmitted without problems */
+ j1939_session_completed(session);
+}
+
+static void
+j1939_xtp_rx_eoma(struct j1939_priv *priv, struct sk_buff *skb,
+ bool transmitter)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+
+ session = j1939_session_get_by_addr(priv, &skcb->addr, true,
+ transmitter);
+ if (!session)
+ return;
+
+ j1939_xtp_rx_eoma_one(session, skb);
+ j1939_session_put(session);
+}
+
+static void
+j1939_xtp_rx_cts_one(struct j1939_session *session, struct sk_buff *skb)
+{
+ enum j1939_xtp_abort err = J1939_XTP_ABORT_FAULT;
+ unsigned int pkt;
+ const u8 *dat;
+
+ dat = skb->data;
+
+ if (j1939_xtp_rx_cmd_bad_pgn(session, skb))
+ return;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ if (session->last_cmd == dat[0]) {
+ err = J1939_XTP_ABORT_DUP_SEQ;
+ goto out_session_cancel;
+ }
+
+ if (session->skcb.addr.type == J1939_ETP)
+ pkt = j1939_etp_ctl_to_packet(dat);
+ else
+ pkt = dat[2];
+
+ if (!pkt)
+ goto out_session_cancel;
+ else if (dat[1] > session->pkt.block /* 0xff for etp */)
+ goto out_session_cancel;
+
+ /* set packet counters only when not CTS(0) */
+ session->pkt.tx_acked = pkt - 1;
+ j1939_session_skb_drop_old(session);
+ session->pkt.last = session->pkt.tx_acked + dat[1];
+ if (session->pkt.last > session->pkt.total)
+ /* safety measure */
+ session->pkt.last = session->pkt.total;
+ /* TODO: do not set tx here, do it in txtimer */
+ session->pkt.tx = session->pkt.tx_acked;
+
+ session->last_cmd = dat[0];
+ if (dat[1]) {
+ j1939_tp_set_rxtimeout(session, 1250);
+ if (session->transmission) {
+ if (session->pkt.tx_acked)
+ j1939_sk_errqueue(session,
+ J1939_ERRQUEUE_SCHED);
+ j1939_session_txtimer_cancel(session);
+ j1939_tp_schedule_txtimer(session, 0);
+ }
+ } else {
+ /* CTS(0) */
+ j1939_tp_set_rxtimeout(session, 550);
+ }
+ return;
+
+ out_session_cancel:
+ j1939_session_timers_cancel(session);
+ j1939_tp_set_rxtimeout(session, J1939_XTP_ABORT_TIMEOUT_MS);
+ j1939_session_cancel(session, err);
+}
+
+static void
+j1939_xtp_rx_cts(struct j1939_priv *priv, struct sk_buff *skb, bool transmitter)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+
+ session = j1939_session_get_by_addr(priv, &skcb->addr, true,
+ transmitter);
+ if (!session)
+ return;
+ j1939_xtp_rx_cts_one(session, skb);
+ j1939_session_put(session);
+}
+
+static struct j1939_session *j1939_session_new(struct j1939_priv *priv,
+ struct sk_buff *skb, size_t size)
+{
+ struct j1939_session *session;
+ struct j1939_sk_buff_cb *skcb;
+
+ session = kzalloc(sizeof(*session), gfp_any());
+ if (!session)
+ return NULL;
+
+ INIT_LIST_HEAD(&session->active_session_list_entry);
+ INIT_LIST_HEAD(&session->sk_session_queue_entry);
+ kref_init(&session->kref);
+
+ j1939_priv_get(priv);
+ session->priv = priv;
+ session->total_message_size = size;
+ session->state = J1939_SESSION_NEW;
+
+ skb_queue_head_init(&session->skb_queue);
+ skb_queue_tail(&session->skb_queue, skb);
+
+ skcb = j1939_skb_to_cb(skb);
+ memcpy(&session->skcb, skcb, sizeof(session->skcb));
+
+ hrtimer_init(&session->txtimer, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ session->txtimer.function = j1939_tp_txtimer;
+ hrtimer_init(&session->rxtimer, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_SOFT);
+ session->rxtimer.function = j1939_tp_rxtimer;
+
+ netdev_dbg(priv->ndev, "%s: 0x%p: sa: %02x, da: %02x\n",
+ __func__, session, skcb->addr.sa, skcb->addr.da);
+
+ return session;
+}
+
+static struct
+j1939_session *j1939_session_fresh_new(struct j1939_priv *priv,
+ int size,
+ const struct j1939_sk_buff_cb *rel_skcb)
+{
+ struct sk_buff *skb;
+ struct j1939_sk_buff_cb *skcb;
+ struct j1939_session *session;
+
+ skb = alloc_skb(size + sizeof(struct can_skb_priv), GFP_ATOMIC);
+ if (unlikely(!skb))
+ return NULL;
+
+ skb->dev = priv->ndev;
+ can_skb_reserve(skb);
+ can_skb_prv(skb)->ifindex = priv->ndev->ifindex;
+ skcb = j1939_skb_to_cb(skb);
+ memcpy(skcb, rel_skcb, sizeof(*skcb));
+
+ session = j1939_session_new(priv, skb, skb->len);
+ if (!session) {
+ kfree_skb(skb);
+ return NULL;
+ }
+
+ /* alloc data area */
+ skb_put(skb, size);
+ /* skb is recounted in j1939_session_new() */
+ return session;
+}
+
+int j1939_session_activate(struct j1939_session *session)
+{
+ struct j1939_priv *priv = session->priv;
+ struct j1939_session *active = NULL;
+ int ret = 0;
+
+ j1939_session_list_lock(priv);
+ if (session->skcb.addr.type != J1939_SIMPLE)
+ active = j1939_session_get_by_addr_locked(priv,
+ &priv->active_session_list,
+ &session->skcb.addr, false,
+ session->transmission);
+ if (active) {
+ j1939_session_put(active);
+ ret = -EAGAIN;
+ } else {
+ WARN_ON_ONCE(session->state != J1939_SESSION_NEW);
+ list_add_tail(&session->active_session_list_entry,
+ &priv->active_session_list);
+ j1939_session_get(session);
+ session->state = J1939_SESSION_ACTIVE;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n",
+ __func__, session);
+ }
+ j1939_session_list_unlock(priv);
+
+ return ret;
+}
+
+static struct
+j1939_session *j1939_xtp_rx_rts_session_new(struct j1939_priv *priv,
+ struct sk_buff *skb)
+{
+ enum j1939_xtp_abort abort = J1939_XTP_NO_ABORT;
+ struct j1939_sk_buff_cb skcb = *j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+ const u8 *dat;
+ pgn_t pgn;
+ int len;
+
+ netdev_dbg(priv->ndev, "%s\n", __func__);
+
+ dat = skb->data;
+ pgn = j1939_xtp_ctl_to_pgn(dat);
+ skcb.addr.pgn = pgn;
+
+ if (!j1939_sk_recv_match(priv, &skcb))
+ return NULL;
+
+ if (skcb.addr.type == J1939_ETP) {
+ len = j1939_etp_ctl_to_size(dat);
+ if (len > J1939_MAX_ETP_PACKET_SIZE)
+ abort = J1939_XTP_ABORT_FAULT;
+ else if (len > priv->tp_max_packet_size)
+ abort = J1939_XTP_ABORT_RESOURCE;
+ else if (len <= J1939_MAX_TP_PACKET_SIZE)
+ abort = J1939_XTP_ABORT_FAULT;
+ } else {
+ len = j1939_tp_ctl_to_size(dat);
+ if (len > J1939_MAX_TP_PACKET_SIZE)
+ abort = J1939_XTP_ABORT_FAULT;
+ else if (len > priv->tp_max_packet_size)
+ abort = J1939_XTP_ABORT_RESOURCE;
+ }
+
+ if (abort != J1939_XTP_NO_ABORT) {
+ j1939_xtp_tx_abort(priv, &skcb, true, abort, pgn);
+ return NULL;
+ }
+
+ session = j1939_session_fresh_new(priv, len, &skcb);
+ if (!session) {
+ j1939_xtp_tx_abort(priv, &skcb, true,
+ J1939_XTP_ABORT_RESOURCE, pgn);
+ return NULL;
+ }
+
+ /* initialize the control buffer: plain copy */
+ session->pkt.total = (len + 6) / 7;
+ session->pkt.block = 0xff;
+ if (skcb.addr.type != J1939_ETP) {
+ if (dat[3] != session->pkt.total)
+ netdev_alert(priv->ndev, "%s: 0x%p: strange total, %u != %u\n",
+ __func__, session, session->pkt.total,
+ dat[3]);
+ session->pkt.total = dat[3];
+ session->pkt.block = min(dat[3], dat[4]);
+ }
+
+ session->pkt.rx = 0;
+ session->pkt.tx = 0;
+
+ WARN_ON_ONCE(j1939_session_activate(session));
+
+ return session;
+}
+
+static int j1939_xtp_rx_rts_session_active(struct j1939_session *session,
+ struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_priv *priv = session->priv;
+
+ if (!session->transmission) {
+ if (j1939_xtp_rx_cmd_bad_pgn(session, skb))
+ return -EBUSY;
+
+ /* RTS on active session */
+ j1939_session_timers_cancel(session);
+ j1939_tp_set_rxtimeout(session, J1939_XTP_ABORT_TIMEOUT_MS);
+ j1939_session_cancel(session, J1939_XTP_ABORT_BUSY);
+ }
+
+ if (session->last_cmd != 0) {
+ /* we received a second rts on the same connection */
+ netdev_alert(priv->ndev, "%s: 0x%p: connection exists (%02x %02x). last cmd: %x\n",
+ __func__, session, skcb->addr.sa, skcb->addr.da,
+ session->last_cmd);
+
+ j1939_session_timers_cancel(session);
+ j1939_tp_set_rxtimeout(session, J1939_XTP_ABORT_TIMEOUT_MS);
+ j1939_session_cancel(session, J1939_XTP_ABORT_BUSY);
+
+ return -EBUSY;
+ }
+
+ if (session->skcb.addr.sa != skcb->addr.sa ||
+ session->skcb.addr.da != skcb->addr.da)
+ netdev_warn(priv->ndev, "%s: 0x%p: session->skcb.addr.sa=0x%02x skcb->addr.sa=0x%02x session->skcb.addr.da=0x%02x skcb->addr.da=0x%02x\n",
+ __func__, session,
+ session->skcb.addr.sa, skcb->addr.sa,
+ session->skcb.addr.da, skcb->addr.da);
+ /* make sure 'sa' & 'da' are correct !
+ * They may be 'not filled in yet' for sending
+ * skb's, since they did not pass the Address Claim ever.
+ */
+ session->skcb.addr.sa = skcb->addr.sa;
+ session->skcb.addr.da = skcb->addr.da;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ return 0;
+}
+
+static void j1939_xtp_rx_rts(struct j1939_priv *priv, struct sk_buff *skb,
+ bool transmitter)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+ u8 cmd = skb->data[0];
+
+ session = j1939_session_get_by_addr(priv, &skcb->addr, false,
+ transmitter);
+
+ if (!session) {
+ if (transmitter) {
+ /* If we're the transmitter and this function is called,
+ * we received our own RTS. A session has already been
+ * created.
+ *
+ * For some reasons however it might have been destroyed
+ * already. So don't create a new one here (using
+ * "j1939_xtp_rx_rts_session_new()") as this will be a
+ * receiver session.
+ *
+ * The reasons the session is already destroyed might
+ * be:
+ * - user space closed socket was and the session was
+ * aborted
+ * - session was aborted due to external abort message
+ */
+ return;
+ }
+ session = j1939_xtp_rx_rts_session_new(priv, skb);
+ if (!session)
+ return;
+ } else {
+ if (j1939_xtp_rx_rts_session_active(session, skb)) {
+ j1939_session_put(session);
+ return;
+ }
+ }
+ session->last_cmd = cmd;
+
+ j1939_tp_set_rxtimeout(session, 1250);
+
+ if (cmd != J1939_TP_CMD_BAM && !session->transmission) {
+ j1939_session_txtimer_cancel(session);
+ j1939_tp_schedule_txtimer(session, 0);
+ }
+
+ j1939_session_put(session);
+}
+
+static void j1939_xtp_rx_dpo_one(struct j1939_session *session,
+ struct sk_buff *skb)
+{
+ const u8 *dat = skb->data;
+
+ if (j1939_xtp_rx_cmd_bad_pgn(session, skb))
+ return;
+
+ netdev_dbg(session->priv->ndev, "%s: 0x%p\n", __func__, session);
+
+ /* transmitted without problems */
+ session->pkt.dpo = j1939_etp_ctl_to_packet(skb->data);
+ session->last_cmd = dat[0];
+ j1939_tp_set_rxtimeout(session, 750);
+}
+
+static void j1939_xtp_rx_dpo(struct j1939_priv *priv, struct sk_buff *skb,
+ bool transmitter)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+
+ session = j1939_session_get_by_addr(priv, &skcb->addr, false,
+ transmitter);
+ if (!session) {
+ netdev_info(priv->ndev,
+ "%s: no connection found\n", __func__);
+ return;
+ }
+
+ j1939_xtp_rx_dpo_one(session, skb);
+ j1939_session_put(session);
+}
+
+static void j1939_xtp_rx_dat_one(struct j1939_session *session,
+ struct sk_buff *skb)
+{
+ struct j1939_priv *priv = session->priv;
+ struct j1939_sk_buff_cb *skcb;
+ struct sk_buff *se_skb;
+ const u8 *dat;
+ u8 *tpdat;
+ int offset;
+ int nbytes;
+ bool final = false;
+ bool do_cts_eoma = false;
+ int packet;
+
+ skcb = j1939_skb_to_cb(skb);
+ dat = skb->data;
+ if (skb->len <= 1)
+ /* makes no sense */
+ goto out_session_cancel;
+
+ switch (session->last_cmd) {
+ case 0xff:
+ break;
+ case J1939_ETP_CMD_DPO:
+ if (skcb->addr.type == J1939_ETP)
+ break;
+ /* fall through */
+ case J1939_TP_CMD_BAM: /* fall through */
+ case J1939_TP_CMD_CTS: /* fall through */
+ if (skcb->addr.type != J1939_ETP)
+ break;
+ /* fall through */
+ default:
+ netdev_info(priv->ndev, "%s: 0x%p: last %02x\n", __func__,
+ session, session->last_cmd);
+ goto out_session_cancel;
+ }
+
+ packet = (dat[0] - 1 + session->pkt.dpo);
+ if (packet > session->pkt.total ||
+ (session->pkt.rx + 1) > session->pkt.total) {
+ netdev_info(priv->ndev, "%s: 0x%p: should have been completed\n",
+ __func__, session);
+ goto out_session_cancel;
+ }
+ se_skb = j1939_session_skb_find(session);
+ if (!se_skb) {
+ netdev_warn(priv->ndev, "%s: 0x%p: no skb found\n", __func__,
+ session);
+ goto out_session_cancel;
+ }
+
+ skcb = j1939_skb_to_cb(se_skb);
+ offset = packet * 7 - skcb->offset;
+ nbytes = se_skb->len - offset;
+ if (nbytes > 7)
+ nbytes = 7;
+ if (nbytes <= 0 || (nbytes + 1) > skb->len) {
+ netdev_info(priv->ndev, "%s: 0x%p: nbytes %i, len %i\n",
+ __func__, session, nbytes, skb->len);
+ goto out_session_cancel;
+ }
+
+ tpdat = se_skb->data;
+ memcpy(&tpdat[offset], &dat[1], nbytes);
+ if (packet == session->pkt.rx)
+ session->pkt.rx++;
+
+ if (skcb->addr.type != J1939_ETP &&
+ j1939_cb_is_broadcast(&session->skcb)) {
+ if (session->pkt.rx >= session->pkt.total)
+ final = true;
+ } else {
+ /* never final, an EOMA must follow */
+ if (session->pkt.rx >= session->pkt.last)
+ do_cts_eoma = true;
+ }
+
+ if (final) {
+ j1939_session_completed(session);
+ } else if (do_cts_eoma) {
+ j1939_tp_set_rxtimeout(session, 1250);
+ if (!session->transmission)
+ j1939_tp_schedule_txtimer(session, 0);
+ } else {
+ j1939_tp_set_rxtimeout(session, 250);
+ }
+ session->last_cmd = 0xff;
+ j1939_session_put(session);
+
+ return;
+
+ out_session_cancel:
+ j1939_session_timers_cancel(session);
+ j1939_tp_set_rxtimeout(session, J1939_XTP_ABORT_TIMEOUT_MS);
+ j1939_session_cancel(session, J1939_XTP_ABORT_FAULT);
+ j1939_session_put(session);
+}
+
+static void j1939_xtp_rx_dat(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb;
+ struct j1939_session *session;
+
+ skcb = j1939_skb_to_cb(skb);
+
+ if (j1939_tp_im_transmitter(skcb)) {
+ session = j1939_session_get_by_addr(priv, &skcb->addr, false,
+ true);
+ if (!session)
+ netdev_info(priv->ndev, "%s: no tx connection found\n",
+ __func__);
+ else
+ j1939_xtp_rx_dat_one(session, skb);
+ }
+
+ if (j1939_tp_im_receiver(skcb)) {
+ session = j1939_session_get_by_addr(priv, &skcb->addr, false,
+ false);
+ if (!session)
+ netdev_info(priv->ndev, "%s: no rx connection found\n",
+ __func__);
+ else
+ j1939_xtp_rx_dat_one(session, skb);
+ }
+}
+
+/* j1939 main intf */
+struct j1939_session *j1939_tp_send(struct j1939_priv *priv,
+ struct sk_buff *skb, size_t size)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ struct j1939_session *session;
+ int ret;
+
+ if (skcb->addr.pgn == J1939_TP_PGN_DAT ||
+ skcb->addr.pgn == J1939_TP_PGN_CTL ||
+ skcb->addr.pgn == J1939_ETP_PGN_DAT ||
+ skcb->addr.pgn == J1939_ETP_PGN_CTL)
+ /* avoid conflict */
+ return ERR_PTR(-EDOM);
+
+ if (size > priv->tp_max_packet_size)
+ return ERR_PTR(-EMSGSIZE);
+
+ if (size <= 8)
+ skcb->addr.type = J1939_SIMPLE;
+ else if (size > J1939_MAX_TP_PACKET_SIZE)
+ skcb->addr.type = J1939_ETP;
+ else
+ skcb->addr.type = J1939_TP;
+
+ if (skcb->addr.type == J1939_ETP &&
+ j1939_cb_is_broadcast(skcb))
+ return ERR_PTR(-EDESTADDRREQ);
+
+ /* fill in addresses from names */
+ ret = j1939_ac_fixup(priv, skb);
+ if (unlikely(ret))
+ return ERR_PTR(ret);
+
+ /* fix DST flags, it may be used there soon */
+ if (j1939_address_is_unicast(skcb->addr.da) &&
+ priv->ents[skcb->addr.da].nusers)
+ skcb->flags |= J1939_ECU_LOCAL_DST;
+
+ /* src is always local, I'm sending ... */
+ skcb->flags |= J1939_ECU_LOCAL_SRC;
+
+ /* prepare new session */
+ session = j1939_session_new(priv, skb, size);
+ if (!session)
+ return ERR_PTR(-ENOMEM);
+
+ /* skb is recounted in j1939_session_new() */
+ session->sk = skb->sk;
+ session->transmission = true;
+ session->pkt.total = (size + 6) / 7;
+ session->pkt.block = skcb->addr.type == J1939_ETP ? 255 :
+ min(j1939_tp_block ?: 255, session->pkt.total);
+
+ if (j1939_cb_is_broadcast(&session->skcb))
+ /* set the end-packet for broadcast */
+ session->pkt.last = session->pkt.total;
+
+ skcb->tskey = session->sk->sk_tskey++;
+ session->tskey = skcb->tskey;
+
+ return session;
+}
+
+static void j1939_tp_cmd_recv(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+ int extd = J1939_TP;
+ u8 cmd = skb->data[0];
+
+ switch (cmd) {
+ case J1939_ETP_CMD_RTS:
+ extd = J1939_ETP;
+ /* fall through */
+ case J1939_TP_CMD_BAM: /* fall through */
+ case J1939_TP_CMD_RTS: /* fall through */
+ if (skcb->addr.type != extd)
+ return;
+
+ if (cmd == J1939_TP_CMD_RTS && j1939_cb_is_broadcast(skcb)) {
+ netdev_alert(priv->ndev, "%s: rts without destination (%02x)\n",
+ __func__, skcb->addr.sa);
+ return;
+ }
+
+ if (j1939_tp_im_transmitter(skcb))
+ j1939_xtp_rx_rts(priv, skb, true);
+
+ if (j1939_tp_im_receiver(skcb))
+ j1939_xtp_rx_rts(priv, skb, false);
+
+ break;
+
+ case J1939_ETP_CMD_CTS:
+ extd = J1939_ETP;
+ /* fall through */
+ case J1939_TP_CMD_CTS:
+ if (skcb->addr.type != extd)
+ return;
+
+ if (j1939_tp_im_transmitter(skcb))
+ j1939_xtp_rx_cts(priv, skb, false);
+
+ if (j1939_tp_im_receiver(skcb))
+ j1939_xtp_rx_cts(priv, skb, true);
+
+ break;
+
+ case J1939_ETP_CMD_DPO:
+ if (skcb->addr.type != J1939_ETP)
+ return;
+
+ if (j1939_tp_im_transmitter(skcb))
+ j1939_xtp_rx_dpo(priv, skb, true);
+
+ if (j1939_tp_im_receiver(skcb))
+ j1939_xtp_rx_dpo(priv, skb, false);
+
+ break;
+
+ case J1939_ETP_CMD_EOMA:
+ extd = J1939_ETP;
+ /* fall through */
+ case J1939_TP_CMD_EOMA:
+ if (skcb->addr.type != extd)
+ return;
+
+ if (j1939_tp_im_transmitter(skcb))
+ j1939_xtp_rx_eoma(priv, skb, false);
+
+ if (j1939_tp_im_receiver(skcb))
+ j1939_xtp_rx_eoma(priv, skb, true);
+
+ break;
+
+ case J1939_ETP_CMD_ABORT: /* && J1939_TP_CMD_ABORT */
+ if (j1939_tp_im_transmitter(skcb))
+ j1939_xtp_rx_abort(priv, skb, true);
+
+ if (j1939_tp_im_receiver(skcb))
+ j1939_xtp_rx_abort(priv, skb, false);
+
+ break;
+ default:
+ return;
+ }
+}
+
+int j1939_tp_recv(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_sk_buff_cb *skcb = j1939_skb_to_cb(skb);
+
+ if (!j1939_tp_im_involved_anydir(skcb))
+ return 0;
+
+ switch (skcb->addr.pgn) {
+ case J1939_ETP_PGN_DAT:
+ skcb->addr.type = J1939_ETP;
+ /* fall through */
+ case J1939_TP_PGN_DAT:
+ j1939_xtp_rx_dat(priv, skb);
+ break;
+
+ case J1939_ETP_PGN_CTL:
+ skcb->addr.type = J1939_ETP;
+ /* fall through */
+ case J1939_TP_PGN_CTL:
+ if (skb->len < 8)
+ return 0; /* Don't care. Nothing to extract here */
+
+ j1939_tp_cmd_recv(priv, skb);
+ break;
+ default:
+ return 0; /* no problem */
+ }
+ return 1; /* "I processed the message" */
+}
+
+void j1939_simple_recv(struct j1939_priv *priv, struct sk_buff *skb)
+{
+ struct j1939_session *session;
+
+ if (!skb->sk)
+ return;
+
+ j1939_session_list_lock(priv);
+ session = j1939_session_get_simple(priv, skb);
+ j1939_session_list_unlock(priv);
+ if (!session) {
+ netdev_warn(priv->ndev,
+ "%s: Received already invalidated message\n",
+ __func__);
+ return;
+ }
+
+ j1939_session_timers_cancel(session);
+ j1939_session_deactivate(session);
+ j1939_session_put(session);
+}
+
+int j1939_cancel_active_session(struct j1939_priv *priv, struct sock *sk)
+{
+ struct j1939_session *session, *saved;
+
+ netdev_dbg(priv->ndev, "%s, sk: %p\n", __func__, sk);
+ j1939_session_list_lock(priv);
+ list_for_each_entry_safe(session, saved,
+ &priv->active_session_list,
+ active_session_list_entry) {
+ if (!sk || sk == session->sk) {
+ j1939_session_timers_cancel(session);
+ session->err = ESHUTDOWN;
+ j1939_session_deactivate_locked(session);
+ }
+ }
+ j1939_session_list_unlock(priv);
+ return NOTIFY_DONE;
+}
+
+void j1939_tp_init(struct j1939_priv *priv)
+{
+ spin_lock_init(&priv->active_session_list_lock);
+ INIT_LIST_HEAD(&priv->active_session_list);
+ priv->tp_max_packet_size = J1939_MAX_ETP_PACKET_SIZE;
+}
diff --git a/net/can/proc.c b/net/can/proc.c
index edb822c..e6881bf 100644
--- a/net/can/proc.c
+++ b/net/can/proc.c
@@ -45,6 +45,7 @@
#include <linux/list.h>
#include <linux/rcupdate.h>
#include <linux/if_arp.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include "af_can.h"
@@ -78,21 +79,21 @@ static const char rx_list_name[][8] = {
static void can_init_stats(struct net *net)
{
- struct s_stats *can_stats = net->can.can_stats;
- struct s_pstats *can_pstats = net->can.can_pstats;
+ struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
+ struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
/*
* This memset function is called from a timer context (when
* can_stattimer is active which is the default) OR in a process
* context (reading the proc_fs when can_stattimer is disabled).
*/
- memset(can_stats, 0, sizeof(struct s_stats));
- can_stats->jiffies_init = jiffies;
+ memset(pkg_stats, 0, sizeof(struct can_pkg_stats));
+ pkg_stats->jiffies_init = jiffies;
- can_pstats->stats_reset++;
+ rcv_lists_stats->stats_reset++;
if (user_reset) {
user_reset = 0;
- can_pstats->user_reset++;
+ rcv_lists_stats->user_reset++;
}
}
@@ -118,8 +119,8 @@ static unsigned long calc_rate(unsigned long oldjif, unsigned long newjif,
void can_stat_update(struct timer_list *t)
{
- struct net *net = from_timer(net, t, can.can_stattimer);
- struct s_stats *can_stats = net->can.can_stats;
+ struct net *net = from_timer(net, t, can.stattimer);
+ struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
unsigned long j = jiffies; /* snapshot */
/* restart counting in timer context on user request */
@@ -127,57 +128,57 @@ void can_stat_update(struct timer_list *t)
can_init_stats(net);
/* restart counting on jiffies overflow */
- if (j < can_stats->jiffies_init)
+ if (j < pkg_stats->jiffies_init)
can_init_stats(net);
/* prevent overflow in calc_rate() */
- if (can_stats->rx_frames > (ULONG_MAX / HZ))
+ if (pkg_stats->rx_frames > (ULONG_MAX / HZ))
can_init_stats(net);
/* prevent overflow in calc_rate() */
- if (can_stats->tx_frames > (ULONG_MAX / HZ))
+ if (pkg_stats->tx_frames > (ULONG_MAX / HZ))
can_init_stats(net);
/* matches overflow - very improbable */
- if (can_stats->matches > (ULONG_MAX / 100))
+ if (pkg_stats->matches > (ULONG_MAX / 100))
can_init_stats(net);
/* calc total values */
- if (can_stats->rx_frames)
- can_stats->total_rx_match_ratio = (can_stats->matches * 100) /
- can_stats->rx_frames;
+ if (pkg_stats->rx_frames)
+ pkg_stats->total_rx_match_ratio = (pkg_stats->matches * 100) /
+ pkg_stats->rx_frames;
- can_stats->total_tx_rate = calc_rate(can_stats->jiffies_init, j,
- can_stats->tx_frames);
- can_stats->total_rx_rate = calc_rate(can_stats->jiffies_init, j,
- can_stats->rx_frames);
+ pkg_stats->total_tx_rate = calc_rate(pkg_stats->jiffies_init, j,
+ pkg_stats->tx_frames);
+ pkg_stats->total_rx_rate = calc_rate(pkg_stats->jiffies_init, j,
+ pkg_stats->rx_frames);
/* calc current values */
- if (can_stats->rx_frames_delta)
- can_stats->current_rx_match_ratio =
- (can_stats->matches_delta * 100) /
- can_stats->rx_frames_delta;
+ if (pkg_stats->rx_frames_delta)
+ pkg_stats->current_rx_match_ratio =
+ (pkg_stats->matches_delta * 100) /
+ pkg_stats->rx_frames_delta;
- can_stats->current_tx_rate = calc_rate(0, HZ, can_stats->tx_frames_delta);
- can_stats->current_rx_rate = calc_rate(0, HZ, can_stats->rx_frames_delta);
+ pkg_stats->current_tx_rate = calc_rate(0, HZ, pkg_stats->tx_frames_delta);
+ pkg_stats->current_rx_rate = calc_rate(0, HZ, pkg_stats->rx_frames_delta);
/* check / update maximum values */
- if (can_stats->max_tx_rate < can_stats->current_tx_rate)
- can_stats->max_tx_rate = can_stats->current_tx_rate;
+ if (pkg_stats->max_tx_rate < pkg_stats->current_tx_rate)
+ pkg_stats->max_tx_rate = pkg_stats->current_tx_rate;
- if (can_stats->max_rx_rate < can_stats->current_rx_rate)
- can_stats->max_rx_rate = can_stats->current_rx_rate;
+ if (pkg_stats->max_rx_rate < pkg_stats->current_rx_rate)
+ pkg_stats->max_rx_rate = pkg_stats->current_rx_rate;
- if (can_stats->max_rx_match_ratio < can_stats->current_rx_match_ratio)
- can_stats->max_rx_match_ratio = can_stats->current_rx_match_ratio;
+ if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio)
+ pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio;
/* clear values for 'current rate' calculation */
- can_stats->tx_frames_delta = 0;
- can_stats->rx_frames_delta = 0;
- can_stats->matches_delta = 0;
+ pkg_stats->tx_frames_delta = 0;
+ pkg_stats->rx_frames_delta = 0;
+ pkg_stats->matches_delta = 0;
/* restart timer (one second) */
- mod_timer(&net->can.can_stattimer, round_jiffies(jiffies + HZ));
+ mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ));
}
/*
@@ -212,60 +213,60 @@ static void can_print_recv_banner(struct seq_file *m)
static int can_stats_proc_show(struct seq_file *m, void *v)
{
struct net *net = m->private;
- struct s_stats *can_stats = net->can.can_stats;
- struct s_pstats *can_pstats = net->can.can_pstats;
+ struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
+ struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
seq_putc(m, '\n');
- seq_printf(m, " %8ld transmitted frames (TXF)\n", can_stats->tx_frames);
- seq_printf(m, " %8ld received frames (RXF)\n", can_stats->rx_frames);
- seq_printf(m, " %8ld matched frames (RXMF)\n", can_stats->matches);
+ seq_printf(m, " %8ld transmitted frames (TXF)\n", pkg_stats->tx_frames);
+ seq_printf(m, " %8ld received frames (RXF)\n", pkg_stats->rx_frames);
+ seq_printf(m, " %8ld matched frames (RXMF)\n", pkg_stats->matches);
seq_putc(m, '\n');
- if (net->can.can_stattimer.function == can_stat_update) {
+ if (net->can.stattimer.function == can_stat_update) {
seq_printf(m, " %8ld %% total match ratio (RXMR)\n",
- can_stats->total_rx_match_ratio);
+ pkg_stats->total_rx_match_ratio);
seq_printf(m, " %8ld frames/s total tx rate (TXR)\n",
- can_stats->total_tx_rate);
+ pkg_stats->total_tx_rate);
seq_printf(m, " %8ld frames/s total rx rate (RXR)\n",
- can_stats->total_rx_rate);
+ pkg_stats->total_rx_rate);
seq_putc(m, '\n');
seq_printf(m, " %8ld %% current match ratio (CRXMR)\n",
- can_stats->current_rx_match_ratio);
+ pkg_stats->current_rx_match_ratio);
seq_printf(m, " %8ld frames/s current tx rate (CTXR)\n",
- can_stats->current_tx_rate);
+ pkg_stats->current_tx_rate);
seq_printf(m, " %8ld frames/s current rx rate (CRXR)\n",
- can_stats->current_rx_rate);
+ pkg_stats->current_rx_rate);
seq_putc(m, '\n');
seq_printf(m, " %8ld %% max match ratio (MRXMR)\n",
- can_stats->max_rx_match_ratio);
+ pkg_stats->max_rx_match_ratio);
seq_printf(m, " %8ld frames/s max tx rate (MTXR)\n",
- can_stats->max_tx_rate);
+ pkg_stats->max_tx_rate);
seq_printf(m, " %8ld frames/s max rx rate (MRXR)\n",
- can_stats->max_rx_rate);
+ pkg_stats->max_rx_rate);
seq_putc(m, '\n');
}
seq_printf(m, " %8ld current receive list entries (CRCV)\n",
- can_pstats->rcv_entries);
+ rcv_lists_stats->rcv_entries);
seq_printf(m, " %8ld maximum receive list entries (MRCV)\n",
- can_pstats->rcv_entries_max);
+ rcv_lists_stats->rcv_entries_max);
- if (can_pstats->stats_reset)
+ if (rcv_lists_stats->stats_reset)
seq_printf(m, "\n %8ld statistic resets (STR)\n",
- can_pstats->stats_reset);
+ rcv_lists_stats->stats_reset);
- if (can_pstats->user_reset)
+ if (rcv_lists_stats->user_reset)
seq_printf(m, " %8ld user statistic resets (USTR)\n",
- can_pstats->user_reset);
+ rcv_lists_stats->user_reset);
seq_putc(m, '\n');
return 0;
@@ -274,20 +275,20 @@ static int can_stats_proc_show(struct seq_file *m, void *v)
static int can_reset_stats_proc_show(struct seq_file *m, void *v)
{
struct net *net = m->private;
- struct s_pstats *can_pstats = net->can.can_pstats;
- struct s_stats *can_stats = net->can.can_stats;
+ struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
+ struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
user_reset = 1;
- if (net->can.can_stattimer.function == can_stat_update) {
+ if (net->can.stattimer.function == can_stat_update) {
seq_printf(m, "Scheduled statistic reset #%ld.\n",
- can_pstats->stats_reset + 1);
+ rcv_lists_stats->stats_reset + 1);
} else {
- if (can_stats->jiffies_init != jiffies)
+ if (pkg_stats->jiffies_init != jiffies)
can_init_stats(net);
seq_printf(m, "Performed statistic reset #%ld.\n",
- can_pstats->stats_reset);
+ rcv_lists_stats->stats_reset);
}
return 0;
}
@@ -300,11 +301,11 @@ static int can_version_proc_show(struct seq_file *m, void *v)
static inline void can_rcvlist_proc_show_one(struct seq_file *m, int idx,
struct net_device *dev,
- struct can_dev_rcv_lists *d)
+ struct can_dev_rcv_lists *dev_rcv_lists)
{
- if (!hlist_empty(&d->rx[idx])) {
+ if (!hlist_empty(&dev_rcv_lists->rx[idx])) {
can_print_recv_banner(m);
- can_print_rcvlist(m, &d->rx[idx], dev);
+ can_print_rcvlist(m, &dev_rcv_lists->rx[idx], dev);
} else
seq_printf(m, " (%s: no entry)\n", DNAME(dev));
@@ -315,7 +316,7 @@ static int can_rcvlist_proc_show(struct seq_file *m, void *v)
/* double cast to prevent GCC warning */
int idx = (int)(long)PDE_DATA(m->file->f_inode);
struct net_device *dev;
- struct can_dev_rcv_lists *d;
+ struct can_dev_rcv_lists *dev_rcv_lists;
struct net *net = m->private;
seq_printf(m, "\nreceive list '%s':\n", rx_list_name[idx]);
@@ -323,8 +324,8 @@ static int can_rcvlist_proc_show(struct seq_file *m, void *v)
rcu_read_lock();
/* receive list for 'all' CAN devices (dev == NULL) */
- d = net->can.can_rx_alldev_list;
- can_rcvlist_proc_show_one(m, idx, NULL, d);
+ dev_rcv_lists = net->can.rx_alldev_list;
+ can_rcvlist_proc_show_one(m, idx, NULL, dev_rcv_lists);
/* receive list for registered CAN devices */
for_each_netdev_rcu(net, dev) {
@@ -366,7 +367,7 @@ static inline void can_rcvlist_proc_show_array(struct seq_file *m,
static int can_rcvlist_sff_proc_show(struct seq_file *m, void *v)
{
struct net_device *dev;
- struct can_dev_rcv_lists *d;
+ struct can_dev_rcv_lists *dev_rcv_lists;
struct net *net = m->private;
/* RX_SFF */
@@ -375,15 +376,16 @@ static int can_rcvlist_sff_proc_show(struct seq_file *m, void *v)
rcu_read_lock();
/* sff receive list for 'all' CAN devices (dev == NULL) */
- d = net->can.can_rx_alldev_list;
- can_rcvlist_proc_show_array(m, NULL, d->rx_sff, ARRAY_SIZE(d->rx_sff));
+ dev_rcv_lists = net->can.rx_alldev_list;
+ can_rcvlist_proc_show_array(m, NULL, dev_rcv_lists->rx_sff,
+ ARRAY_SIZE(dev_rcv_lists->rx_sff));
/* sff receive list for registered CAN devices */
for_each_netdev_rcu(net, dev) {
if (dev->type == ARPHRD_CAN && dev->ml_priv) {
- d = dev->ml_priv;
- can_rcvlist_proc_show_array(m, dev, d->rx_sff,
- ARRAY_SIZE(d->rx_sff));
+ dev_rcv_lists = dev->ml_priv;
+ can_rcvlist_proc_show_array(m, dev, dev_rcv_lists->rx_sff,
+ ARRAY_SIZE(dev_rcv_lists->rx_sff));
}
}
@@ -396,7 +398,7 @@ static int can_rcvlist_sff_proc_show(struct seq_file *m, void *v)
static int can_rcvlist_eff_proc_show(struct seq_file *m, void *v)
{
struct net_device *dev;
- struct can_dev_rcv_lists *d;
+ struct can_dev_rcv_lists *dev_rcv_lists;
struct net *net = m->private;
/* RX_EFF */
@@ -405,15 +407,16 @@ static int can_rcvlist_eff_proc_show(struct seq_file *m, void *v)
rcu_read_lock();
/* eff receive list for 'all' CAN devices (dev == NULL) */
- d = net->can.can_rx_alldev_list;
- can_rcvlist_proc_show_array(m, NULL, d->rx_eff, ARRAY_SIZE(d->rx_eff));
+ dev_rcv_lists = net->can.rx_alldev_list;
+ can_rcvlist_proc_show_array(m, NULL, dev_rcv_lists->rx_eff,
+ ARRAY_SIZE(dev_rcv_lists->rx_eff));
/* eff receive list for registered CAN devices */
for_each_netdev_rcu(net, dev) {
if (dev->type == ARPHRD_CAN && dev->ml_priv) {
- d = dev->ml_priv;
- can_rcvlist_proc_show_array(m, dev, d->rx_eff,
- ARRAY_SIZE(d->rx_eff));
+ dev_rcv_lists = dev->ml_priv;
+ can_rcvlist_proc_show_array(m, dev, dev_rcv_lists->rx_eff,
+ ARRAY_SIZE(dev_rcv_lists->rx_eff));
}
}
diff --git a/net/can/raw.c b/net/can/raw.c
index fdbc361..59c039d 100644
--- a/net/can/raw.c
+++ b/net/can/raw.c
@@ -396,7 +396,7 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len)
int err = 0;
int notify_enetdown = 0;
- if (len < sizeof(*addr))
+ if (len < CAN_REQUIRED_SIZE(*addr, can_ifindex))
return -EINVAL;
if (addr->can_family != AF_CAN)
return -EINVAL;
@@ -733,7 +733,7 @@ static int raw_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
if (msg->msg_name) {
DECLARE_SOCKADDR(struct sockaddr_can *, addr, msg->msg_name);
- if (msg->msg_namelen < sizeof(*addr))
+ if (msg->msg_namelen < CAN_REQUIRED_SIZE(*addr, can_ifindex))
return -EINVAL;
if (addr->can_family != AF_CAN)
diff --git a/net/ceph/crypto.c b/net/ceph/crypto.c
index 5d6724c..4f75df4 100644
--- a/net/ceph/crypto.c
+++ b/net/ceph/crypto.c
@@ -136,8 +136,10 @@ void ceph_crypto_key_destroy(struct ceph_crypto_key *key)
if (key) {
kfree(key->key);
key->key = NULL;
- crypto_free_sync_skcipher(key->tfm);
- key->tfm = NULL;
+ if (key->tfm) {
+ crypto_free_sync_skcipher(key->tfm);
+ key->tfm = NULL;
+ }
}
}
diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
index 0b2df09..78ae6e8 100644
--- a/net/ceph/osd_client.c
+++ b/net/ceph/osd_client.c
@@ -1496,7 +1496,7 @@ static enum calc_target_result calc_target(struct ceph_osd_client *osdc,
struct ceph_osds up, acting;
bool force_resend = false;
bool unpaused = false;
- bool legacy_change;
+ bool legacy_change = false;
bool split = false;
bool sort_bitwise = ceph_osdmap_flag(osdc, CEPH_OSDMAP_SORTBITWISE);
bool recovery_deletes = ceph_osdmap_flag(osdc,
@@ -1584,15 +1584,14 @@ static enum calc_target_result calc_target(struct ceph_osd_client *osdc,
t->osd = acting.primary;
}
- if (unpaused || legacy_change || force_resend ||
- (split && con && CEPH_HAVE_FEATURE(con->peer_features,
- RESEND_ON_SPLIT)))
+ if (unpaused || legacy_change || force_resend || split)
ct_res = CALC_TARGET_NEED_RESEND;
else
ct_res = CALC_TARGET_NO_ACTION;
out:
- dout("%s t %p -> ct_res %d osd %d\n", __func__, t, ct_res, t->osd);
+ dout("%s t %p -> %d%d%d%d ct_res %d osd%d\n", __func__, t, unpaused,
+ legacy_change, force_resend, split, ct_res, t->osd);
return ct_res;
}
diff --git a/net/core/bpf_sk_storage.c b/net/core/bpf_sk_storage.c
index 94c7f77..da5639a 100644
--- a/net/core/bpf_sk_storage.c
+++ b/net/core/bpf_sk_storage.c
@@ -12,6 +12,9 @@
static atomic_t cache_idx;
+#define SK_STORAGE_CREATE_FLAG_MASK \
+ (BPF_F_NO_PREALLOC | BPF_F_CLONE)
+
struct bucket {
struct hlist_head list;
raw_spinlock_t lock;
@@ -209,7 +212,6 @@ static void selem_unlink_sk(struct bpf_sk_storage_elem *selem)
kfree_rcu(sk_storage, rcu);
}
-/* sk_storage->lock must be held and sk_storage->list cannot be empty */
static void __selem_link_sk(struct bpf_sk_storage *sk_storage,
struct bpf_sk_storage_elem *selem)
{
@@ -509,7 +511,7 @@ static int sk_storage_delete(struct sock *sk, struct bpf_map *map)
return 0;
}
-/* Called by __sk_destruct() */
+/* Called by __sk_destruct() & bpf_sk_storage_clone() */
void bpf_sk_storage_free(struct sock *sk)
{
struct bpf_sk_storage_elem *selem;
@@ -557,6 +559,11 @@ static void bpf_sk_storage_map_free(struct bpf_map *map)
smap = (struct bpf_sk_storage_map *)map;
+ /* Note that this map might be concurrently cloned from
+ * bpf_sk_storage_clone. Wait for any existing bpf_sk_storage_clone
+ * RCU read section to finish before proceeding. New RCU
+ * read sections should be prevented via bpf_map_inc_not_zero.
+ */
synchronize_rcu();
/* bpf prog and the userspace can no longer access this map
@@ -601,7 +608,9 @@ static void bpf_sk_storage_map_free(struct bpf_map *map)
static int bpf_sk_storage_map_alloc_check(union bpf_attr *attr)
{
- if (attr->map_flags != BPF_F_NO_PREALLOC || attr->max_entries ||
+ if (attr->map_flags & ~SK_STORAGE_CREATE_FLAG_MASK ||
+ !(attr->map_flags & BPF_F_NO_PREALLOC) ||
+ attr->max_entries ||
attr->key_size != sizeof(int) || !attr->value_size ||
/* Enforce BTF for userspace sk dumping */
!attr->btf_key_type_id || !attr->btf_value_type_id)
@@ -739,6 +748,95 @@ static int bpf_fd_sk_storage_delete_elem(struct bpf_map *map, void *key)
return err;
}
+static struct bpf_sk_storage_elem *
+bpf_sk_storage_clone_elem(struct sock *newsk,
+ struct bpf_sk_storage_map *smap,
+ struct bpf_sk_storage_elem *selem)
+{
+ struct bpf_sk_storage_elem *copy_selem;
+
+ copy_selem = selem_alloc(smap, newsk, NULL, true);
+ if (!copy_selem)
+ return NULL;
+
+ if (map_value_has_spin_lock(&smap->map))
+ copy_map_value_locked(&smap->map, SDATA(copy_selem)->data,
+ SDATA(selem)->data, true);
+ else
+ copy_map_value(&smap->map, SDATA(copy_selem)->data,
+ SDATA(selem)->data);
+
+ return copy_selem;
+}
+
+int bpf_sk_storage_clone(const struct sock *sk, struct sock *newsk)
+{
+ struct bpf_sk_storage *new_sk_storage = NULL;
+ struct bpf_sk_storage *sk_storage;
+ struct bpf_sk_storage_elem *selem;
+ int ret = 0;
+
+ RCU_INIT_POINTER(newsk->sk_bpf_storage, NULL);
+
+ rcu_read_lock();
+ sk_storage = rcu_dereference(sk->sk_bpf_storage);
+
+ if (!sk_storage || hlist_empty(&sk_storage->list))
+ goto out;
+
+ hlist_for_each_entry_rcu(selem, &sk_storage->list, snode) {
+ struct bpf_sk_storage_elem *copy_selem;
+ struct bpf_sk_storage_map *smap;
+ struct bpf_map *map;
+
+ smap = rcu_dereference(SDATA(selem)->smap);
+ if (!(smap->map.map_flags & BPF_F_CLONE))
+ continue;
+
+ /* Note that for lockless listeners adding new element
+ * here can race with cleanup in bpf_sk_storage_map_free.
+ * Try to grab map refcnt to make sure that it's still
+ * alive and prevent concurrent removal.
+ */
+ map = bpf_map_inc_not_zero(&smap->map, false);
+ if (IS_ERR(map))
+ continue;
+
+ copy_selem = bpf_sk_storage_clone_elem(newsk, smap, selem);
+ if (!copy_selem) {
+ ret = -ENOMEM;
+ bpf_map_put(map);
+ goto out;
+ }
+
+ if (new_sk_storage) {
+ selem_link_map(smap, copy_selem);
+ __selem_link_sk(new_sk_storage, copy_selem);
+ } else {
+ ret = sk_storage_alloc(newsk, smap, copy_selem);
+ if (ret) {
+ kfree(copy_selem);
+ atomic_sub(smap->elem_size,
+ &newsk->sk_omem_alloc);
+ bpf_map_put(map);
+ goto out;
+ }
+
+ new_sk_storage = rcu_dereference(copy_selem->sk_storage);
+ }
+ bpf_map_put(map);
+ }
+
+out:
+ rcu_read_unlock();
+
+ /* In case of an error, don't free anything explicitly here, the
+ * caller is responsible to call bpf_sk_storage_free.
+ */
+
+ return ret;
+}
+
BPF_CALL_4(bpf_sk_storage_get, struct bpf_map *, map, struct sock *, sk,
void *, value, u64, flags)
{
diff --git a/net/core/dev.c b/net/core/dev.c
index 49589ed..b1afafe 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -8126,12 +8126,15 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack,
bpf_chk = generic_xdp_install;
if (fd >= 0) {
+ u32 prog_id;
+
if (!offload && __dev_xdp_query(dev, bpf_chk, XDP_QUERY_PROG)) {
NL_SET_ERR_MSG(extack, "native and generic XDP can't be active at the same time");
return -EEXIST;
}
- if ((flags & XDP_FLAGS_UPDATE_IF_NOEXIST) &&
- __dev_xdp_query(dev, bpf_op, query)) {
+
+ prog_id = __dev_xdp_query(dev, bpf_op, query);
+ if ((flags & XDP_FLAGS_UPDATE_IF_NOEXIST) && prog_id) {
NL_SET_ERR_MSG(extack, "XDP program already attached");
return -EBUSY;
}
@@ -8146,6 +8149,14 @@ int dev_change_xdp_fd(struct net_device *dev, struct netlink_ext_ack *extack,
bpf_prog_put(prog);
return -EINVAL;
}
+
+ if (prog->aux->id == prog_id) {
+ bpf_prog_put(prog);
+ return 0;
+ }
+ } else {
+ if (!__dev_xdp_query(dev, bpf_op, query))
+ return 0;
}
err = dev_xdp_install(dev, bpf_op, extack, flags, prog);
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 650f363..4a2fb94 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -136,7 +136,7 @@ static struct devlink *devlink_get_from_info(struct genl_info *info)
}
static struct devlink_port *devlink_port_get_by_index(struct devlink *devlink,
- int port_index)
+ unsigned int port_index)
{
struct devlink_port *devlink_port;
@@ -147,7 +147,8 @@ static struct devlink_port *devlink_port_get_by_index(struct devlink *devlink,
return NULL;
}
-static bool devlink_port_index_exists(struct devlink *devlink, int port_index)
+static bool devlink_port_index_exists(struct devlink *devlink,
+ unsigned int port_index)
{
return devlink_port_get_by_index(devlink, port_index);
}
@@ -509,32 +510,37 @@ static int devlink_nl_port_attrs_put(struct sk_buff *msg,
return 0;
if (nla_put_u16(msg, DEVLINK_ATTR_PORT_FLAVOUR, attrs->flavour))
return -EMSGSIZE;
- if (devlink_port->attrs.flavour == DEVLINK_PORT_FLAVOUR_PCI_PF) {
+ switch (devlink_port->attrs.flavour) {
+ case DEVLINK_PORT_FLAVOUR_PCI_PF:
if (nla_put_u16(msg, DEVLINK_ATTR_PORT_PCI_PF_NUMBER,
attrs->pci_pf.pf))
return -EMSGSIZE;
- } else if (devlink_port->attrs.flavour == DEVLINK_PORT_FLAVOUR_PCI_VF) {
+ break;
+ case DEVLINK_PORT_FLAVOUR_PCI_VF:
if (nla_put_u16(msg, DEVLINK_ATTR_PORT_PCI_PF_NUMBER,
attrs->pci_vf.pf) ||
nla_put_u16(msg, DEVLINK_ATTR_PORT_PCI_VF_NUMBER,
attrs->pci_vf.vf))
return -EMSGSIZE;
+ break;
+ case DEVLINK_PORT_FLAVOUR_PHYSICAL:
+ case DEVLINK_PORT_FLAVOUR_CPU:
+ case DEVLINK_PORT_FLAVOUR_DSA:
+ if (nla_put_u32(msg, DEVLINK_ATTR_PORT_NUMBER,
+ attrs->phys.port_number))
+ return -EMSGSIZE;
+ if (!attrs->split)
+ return 0;
+ if (nla_put_u32(msg, DEVLINK_ATTR_PORT_SPLIT_GROUP,
+ attrs->phys.port_number))
+ return -EMSGSIZE;
+ if (nla_put_u32(msg, DEVLINK_ATTR_PORT_SPLIT_SUBPORT_NUMBER,
+ attrs->phys.split_subport_number))
+ return -EMSGSIZE;
+ break;
+ default:
+ break;
}
- if (devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PHYSICAL &&
- devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_CPU &&
- devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_DSA)
- return 0;
- if (nla_put_u32(msg, DEVLINK_ATTR_PORT_NUMBER,
- attrs->phys.port_number))
- return -EMSGSIZE;
- if (!attrs->split)
- return 0;
- if (nla_put_u32(msg, DEVLINK_ATTR_PORT_SPLIT_GROUP,
- attrs->phys.port_number))
- return -EMSGSIZE;
- if (nla_put_u32(msg, DEVLINK_ATTR_PORT_SPLIT_SUBPORT_NUMBER,
- attrs->phys.split_subport_number))
- return -EMSGSIZE;
return 0;
}
@@ -2846,6 +2852,11 @@ static const struct devlink_param devlink_param_generic[] = {
.name = DEVLINK_PARAM_GENERIC_FW_LOAD_POLICY_NAME,
.type = DEVLINK_PARAM_GENERIC_FW_LOAD_POLICY_TYPE,
},
+ {
+ .id = DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE,
+ .name = DEVLINK_PARAM_GENERIC_RESET_DEV_ON_DRV_PROBE_NAME,
+ .type = DEVLINK_PARAM_GENERIC_RESET_DEV_ON_DRV_PROBE_TYPE,
+ },
};
static int devlink_param_generic_verify(const struct devlink_param *param)
diff --git a/net/core/drop_monitor.c b/net/core/drop_monitor.c
index bfc0240..cc60cc2 100644
--- a/net/core/drop_monitor.c
+++ b/net/core/drop_monitor.c
@@ -552,7 +552,7 @@ static size_t net_dm_packet_report_size(size_t payload_len)
/* NET_DM_ATTR_IN_PORT */
net_dm_in_port_size() +
/* NET_DM_ATTR_TIMESTAMP */
- nla_total_size(sizeof(struct timespec)) +
+ nla_total_size(sizeof(u64)) +
/* NET_DM_ATTR_ORIG_LEN */
nla_total_size(sizeof(u32)) +
/* NET_DM_ATTR_PROTO */
@@ -592,7 +592,6 @@ static int net_dm_packet_report_fill(struct sk_buff *msg, struct sk_buff *skb,
u64 pc = (u64)(uintptr_t) NET_DM_SKB_CB(skb)->pc;
char buf[NET_DM_MAX_SYMBOL_LEN];
struct nlattr *attr;
- struct timespec ts;
void *hdr;
int rc;
@@ -615,8 +614,8 @@ static int net_dm_packet_report_fill(struct sk_buff *msg, struct sk_buff *skb,
if (rc)
goto nla_put_failure;
- if (ktime_to_timespec_cond(skb->tstamp, &ts) &&
- nla_put(msg, NET_DM_ATTR_TIMESTAMP, sizeof(ts), &ts))
+ if (nla_put_u64_64bit(msg, NET_DM_ATTR_TIMESTAMP,
+ ktime_to_ns(skb->tstamp), NET_DM_ATTR_PAD))
goto nla_put_failure;
if (nla_put_u32(msg, NET_DM_ATTR_ORIG_LEN, skb->len))
@@ -716,7 +715,7 @@ net_dm_hw_packet_report_size(size_t payload_len,
/* NET_DM_ATTR_IN_PORT */
net_dm_in_port_size() +
/* NET_DM_ATTR_TIMESTAMP */
- nla_total_size(sizeof(struct timespec)) +
+ nla_total_size(sizeof(u64)) +
/* NET_DM_ATTR_ORIG_LEN */
nla_total_size(sizeof(u32)) +
/* NET_DM_ATTR_PROTO */
@@ -730,7 +729,6 @@ static int net_dm_hw_packet_report_fill(struct sk_buff *msg,
{
struct net_dm_hw_metadata *hw_metadata;
struct nlattr *attr;
- struct timespec ts;
void *hdr;
hw_metadata = NET_DM_SKB_CB(skb)->hw_metadata;
@@ -761,8 +759,8 @@ static int net_dm_hw_packet_report_fill(struct sk_buff *msg,
goto nla_put_failure;
}
- if (ktime_to_timespec_cond(skb->tstamp, &ts) &&
- nla_put(msg, NET_DM_ATTR_TIMESTAMP, sizeof(ts), &ts))
+ if (nla_put_u64_64bit(msg, NET_DM_ATTR_TIMESTAMP,
+ ktime_to_ns(skb->tstamp), NET_DM_ATTR_PAD))
goto nla_put_failure;
if (nla_put_u32(msg, NET_DM_ATTR_ORIG_LEN, skb->len))
diff --git a/net/core/fib_notifier.c b/net/core/fib_notifier.c
index 13a40b8..470a606 100644
--- a/net/core/fib_notifier.c
+++ b/net/core/fib_notifier.c
@@ -5,8 +5,15 @@
#include <linux/module.h>
#include <linux/init.h>
#include <net/net_namespace.h>
+#include <net/netns/generic.h>
#include <net/fib_notifier.h>
+static unsigned int fib_notifier_net_id;
+
+struct fib_notifier_net {
+ struct list_head fib_notifier_ops;
+};
+
static ATOMIC_NOTIFIER_HEAD(fib_chain);
int call_fib_notifier(struct notifier_block *nb, struct net *net,
@@ -34,6 +41,7 @@ EXPORT_SYMBOL(call_fib_notifiers);
static unsigned int fib_seq_sum(void)
{
+ struct fib_notifier_net *fn_net;
struct fib_notifier_ops *ops;
unsigned int fib_seq = 0;
struct net *net;
@@ -41,8 +49,9 @@ static unsigned int fib_seq_sum(void)
rtnl_lock();
down_read(&net_rwsem);
for_each_net(net) {
+ fn_net = net_generic(net, fib_notifier_net_id);
rcu_read_lock();
- list_for_each_entry_rcu(ops, &net->fib_notifier_ops, list) {
+ list_for_each_entry_rcu(ops, &fn_net->fib_notifier_ops, list) {
if (!try_module_get(ops->owner))
continue;
fib_seq += ops->fib_seq_read(net);
@@ -58,9 +67,10 @@ static unsigned int fib_seq_sum(void)
static int fib_net_dump(struct net *net, struct notifier_block *nb)
{
+ struct fib_notifier_net *fn_net = net_generic(net, fib_notifier_net_id);
struct fib_notifier_ops *ops;
- list_for_each_entry_rcu(ops, &net->fib_notifier_ops, list) {
+ list_for_each_entry_rcu(ops, &fn_net->fib_notifier_ops, list) {
int err;
if (!try_module_get(ops->owner))
@@ -127,12 +137,13 @@ EXPORT_SYMBOL(unregister_fib_notifier);
static int __fib_notifier_ops_register(struct fib_notifier_ops *ops,
struct net *net)
{
+ struct fib_notifier_net *fn_net = net_generic(net, fib_notifier_net_id);
struct fib_notifier_ops *o;
- list_for_each_entry(o, &net->fib_notifier_ops, list)
+ list_for_each_entry(o, &fn_net->fib_notifier_ops, list)
if (ops->family == o->family)
return -EEXIST;
- list_add_tail_rcu(&ops->list, &net->fib_notifier_ops);
+ list_add_tail_rcu(&ops->list, &fn_net->fib_notifier_ops);
return 0;
}
@@ -167,18 +178,24 @@ EXPORT_SYMBOL(fib_notifier_ops_unregister);
static int __net_init fib_notifier_net_init(struct net *net)
{
- INIT_LIST_HEAD(&net->fib_notifier_ops);
+ struct fib_notifier_net *fn_net = net_generic(net, fib_notifier_net_id);
+
+ INIT_LIST_HEAD(&fn_net->fib_notifier_ops);
return 0;
}
static void __net_exit fib_notifier_net_exit(struct net *net)
{
- WARN_ON_ONCE(!list_empty(&net->fib_notifier_ops));
+ struct fib_notifier_net *fn_net = net_generic(net, fib_notifier_net_id);
+
+ WARN_ON_ONCE(!list_empty(&fn_net->fib_notifier_ops));
}
static struct pernet_operations fib_notifier_net_ops = {
.init = fib_notifier_net_init,
.exit = fib_notifier_net_exit,
+ .id = &fib_notifier_net_id,
+ .size = sizeof(struct fib_notifier_net),
};
static int __init fib_notifier_init(void)
diff --git a/net/core/filter.c b/net/core/filter.c
index 0c1059c..ed65636 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5903,7 +5903,7 @@ BPF_CALL_5(bpf_tcp_gen_syncookie, struct sock *, sk, void *, iph, u32, iph_len,
default:
return -EPROTONOSUPPORT;
}
- if (mss <= 0)
+ if (mss == 0)
return -ENOENT;
return cookie | ((u64)mss << 32);
@@ -8841,13 +8841,13 @@ sk_reuseport_is_valid_access(int off, int size,
return size == size_default;
/* Fields that allow narrowing */
- case offsetof(struct sk_reuseport_md, eth_protocol):
+ case bpf_ctx_range(struct sk_reuseport_md, eth_protocol):
if (size < FIELD_SIZEOF(struct sk_buff, protocol))
return false;
/* fall through */
- case offsetof(struct sk_reuseport_md, ip_protocol):
- case offsetof(struct sk_reuseport_md, bind_inany):
- case offsetof(struct sk_reuseport_md, len):
+ case bpf_ctx_range(struct sk_reuseport_md, ip_protocol):
+ case bpf_ctx_range(struct sk_reuseport_md, bind_inany):
+ case bpf_ctx_range(struct sk_reuseport_md, len):
bpf_ctx_record_field_size(info, size_default);
return bpf_ctx_narrow_access_ok(off, size, size_default);
diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index 9741b59..7c09d87 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -142,8 +142,8 @@ int skb_flow_dissector_bpf_prog_detach(const union bpf_attr *attr)
mutex_unlock(&flow_dissector_mutex);
return -ENOENT;
}
- bpf_prog_put(attached);
RCU_INIT_POINTER(net->flow_dissector_prog, NULL);
+ bpf_prog_put(attached);
mutex_unlock(&flow_dissector_mutex);
return 0;
}
diff --git a/net/core/netpoll.c b/net/core/netpoll.c
index 2cf27da..849380a 100644
--- a/net/core/netpoll.c
+++ b/net/core/netpoll.c
@@ -122,7 +122,7 @@ static void queue_process(struct work_struct *work)
txq = netdev_get_tx_queue(dev, q_index);
HARD_TX_LOCK(dev, txq, smp_processor_id());
if (netif_xmit_frozen_or_stopped(txq) ||
- netpoll_start_xmit(skb, dev, txq) != NETDEV_TX_OK) {
+ !dev_xmit_complete(netpoll_start_xmit(skb, dev, txq))) {
skb_queue_head(&npinfo->txq, skb);
HARD_TX_UNLOCK(dev, txq);
local_irq_restore(flags);
@@ -335,7 +335,7 @@ void netpoll_send_skb_on_dev(struct netpoll *np, struct sk_buff *skb,
HARD_TX_UNLOCK(dev, txq);
- if (status == NETDEV_TX_OK)
+ if (dev_xmit_complete(status))
break;
}
@@ -352,7 +352,7 @@ void netpoll_send_skb_on_dev(struct netpoll *np, struct sk_buff *skb,
}
- if (status != NETDEV_TX_OK) {
+ if (!dev_xmit_complete(status)) {
skb_queue_tail(&npinfo->txq, skb);
schedule_delayed_work(&npinfo->tx_work,0);
}
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index ea8e8d3..2b40b5a 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -4087,6 +4087,9 @@ static const u8 skb_ext_type_len[] = {
#ifdef CONFIG_XFRM
[SKB_EXT_SEC_PATH] = SKB_EXT_CHUNKSIZEOF(struct sec_path),
#endif
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ [TC_SKB_EXT] = SKB_EXT_CHUNKSIZEOF(struct tc_skb_ext),
+#endif
};
static __always_inline unsigned int skb_ext_total_length(void)
@@ -4098,6 +4101,9 @@ static __always_inline unsigned int skb_ext_total_length(void)
#ifdef CONFIG_XFRM
skb_ext_type_len[SKB_EXT_SEC_PATH] +
#endif
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ skb_ext_type_len[TC_SKB_EXT] +
+#endif
0;
}
diff --git a/net/core/skmsg.c b/net/core/skmsg.c
index 6832eeb..cf390e0 100644
--- a/net/core/skmsg.c
+++ b/net/core/skmsg.c
@@ -190,8 +190,7 @@ static int __sk_msg_free(struct sock *sk, struct sk_msg *msg, u32 i,
sk_msg_check_to_free(msg, i, msg->sg.size);
sge = sk_msg_elem(msg, i);
}
- if (msg->skb)
- consume_skb(msg->skb);
+ consume_skb(msg->skb);
sk_msg_init(msg);
return freed;
}
diff --git a/net/core/sock.c b/net/core/sock.c
index 6d08553..07863ed 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -1851,9 +1851,12 @@ struct sock *sk_clone_lock(const struct sock *sk, const gfp_t priority)
goto out;
}
RCU_INIT_POINTER(newsk->sk_reuseport_cb, NULL);
-#ifdef CONFIG_BPF_SYSCALL
- RCU_INIT_POINTER(newsk->sk_bpf_storage, NULL);
-#endif
+
+ if (bpf_sk_storage_clone(sk, newsk)) {
+ sk_free_unlock_clone(newsk);
+ newsk = NULL;
+ goto out;
+ }
newsk->sk_err = 0;
newsk->sk_err_soft = 0;
@@ -3287,16 +3290,17 @@ static __init int net_inuse_init(void)
core_initcall(net_inuse_init);
-static void assign_proto_idx(struct proto *prot)
+static int assign_proto_idx(struct proto *prot)
{
prot->inuse_idx = find_first_zero_bit(proto_inuse_idx, PROTO_INUSE_NR);
if (unlikely(prot->inuse_idx == PROTO_INUSE_NR - 1)) {
pr_err("PROTO_INUSE_NR exhausted\n");
- return;
+ return -ENOSPC;
}
set_bit(prot->inuse_idx, proto_inuse_idx);
+ return 0;
}
static void release_proto_idx(struct proto *prot)
@@ -3305,8 +3309,9 @@ static void release_proto_idx(struct proto *prot)
clear_bit(prot->inuse_idx, proto_inuse_idx);
}
#else
-static inline void assign_proto_idx(struct proto *prot)
+static inline int assign_proto_idx(struct proto *prot)
{
+ return 0;
}
static inline void release_proto_idx(struct proto *prot)
@@ -3355,6 +3360,8 @@ static int req_prot_init(const struct proto *prot)
int proto_register(struct proto *prot, int alloc_slab)
{
+ int ret = -ENOBUFS;
+
if (alloc_slab) {
prot->slab = kmem_cache_create_usercopy(prot->name,
prot->obj_size, 0,
@@ -3391,20 +3398,27 @@ int proto_register(struct proto *prot, int alloc_slab)
}
mutex_lock(&proto_list_mutex);
+ ret = assign_proto_idx(prot);
+ if (ret) {
+ mutex_unlock(&proto_list_mutex);
+ goto out_free_timewait_sock_slab_name;
+ }
list_add(&prot->node, &proto_list);
- assign_proto_idx(prot);
mutex_unlock(&proto_list_mutex);
- return 0;
+ return ret;
out_free_timewait_sock_slab_name:
- kfree(prot->twsk_prot->twsk_slab_name);
+ if (alloc_slab && prot->twsk_prot)
+ kfree(prot->twsk_prot->twsk_slab_name);
out_free_request_sock_slab:
- req_prot_cleanup(prot->rsk_prot);
+ if (alloc_slab) {
+ req_prot_cleanup(prot->rsk_prot);
- kmem_cache_destroy(prot->slab);
- prot->slab = NULL;
+ kmem_cache_destroy(prot->slab);
+ prot->slab = NULL;
+ }
out:
- return -ENOBUFS;
+ return ret;
}
EXPORT_SYMBOL(proto_register);
diff --git a/net/core/sock_map.c b/net/core/sock_map.c
index 1330a74..0199886 100644
--- a/net/core/sock_map.c
+++ b/net/core/sock_map.c
@@ -345,7 +345,7 @@ static int sock_map_update_common(struct bpf_map *map, u32 idx,
return -EINVAL;
if (unlikely(idx >= map->max_entries))
return -E2BIG;
- if (unlikely(icsk->icsk_ulp_data))
+ if (unlikely(rcu_access_pointer(icsk->icsk_ulp_data)))
return -EINVAL;
link = sk_psock_init_link();
diff --git a/net/core/stream.c b/net/core/stream.c
index e94bb02..4f1d4aa 100644
--- a/net/core/stream.c
+++ b/net/core/stream.c
@@ -120,7 +120,6 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_p)
int err = 0;
long vm_wait = 0;
long current_timeo = *timeo_p;
- bool noblock = (*timeo_p ? false : true);
DEFINE_WAIT_FUNC(wait, woken_wake_function);
if (sk_stream_memory_free(sk))
@@ -133,11 +132,8 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_p)
if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN))
goto do_error;
- if (!*timeo_p) {
- if (noblock)
- set_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
- goto do_nonblock;
- }
+ if (!*timeo_p)
+ goto do_eagain;
if (signal_pending(current))
goto do_interrupted;
sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk);
@@ -169,7 +165,13 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_p)
do_error:
err = -EPIPE;
goto out;
-do_nonblock:
+do_eagain:
+ /* Make sure that whenever EAGAIN is returned, EPOLLOUT event can
+ * be generated later.
+ * When TCP receives ACK packets that make room, tcp_check_space()
+ * only calls tcp_new_space() if SOCK_NOSPACE is set.
+ */
+ set_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
err = -EAGAIN;
goto out;
do_interrupted:
diff --git a/net/dsa/dsa2.c b/net/dsa/dsa2.c
index 8c4eccb..b501c90 100644
--- a/net/dsa/dsa2.c
+++ b/net/dsa/dsa2.c
@@ -259,8 +259,11 @@ static int dsa_port_setup(struct dsa_port *dp)
const unsigned char *id = (const unsigned char *)&dst->index;
const unsigned char len = sizeof(dst->index);
struct devlink_port *dlp = &dp->devlink_port;
+ bool dsa_port_link_registered = false;
+ bool devlink_port_registered = false;
struct devlink *dl = ds->devlink;
- int err;
+ bool dsa_port_enabled = false;
+ int err = 0;
switch (dp->type) {
case DSA_PORT_TYPE_UNUSED:
@@ -272,15 +275,19 @@ static int dsa_port_setup(struct dsa_port *dp)
dp->index, false, 0, id, len);
err = devlink_port_register(dl, dlp, dp->index);
if (err)
- return err;
+ break;
+ devlink_port_registered = true;
err = dsa_port_link_register_of(dp);
if (err)
- return err;
+ break;
+ dsa_port_link_registered = true;
err = dsa_port_enable(dp, NULL);
if (err)
- return err;
+ break;
+ dsa_port_enabled = true;
+
break;
case DSA_PORT_TYPE_DSA:
memset(dlp, 0, sizeof(*dlp));
@@ -288,15 +295,19 @@ static int dsa_port_setup(struct dsa_port *dp)
dp->index, false, 0, id, len);
err = devlink_port_register(dl, dlp, dp->index);
if (err)
- return err;
+ break;
+ devlink_port_registered = true;
err = dsa_port_link_register_of(dp);
if (err)
- return err;
+ break;
+ dsa_port_link_registered = true;
err = dsa_port_enable(dp, NULL);
if (err)
- return err;
+ break;
+ dsa_port_enabled = true;
+
break;
case DSA_PORT_TYPE_USER:
memset(dlp, 0, sizeof(*dlp));
@@ -304,18 +315,26 @@ static int dsa_port_setup(struct dsa_port *dp)
dp->index, false, 0, id, len);
err = devlink_port_register(dl, dlp, dp->index);
if (err)
- return err;
+ break;
+ devlink_port_registered = true;
dp->mac = of_get_mac_address(dp->dn);
err = dsa_slave_create(dp);
if (err)
- return err;
+ break;
devlink_port_type_eth_set(dlp, dp->slave);
break;
}
- return 0;
+ if (err && dsa_port_enabled)
+ dsa_port_disable(dp);
+ if (err && dsa_port_link_registered)
+ dsa_port_link_unregister_of(dp);
+ if (err && devlink_port_registered)
+ devlink_port_unregister(dlp);
+
+ return err;
}
static void dsa_port_teardown(struct dsa_port *dp)
@@ -834,20 +853,6 @@ struct dsa_switch *dsa_switch_alloc(struct device *dev, size_t n)
if (!ds)
return NULL;
- /* We avoid allocating memory outside dsa_switch
- * if it is not needed.
- */
- if (n <= sizeof(ds->_bitmap) * 8) {
- ds->bitmap = &ds->_bitmap;
- } else {
- ds->bitmap = devm_kcalloc(dev,
- BITS_TO_LONGS(n),
- sizeof(unsigned long),
- GFP_KERNEL);
- if (unlikely(!ds->bitmap))
- return NULL;
- }
-
ds->dev = dev;
ds->num_ports = n;
diff --git a/net/dsa/port.c b/net/dsa/port.c
index f753014..9b54e5a 100644
--- a/net/dsa/port.c
+++ b/net/dsa/port.c
@@ -348,10 +348,7 @@ int dsa_port_vlan_add(struct dsa_port *dp,
.vlan = vlan,
};
- if (!dp->bridge_dev || br_vlan_enabled(dp->bridge_dev))
- return dsa_port_notify(dp, DSA_NOTIFIER_VLAN_ADD, &info);
-
- return 0;
+ return dsa_port_notify(dp, DSA_NOTIFIER_VLAN_ADD, &info);
}
int dsa_port_vlan_del(struct dsa_port *dp,
@@ -363,10 +360,7 @@ int dsa_port_vlan_del(struct dsa_port *dp,
.vlan = vlan,
};
- if (!dp->bridge_dev || br_vlan_enabled(dp->bridge_dev))
- return dsa_port_notify(dp, DSA_NOTIFIER_VLAN_DEL, &info);
-
- return 0;
+ return dsa_port_notify(dp, DSA_NOTIFIER_VLAN_DEL, &info);
}
int dsa_port_vid_add(struct dsa_port *dp, u16 vid, u16 flags)
@@ -382,8 +376,8 @@ int dsa_port_vid_add(struct dsa_port *dp, u16 vid, u16 flags)
trans.ph_prepare = true;
err = dsa_port_vlan_add(dp, &vlan, &trans);
- if (err == -EOPNOTSUPP)
- return 0;
+ if (err)
+ return err;
trans.ph_prepare = false;
return dsa_port_vlan_add(dp, &vlan, &trans);
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index 33f4117..9a88035 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -312,6 +312,39 @@ static int dsa_slave_port_attr_set(struct net_device *dev,
return ret;
}
+static int dsa_slave_vlan_add(struct net_device *dev,
+ const struct switchdev_obj *obj,
+ struct switchdev_trans *trans)
+{
+ struct dsa_port *dp = dsa_slave_to_port(dev);
+ struct switchdev_obj_port_vlan vlan;
+ int err;
+
+ if (obj->orig_dev != dev)
+ return -EOPNOTSUPP;
+
+ if (dp->bridge_dev && !br_vlan_enabled(dp->bridge_dev))
+ return 0;
+
+ vlan = *SWITCHDEV_OBJ_PORT_VLAN(obj);
+
+ err = dsa_port_vlan_add(dp, &vlan, trans);
+ if (err)
+ return err;
+
+ /* We need the dedicated CPU port to be a member of the VLAN as well.
+ * Even though drivers often handle CPU membership in special ways,
+ * it doesn't make sense to program a PVID, so clear this flag.
+ */
+ vlan.flags &= ~BRIDGE_VLAN_INFO_PVID;
+
+ err = dsa_port_vlan_add(dp->cpu_dp, &vlan, trans);
+ if (err)
+ return err;
+
+ return 0;
+}
+
static int dsa_slave_port_obj_add(struct net_device *dev,
const struct switchdev_obj *obj,
struct switchdev_trans *trans,
@@ -339,10 +372,7 @@ static int dsa_slave_port_obj_add(struct net_device *dev,
trans);
break;
case SWITCHDEV_OBJ_ID_PORT_VLAN:
- if (obj->orig_dev != dev)
- return -EOPNOTSUPP;
- err = dsa_port_vlan_add(dp, SWITCHDEV_OBJ_PORT_VLAN(obj),
- trans);
+ err = dsa_slave_vlan_add(dev, obj, trans);
break;
default:
err = -EOPNOTSUPP;
@@ -352,6 +382,23 @@ static int dsa_slave_port_obj_add(struct net_device *dev,
return err;
}
+static int dsa_slave_vlan_del(struct net_device *dev,
+ const struct switchdev_obj *obj)
+{
+ struct dsa_port *dp = dsa_slave_to_port(dev);
+
+ if (obj->orig_dev != dev)
+ return -EOPNOTSUPP;
+
+ if (dp->bridge_dev && !br_vlan_enabled(dp->bridge_dev))
+ return 0;
+
+ /* Do not deprogram the CPU port as it may be shared with other user
+ * ports which can be members of this VLAN as well.
+ */
+ return dsa_port_vlan_del(dp, SWITCHDEV_OBJ_PORT_VLAN(obj));
+}
+
static int dsa_slave_port_obj_del(struct net_device *dev,
const struct switchdev_obj *obj)
{
@@ -371,9 +418,7 @@ static int dsa_slave_port_obj_del(struct net_device *dev,
err = dsa_port_mdb_del(dp->cpu_dp, SWITCHDEV_OBJ_PORT_MDB(obj));
break;
case SWITCHDEV_OBJ_ID_PORT_VLAN:
- if (obj->orig_dev != dev)
- return -EOPNOTSUPP;
- err = dsa_port_vlan_del(dp, SWITCHDEV_OBJ_PORT_VLAN(obj));
+ err = dsa_slave_vlan_del(dev, obj);
break;
default:
err = -EOPNOTSUPP;
@@ -1073,6 +1118,9 @@ static int dsa_slave_vlan_rx_add_vid(struct net_device *dev, __be16 proto,
* need to emulate the switchdev prepare + commit phase.
*/
if (dp->bridge_dev) {
+ if (!br_vlan_enabled(dp->bridge_dev))
+ return 0;
+
/* br_vlan_get_info() returns -EINVAL or -ENOENT if the
* device, respectively the VID is not found, returning
* 0 means success, which is a failure for us here.
@@ -1082,8 +1130,15 @@ static int dsa_slave_vlan_rx_add_vid(struct net_device *dev, __be16 proto,
return -EBUSY;
}
- /* This API only allows programming tagged, non-PVID VIDs */
- return dsa_port_vid_add(dp, vid, 0);
+ ret = dsa_port_vid_add(dp, vid, 0);
+ if (ret)
+ return ret;
+
+ ret = dsa_port_vid_add(dp->cpu_dp, vid, 0);
+ if (ret)
+ return ret;
+
+ return 0;
}
static int dsa_slave_vlan_rx_kill_vid(struct net_device *dev, __be16 proto,
@@ -1097,6 +1152,9 @@ static int dsa_slave_vlan_rx_kill_vid(struct net_device *dev, __be16 proto,
* need to emulate the switchdev prepare + commit phase.
*/
if (dp->bridge_dev) {
+ if (!br_vlan_enabled(dp->bridge_dev))
+ return 0;
+
/* br_vlan_get_info() returns -EINVAL or -ENOENT if the
* device, respectively the VID is not found, returning
* 0 means success, which is a failure for us here.
@@ -1106,11 +1164,10 @@ static int dsa_slave_vlan_rx_kill_vid(struct net_device *dev, __be16 proto,
return -EBUSY;
}
- ret = dsa_port_vid_del(dp, vid);
- if (ret == -EOPNOTSUPP)
- ret = 0;
-
- return ret;
+ /* Do not deprogram the CPU port as it may be shared with other user
+ * ports which can be members of this VLAN as well.
+ */
+ return dsa_port_vid_del(dp, vid);
}
static const struct ethtool_ops dsa_slave_ethtool_ops = {
@@ -1357,8 +1414,9 @@ int dsa_slave_create(struct dsa_port *port)
if (slave_dev == NULL)
return -ENOMEM;
- slave_dev->features = master->vlan_features | NETIF_F_HW_TC |
- NETIF_F_HW_VLAN_CTAG_FILTER;
+ slave_dev->features = master->vlan_features | NETIF_F_HW_TC;
+ if (ds->ops->port_vlan_add && ds->ops->port_vlan_del)
+ slave_dev->features |= NETIF_F_HW_VLAN_CTAG_FILTER;
slave_dev->hw_features |= NETIF_F_HW_TC;
slave_dev->ethtool_ops = &dsa_slave_ethtool_ops;
if (!IS_ERR_OR_NULL(port->mac))
diff --git a/net/dsa/switch.c b/net/dsa/switch.c
index 09d9286..6a96075 100644
--- a/net/dsa/switch.c
+++ b/net/dsa/switch.c
@@ -128,57 +128,51 @@ static int dsa_switch_fdb_del(struct dsa_switch *ds,
return ds->ops->port_fdb_del(ds, port, info->addr, info->vid);
}
-static int
-dsa_switch_mdb_prepare_bitmap(struct dsa_switch *ds,
- const struct switchdev_obj_port_mdb *mdb,
- const unsigned long *bitmap)
+static bool dsa_switch_mdb_match(struct dsa_switch *ds, int port,
+ struct dsa_notifier_mdb_info *info)
+{
+ if (ds->index == info->sw_index && port == info->port)
+ return true;
+
+ if (dsa_is_dsa_port(ds, port))
+ return true;
+
+ return false;
+}
+
+static int dsa_switch_mdb_prepare(struct dsa_switch *ds,
+ struct dsa_notifier_mdb_info *info)
{
int port, err;
if (!ds->ops->port_mdb_prepare || !ds->ops->port_mdb_add)
return -EOPNOTSUPP;
- for_each_set_bit(port, bitmap, ds->num_ports) {
- err = ds->ops->port_mdb_prepare(ds, port, mdb);
- if (err)
- return err;
+ for (port = 0; port < ds->num_ports; port++) {
+ if (dsa_switch_mdb_match(ds, port, info)) {
+ err = ds->ops->port_mdb_prepare(ds, port, info->mdb);
+ if (err)
+ return err;
+ }
}
return 0;
}
-static void dsa_switch_mdb_add_bitmap(struct dsa_switch *ds,
- const struct switchdev_obj_port_mdb *mdb,
- const unsigned long *bitmap)
-{
- int port;
-
- if (!ds->ops->port_mdb_add)
- return;
-
- for_each_set_bit(port, bitmap, ds->num_ports)
- ds->ops->port_mdb_add(ds, port, mdb);
-}
-
static int dsa_switch_mdb_add(struct dsa_switch *ds,
struct dsa_notifier_mdb_info *info)
{
- const struct switchdev_obj_port_mdb *mdb = info->mdb;
- struct switchdev_trans *trans = info->trans;
int port;
- /* Build a mask of Multicast group members */
- bitmap_zero(ds->bitmap, ds->num_ports);
- if (ds->index == info->sw_index)
- set_bit(info->port, ds->bitmap);
+ if (switchdev_trans_ph_prepare(info->trans))
+ return dsa_switch_mdb_prepare(ds, info);
+
+ if (!ds->ops->port_mdb_add)
+ return 0;
+
for (port = 0; port < ds->num_ports; port++)
- if (dsa_is_dsa_port(ds, port))
- set_bit(port, ds->bitmap);
-
- if (switchdev_trans_ph_prepare(trans))
- return dsa_switch_mdb_prepare_bitmap(ds, mdb, ds->bitmap);
-
- dsa_switch_mdb_add_bitmap(ds, mdb, ds->bitmap);
+ if (dsa_switch_mdb_match(ds, port, info))
+ ds->ops->port_mdb_add(ds, port, info->mdb);
return 0;
}
@@ -186,13 +180,11 @@ static int dsa_switch_mdb_add(struct dsa_switch *ds,
static int dsa_switch_mdb_del(struct dsa_switch *ds,
struct dsa_notifier_mdb_info *info)
{
- const struct switchdev_obj_port_mdb *mdb = info->mdb;
-
if (!ds->ops->port_mdb_del)
return -EOPNOTSUPP;
if (ds->index == info->sw_index)
- return ds->ops->port_mdb_del(ds, info->port, mdb);
+ return ds->ops->port_mdb_del(ds, info->port, info->mdb);
return 0;
}
@@ -234,59 +226,55 @@ static int dsa_port_vlan_check(struct dsa_switch *ds, int port,
(void *)vlan);
}
-static int
-dsa_switch_vlan_prepare_bitmap(struct dsa_switch *ds,
- const struct switchdev_obj_port_vlan *vlan,
- const unsigned long *bitmap)
+static bool dsa_switch_vlan_match(struct dsa_switch *ds, int port,
+ struct dsa_notifier_vlan_info *info)
+{
+ if (ds->index == info->sw_index && port == info->port)
+ return true;
+
+ if (dsa_is_dsa_port(ds, port))
+ return true;
+
+ return false;
+}
+
+static int dsa_switch_vlan_prepare(struct dsa_switch *ds,
+ struct dsa_notifier_vlan_info *info)
{
int port, err;
if (!ds->ops->port_vlan_prepare || !ds->ops->port_vlan_add)
return -EOPNOTSUPP;
- for_each_set_bit(port, bitmap, ds->num_ports) {
- err = dsa_port_vlan_check(ds, port, vlan);
- if (err)
- return err;
+ for (port = 0; port < ds->num_ports; port++) {
+ if (dsa_switch_vlan_match(ds, port, info)) {
+ err = dsa_port_vlan_check(ds, port, info->vlan);
+ if (err)
+ return err;
- err = ds->ops->port_vlan_prepare(ds, port, vlan);
- if (err)
- return err;
+ err = ds->ops->port_vlan_prepare(ds, port, info->vlan);
+ if (err)
+ return err;
+ }
}
return 0;
}
-static void
-dsa_switch_vlan_add_bitmap(struct dsa_switch *ds,
- const struct switchdev_obj_port_vlan *vlan,
- const unsigned long *bitmap)
-{
- int port;
-
- for_each_set_bit(port, bitmap, ds->num_ports)
- ds->ops->port_vlan_add(ds, port, vlan);
-}
-
static int dsa_switch_vlan_add(struct dsa_switch *ds,
struct dsa_notifier_vlan_info *info)
{
- const struct switchdev_obj_port_vlan *vlan = info->vlan;
- struct switchdev_trans *trans = info->trans;
int port;
- /* Build a mask of VLAN members */
- bitmap_zero(ds->bitmap, ds->num_ports);
- if (ds->index == info->sw_index)
- set_bit(info->port, ds->bitmap);
+ if (switchdev_trans_ph_prepare(info->trans))
+ return dsa_switch_vlan_prepare(ds, info);
+
+ if (!ds->ops->port_vlan_add)
+ return 0;
+
for (port = 0; port < ds->num_ports; port++)
- if (dsa_is_cpu_port(ds, port) || dsa_is_dsa_port(ds, port))
- set_bit(port, ds->bitmap);
-
- if (switchdev_trans_ph_prepare(trans))
- return dsa_switch_vlan_prepare_bitmap(ds, vlan, ds->bitmap);
-
- dsa_switch_vlan_add_bitmap(ds, vlan, ds->bitmap);
+ if (dsa_switch_vlan_match(ds, port, info))
+ ds->ops->port_vlan_add(ds, port, info->vlan);
return 0;
}
@@ -294,14 +282,15 @@ static int dsa_switch_vlan_add(struct dsa_switch *ds,
static int dsa_switch_vlan_del(struct dsa_switch *ds,
struct dsa_notifier_vlan_info *info)
{
- const struct switchdev_obj_port_vlan *vlan = info->vlan;
-
if (!ds->ops->port_vlan_del)
return -EOPNOTSUPP;
if (ds->index == info->sw_index)
- return ds->ops->port_vlan_del(ds, info->port, vlan);
+ return ds->ops->port_vlan_del(ds, info->port, info->vlan);
+ /* Do not deprogram the DSA links as they may be used as conduit
+ * for other VLAN members in the fabric.
+ */
return 0;
}
diff --git a/net/dsa/tag_8021q.c b/net/dsa/tag_8021q.c
index 6ebbd79..9c1cc24 100644
--- a/net/dsa/tag_8021q.c
+++ b/net/dsa/tag_8021q.c
@@ -28,6 +28,7 @@
*
* RSV - VID[9]:
* To be used for further expansion of SWITCH_ID or for other purposes.
+ * Must be transmitted as zero and ignored on receive.
*
* SWITCH_ID - VID[8:6]:
* Index of switch within DSA tree. Must be between 0 and
@@ -35,6 +36,7 @@
*
* RSV - VID[5:4]:
* To be used for further expansion of PORT or for other purposes.
+ * Must be transmitted as zero and ignored on receive.
*
* PORT - VID[3:0]:
* Index of switch port. Must be between 0 and DSA_MAX_PORTS - 1.
@@ -91,6 +93,79 @@ int dsa_8021q_rx_source_port(u16 vid)
}
EXPORT_SYMBOL_GPL(dsa_8021q_rx_source_port);
+static int dsa_8021q_restore_pvid(struct dsa_switch *ds, int port)
+{
+ struct bridge_vlan_info vinfo;
+ struct net_device *slave;
+ u16 pvid;
+ int err;
+
+ if (!dsa_is_user_port(ds, port))
+ return 0;
+
+ slave = ds->ports[port].slave;
+
+ err = br_vlan_get_pvid(slave, &pvid);
+ if (err < 0)
+ /* There is no pvid on the bridge for this port, which is
+ * perfectly valid. Nothing to restore, bye-bye!
+ */
+ return 0;
+
+ err = br_vlan_get_info(slave, pvid, &vinfo);
+ if (err < 0) {
+ dev_err(ds->dev, "Couldn't determine PVID attributes\n");
+ return err;
+ }
+
+ return dsa_port_vid_add(&ds->ports[port], pvid, vinfo.flags);
+}
+
+/* If @enabled is true, installs @vid with @flags into the switch port's HW
+ * filter.
+ * If @enabled is false, deletes @vid (ignores @flags) from the port. Had the
+ * user explicitly configured this @vid through the bridge core, then the @vid
+ * is installed again, but this time with the flags from the bridge layer.
+ */
+static int dsa_8021q_vid_apply(struct dsa_switch *ds, int port, u16 vid,
+ u16 flags, bool enabled)
+{
+ struct dsa_port *dp = &ds->ports[port];
+ struct bridge_vlan_info vinfo;
+ int err;
+
+ if (enabled)
+ return dsa_port_vid_add(dp, vid, flags);
+
+ err = dsa_port_vid_del(dp, vid);
+ if (err < 0)
+ return err;
+
+ /* Nothing to restore from the bridge for a non-user port.
+ * The CPU port VLANs are restored implicitly with the user ports,
+ * similar to how the bridge does in dsa_slave_vlan_add and
+ * dsa_slave_vlan_del.
+ */
+ if (!dsa_is_user_port(ds, port))
+ return 0;
+
+ err = br_vlan_get_info(dp->slave, vid, &vinfo);
+ /* Couldn't determine bridge attributes for this vid,
+ * it means the bridge had not configured it.
+ */
+ if (err < 0)
+ return 0;
+
+ /* Restore the VID from the bridge */
+ err = dsa_port_vid_add(dp, vid, vinfo.flags);
+ if (err < 0)
+ return err;
+
+ vinfo.flags &= ~BRIDGE_VLAN_INFO_PVID;
+
+ return dsa_port_vid_add(dp->cpu_dp, vid, vinfo.flags);
+}
+
/* RX VLAN tagging (left) and TX VLAN tagging (right) setup shown for a single
* front-panel switch port (here swp0).
*
@@ -146,8 +221,6 @@ EXPORT_SYMBOL_GPL(dsa_8021q_rx_source_port);
int dsa_port_setup_8021q_tagging(struct dsa_switch *ds, int port, bool enabled)
{
int upstream = dsa_upstream_port(ds, port);
- struct dsa_port *dp = &ds->ports[port];
- struct dsa_port *upstream_dp = &ds->ports[upstream];
u16 rx_vid = dsa_8021q_rx_vid(ds, port);
u16 tx_vid = dsa_8021q_tx_vid(ds, port);
int i, err;
@@ -164,7 +237,6 @@ int dsa_port_setup_8021q_tagging(struct dsa_switch *ds, int port, bool enabled)
* restrictions, so there are no concerns about leaking traffic.
*/
for (i = 0; i < ds->num_ports; i++) {
- struct dsa_port *other_dp = &ds->ports[i];
u16 flags;
if (i == upstream)
@@ -177,10 +249,7 @@ int dsa_port_setup_8021q_tagging(struct dsa_switch *ds, int port, bool enabled)
/* The RX VID is a regular VLAN on all others */
flags = BRIDGE_VLAN_INFO_UNTAGGED;
- if (enabled)
- err = dsa_port_vid_add(other_dp, rx_vid, flags);
- else
- err = dsa_port_vid_del(other_dp, rx_vid);
+ err = dsa_8021q_vid_apply(ds, i, rx_vid, flags, enabled);
if (err) {
dev_err(ds->dev, "Failed to apply RX VID %d to port %d: %d\n",
rx_vid, port, err);
@@ -191,10 +260,7 @@ int dsa_port_setup_8021q_tagging(struct dsa_switch *ds, int port, bool enabled)
/* CPU port needs to see this port's RX VID
* as tagged egress.
*/
- if (enabled)
- err = dsa_port_vid_add(upstream_dp, rx_vid, 0);
- else
- err = dsa_port_vid_del(upstream_dp, rx_vid);
+ err = dsa_8021q_vid_apply(ds, upstream, rx_vid, 0, enabled);
if (err) {
dev_err(ds->dev, "Failed to apply RX VID %d to port %d: %d\n",
rx_vid, port, err);
@@ -202,26 +268,24 @@ int dsa_port_setup_8021q_tagging(struct dsa_switch *ds, int port, bool enabled)
}
/* Finally apply the TX VID on this port and on the CPU port */
- if (enabled)
- err = dsa_port_vid_add(dp, tx_vid, BRIDGE_VLAN_INFO_UNTAGGED);
- else
- err = dsa_port_vid_del(dp, tx_vid);
+ err = dsa_8021q_vid_apply(ds, port, tx_vid, BRIDGE_VLAN_INFO_UNTAGGED,
+ enabled);
if (err) {
dev_err(ds->dev, "Failed to apply TX VID %d on port %d: %d\n",
tx_vid, port, err);
return err;
}
- if (enabled)
- err = dsa_port_vid_add(upstream_dp, tx_vid, 0);
- else
- err = dsa_port_vid_del(upstream_dp, tx_vid);
+ err = dsa_8021q_vid_apply(ds, upstream, tx_vid, 0, enabled);
if (err) {
dev_err(ds->dev, "Failed to apply TX VID %d on port %d: %d\n",
tx_vid, upstream, err);
return err;
}
- return 0;
+ if (!enabled)
+ err = dsa_8021q_restore_pvid(ds, port);
+
+ return err;
}
EXPORT_SYMBOL_GPL(dsa_port_setup_8021q_tagging);
diff --git a/net/ieee802154/socket.c b/net/ieee802154/socket.c
index dacbd58..badc5cf 100644
--- a/net/ieee802154/socket.c
+++ b/net/ieee802154/socket.c
@@ -1092,7 +1092,7 @@ static struct packet_type ieee802154_packet_type = {
static int __init af_ieee802154_init(void)
{
- int rc = -EINVAL;
+ int rc;
rc = proto_register(&ieee802154_raw_prot, 1);
if (rc)
diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c
index 2b2b3d2..1ab2fb6b 100644
--- a/net/ipv4/fib_trie.c
+++ b/net/ipv4/fib_trie.c
@@ -2145,7 +2145,7 @@ static int fn_trie_dump_leaf(struct key_vector *l, struct fib_table *tb,
if (filter->dump_exceptions) {
err = fib_dump_info_fnhe(skb, cb, tb->tb_id, fi,
- &i_fa, s_fa);
+ &i_fa, s_fa, flags);
if (err < 0)
goto stop;
}
diff --git a/net/ipv4/icmp.c b/net/ipv4/icmp.c
index 1510e95..4298aae 100644
--- a/net/ipv4/icmp.c
+++ b/net/ipv4/icmp.c
@@ -582,7 +582,13 @@ void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info,
if (!rt)
goto out;
- net = dev_net(rt->dst.dev);
+
+ if (rt->dst.dev)
+ net = dev_net(rt->dst.dev);
+ else if (skb_in->dev)
+ net = dev_net(skb_in->dev);
+ else
+ goto out;
/*
* Find the original header. It is expected to be valid, of course.
@@ -902,7 +908,7 @@ static bool icmp_redirect(struct sk_buff *skb)
return false;
}
- icmp_socket_deliver(skb, icmp_hdr(skb)->un.gateway);
+ icmp_socket_deliver(skb, ntohl(icmp_hdr(skb)->un.gateway));
return true;
}
diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c
index 180f689..480d0b2 100644
--- a/net/ipv4/igmp.c
+++ b/net/ipv4/igmp.c
@@ -1475,7 +1475,7 @@ EXPORT_SYMBOL(__ip_mc_inc_group);
void ip_mc_inc_group(struct in_device *in_dev, __be32 addr)
{
- __ip_mc_inc_group(in_dev, addr, MCAST_EXCLUDE);
+ __ip_mc_inc_group(in_dev, addr, GFP_KERNEL);
}
EXPORT_SYMBOL(ip_mc_inc_group);
@@ -2197,7 +2197,7 @@ static int __ip_mc_join_group(struct sock *sk, struct ip_mreqn *imr,
iml->sflist = NULL;
iml->sfmode = mode;
rcu_assign_pointer(inet->mc_list, iml);
- __ip_mc_inc_group(in_dev, addr, mode);
+ ____ip_mc_inc_group(in_dev, addr, mode, GFP_KERNEL);
err = 0;
done:
return err;
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index c07bc82..313470f 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -1134,8 +1134,8 @@ static int ipmr_cache_unresolved(struct mr_table *mrt, vifi_t vifi,
if (!found) {
/* Create a new entry if allowable */
- if (atomic_read(&mrt->cache_resolve_queue_len) >= 10 ||
- (c = ipmr_cache_alloc_unres()) == NULL) {
+ c = ipmr_cache_alloc_unres();
+ if (!c) {
spin_unlock_bh(&mfc_unres_lock);
kfree_skb(skb);
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 517300d..b6a6f18 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -2728,7 +2728,8 @@ EXPORT_SYMBOL_GPL(ip_route_output_flow);
/* called with rcu_read_lock held */
static int rt_fill_info(struct net *net, __be32 dst, __be32 src,
struct rtable *rt, u32 table_id, struct flowi4 *fl4,
- struct sk_buff *skb, u32 portid, u32 seq)
+ struct sk_buff *skb, u32 portid, u32 seq,
+ unsigned int flags)
{
struct rtmsg *r;
struct nlmsghdr *nlh;
@@ -2736,7 +2737,7 @@ static int rt_fill_info(struct net *net, __be32 dst, __be32 src,
u32 error;
u32 metrics[RTAX_MAX];
- nlh = nlmsg_put(skb, portid, seq, RTM_NEWROUTE, sizeof(*r), 0);
+ nlh = nlmsg_put(skb, portid, seq, RTM_NEWROUTE, sizeof(*r), flags);
if (!nlh)
return -EMSGSIZE;
@@ -2860,7 +2861,7 @@ static int rt_fill_info(struct net *net, __be32 dst, __be32 src,
static int fnhe_dump_bucket(struct net *net, struct sk_buff *skb,
struct netlink_callback *cb, u32 table_id,
struct fnhe_hash_bucket *bucket, int genid,
- int *fa_index, int fa_start)
+ int *fa_index, int fa_start, unsigned int flags)
{
int i;
@@ -2891,7 +2892,7 @@ static int fnhe_dump_bucket(struct net *net, struct sk_buff *skb,
err = rt_fill_info(net, fnhe->fnhe_daddr, 0, rt,
table_id, NULL, skb,
NETLINK_CB(cb->skb).portid,
- cb->nlh->nlmsg_seq);
+ cb->nlh->nlmsg_seq, flags);
if (err)
return err;
next:
@@ -2904,7 +2905,7 @@ static int fnhe_dump_bucket(struct net *net, struct sk_buff *skb,
int fib_dump_info_fnhe(struct sk_buff *skb, struct netlink_callback *cb,
u32 table_id, struct fib_info *fi,
- int *fa_index, int fa_start)
+ int *fa_index, int fa_start, unsigned int flags)
{
struct net *net = sock_net(cb->skb->sk);
int nhsel, genid = fnhe_genid(net);
@@ -2922,7 +2923,8 @@ int fib_dump_info_fnhe(struct sk_buff *skb, struct netlink_callback *cb,
err = 0;
if (bucket)
err = fnhe_dump_bucket(net, skb, cb, table_id, bucket,
- genid, fa_index, fa_start);
+ genid, fa_index, fa_start,
+ flags);
rcu_read_unlock();
if (err)
return err;
@@ -3183,7 +3185,8 @@ static int inet_rtm_getroute(struct sk_buff *in_skb, struct nlmsghdr *nlh,
fl4.flowi4_tos, res.fi, 0);
} else {
err = rt_fill_info(net, dst, src, rt, table_id, &fl4, skb,
- NETLINK_CB(in_skb).portid, nlh->nlmsg_seq);
+ NETLINK_CB(in_skb).portid,
+ nlh->nlmsg_seq, 0);
}
if (err < 0)
goto errout_rcu;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 051ef10..94df48b 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -935,6 +935,22 @@ static int tcp_send_mss(struct sock *sk, int *size_goal, int flags)
return mss_now;
}
+/* In some cases, both sendpage() and sendmsg() could have added
+ * an skb to the write queue, but failed adding payload on it.
+ * We need to remove it to consume less memory, but more
+ * importantly be able to generate EPOLLOUT for Edge Trigger epoll()
+ * users.
+ */
+static void tcp_remove_empty_skb(struct sock *sk, struct sk_buff *skb)
+{
+ if (skb && !skb->len) {
+ tcp_unlink_write_queue(skb, sk);
+ if (tcp_write_queue_empty(sk))
+ tcp_chrono_stop(sk, TCP_CHRONO_BUSY);
+ sk_wmem_free_skb(sk, skb);
+ }
+}
+
ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset,
size_t size, int flags)
{
@@ -1064,6 +1080,7 @@ ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset,
return copied;
do_error:
+ tcp_remove_empty_skb(sk, tcp_write_queue_tail(sk));
if (copied)
goto out;
out_err:
@@ -1389,18 +1406,11 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
sock_zerocopy_put(uarg);
return copied + copied_syn;
-do_fault:
- if (!skb->len) {
- tcp_unlink_write_queue(skb, sk);
- /* It is the one place in all of TCP, except connection
- * reset, where we can be unlinking the send_head.
- */
- if (tcp_write_queue_empty(sk))
- tcp_chrono_stop(sk, TCP_CHRONO_BUSY);
- sk_wmem_free_skb(sk, skb);
- }
-
do_error:
+ skb = tcp_write_queue_tail(sk);
+do_fault:
+ tcp_remove_empty_skb(sk, skb);
+
if (copied + copied_syn)
goto out;
out_err:
diff --git a/net/ipv4/tcp_bbr.c b/net/ipv4/tcp_bbr.c
index 56be7d2..95b5954 100644
--- a/net/ipv4/tcp_bbr.c
+++ b/net/ipv4/tcp_bbr.c
@@ -346,7 +346,7 @@ static void bbr_cwnd_event(struct sock *sk, enum tcp_ca_event event)
/* Calculate bdp based on min RTT and the estimated bottleneck bandwidth:
*
- * bdp = bw * min_rtt * gain
+ * bdp = ceil(bw * min_rtt * gain)
*
* The key factor, gain, controls the amount of queue. While a small gain
* builds a smaller queue, it becomes more vulnerable to noise in RTT
@@ -370,7 +370,9 @@ static u32 bbr_bdp(struct sock *sk, u32 bw, int gain)
w = (u64)bw * bbr->min_rtt_us;
- /* Apply a gain to the given value, then remove the BW_SCALE shift. */
+ /* Apply a gain to the given value, remove the BW_SCALE shift, and
+ * round the value up to avoid a negative feedback loop.
+ */
bdp = (((w * gain) >> BBR_SCALE) + BW_UNIT - 1) / BW_UNIT;
return bdp;
diff --git a/net/ipv4/tcp_diag.c b/net/ipv4/tcp_diag.c
index a3a3862..81a8221 100644
--- a/net/ipv4/tcp_diag.c
+++ b/net/ipv4/tcp_diag.c
@@ -81,13 +81,42 @@ static int tcp_diag_put_md5sig(struct sk_buff *skb,
}
#endif
+static int tcp_diag_put_ulp(struct sk_buff *skb, struct sock *sk,
+ const struct tcp_ulp_ops *ulp_ops)
+{
+ struct nlattr *nest;
+ int err;
+
+ nest = nla_nest_start_noflag(skb, INET_DIAG_ULP_INFO);
+ if (!nest)
+ return -EMSGSIZE;
+
+ err = nla_put_string(skb, INET_ULP_INFO_NAME, ulp_ops->name);
+ if (err)
+ goto nla_failure;
+
+ if (ulp_ops->get_info)
+ err = ulp_ops->get_info(sk, skb);
+ if (err)
+ goto nla_failure;
+
+ nla_nest_end(skb, nest);
+ return 0;
+
+nla_failure:
+ nla_nest_cancel(skb, nest);
+ return err;
+}
+
static int tcp_diag_get_aux(struct sock *sk, bool net_admin,
struct sk_buff *skb)
{
+ struct inet_connection_sock *icsk = inet_csk(sk);
+ int err = 0;
+
#ifdef CONFIG_TCP_MD5SIG
if (net_admin) {
struct tcp_md5sig_info *md5sig;
- int err = 0;
rcu_read_lock();
md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
@@ -99,11 +128,21 @@ static int tcp_diag_get_aux(struct sock *sk, bool net_admin,
}
#endif
+ if (net_admin) {
+ const struct tcp_ulp_ops *ulp_ops;
+
+ ulp_ops = icsk->icsk_ulp_ops;
+ if (ulp_ops)
+ err = tcp_diag_put_ulp(skb, sk, ulp_ops);
+ if (err)
+ return err;
+ }
return 0;
}
static size_t tcp_diag_get_aux_size(struct sock *sk, bool net_admin)
{
+ struct inet_connection_sock *icsk = inet_csk(sk);
size_t size = 0;
#ifdef CONFIG_TCP_MD5SIG
@@ -124,6 +163,17 @@ static size_t tcp_diag_get_aux_size(struct sock *sk, bool net_admin)
}
#endif
+ if (net_admin && sk_fullsock(sk)) {
+ const struct tcp_ulp_ops *ulp_ops;
+
+ ulp_ops = icsk->icsk_ulp_ops;
+ if (ulp_ops) {
+ size += nla_total_size(0) +
+ nla_total_size(TCP_ULP_NAME_MAX);
+ if (ulp_ops->get_info_size)
+ size += ulp_ops->get_info_size(sk);
+ }
+ }
return size;
}
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 5c46bc4..42abc9b 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -2053,7 +2053,7 @@ static bool tcp_can_coalesce_send_queue_head(struct sock *sk, int len)
if (len <= skb->len)
break;
- if (unlikely(TCP_SKB_CB(skb)->eor))
+ if (unlikely(TCP_SKB_CB(skb)->eor) || tcp_has_tx_tstamp(skb))
return false;
len -= skb->len;
@@ -2170,6 +2170,7 @@ static int tcp_mtu_probe(struct sock *sk)
* we need to propagate it to the new skb.
*/
TCP_SKB_CB(nskb)->eor = TCP_SKB_CB(skb)->eor;
+ tcp_skb_collapse_tstamp(nskb, skb);
tcp_unlink_write_queue(skb, sk);
sk_wmem_free_skb(sk, skb);
} else {
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index dc73888..6a576ff 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -478,7 +478,7 @@ static struct inet6_dev *ipv6_find_idev(struct net_device *dev)
if (!idev) {
idev = ipv6_add_dev(dev);
if (IS_ERR(idev))
- return NULL;
+ return idev;
}
if (dev->flags&IFF_UP)
@@ -1045,7 +1045,8 @@ ipv6_add_addr(struct inet6_dev *idev, struct ifa6_config *cfg,
int err = 0;
if (addr_type == IPV6_ADDR_ANY ||
- addr_type & IPV6_ADDR_MULTICAST ||
+ (addr_type & IPV6_ADDR_MULTICAST &&
+ !(cfg->ifa_flags & IFA_F_MCAUTOJOIN)) ||
(!(idev->dev->flags & IFF_LOOPBACK) &&
!netif_is_l3_master(idev->dev) &&
addr_type & IPV6_ADDR_LOOPBACK))
@@ -2465,8 +2466,8 @@ static struct inet6_dev *addrconf_add_dev(struct net_device *dev)
ASSERT_RTNL();
idev = ipv6_find_idev(dev);
- if (!idev)
- return ERR_PTR(-ENOBUFS);
+ if (IS_ERR(idev))
+ return idev;
if (idev->cnf.disable_ipv6)
return ERR_PTR(-EACCES);
@@ -3158,7 +3159,7 @@ static void init_loopback(struct net_device *dev)
ASSERT_RTNL();
idev = ipv6_find_idev(dev);
- if (!idev) {
+ if (IS_ERR(idev)) {
pr_debug("%s: add_dev failed\n", __func__);
return;
}
@@ -3373,7 +3374,7 @@ static void addrconf_sit_config(struct net_device *dev)
*/
idev = ipv6_find_idev(dev);
- if (!idev) {
+ if (IS_ERR(idev)) {
pr_debug("%s: add_dev failed\n", __func__);
return;
}
@@ -3398,7 +3399,7 @@ static void addrconf_gre_config(struct net_device *dev)
ASSERT_RTNL();
idev = ipv6_find_idev(dev);
- if (!idev) {
+ if (IS_ERR(idev)) {
pr_debug("%s: add_dev failed\n", __func__);
return;
}
@@ -4772,8 +4773,8 @@ inet6_rtm_newaddr(struct sk_buff *skb, struct nlmsghdr *nlh,
IFA_F_MCAUTOJOIN | IFA_F_OPTIMISTIC;
idev = ipv6_find_idev(dev);
- if (!idev)
- return -ENOBUFS;
+ if (IS_ERR(idev))
+ return PTR_ERR(idev);
if (!ipv6_allow_optimistic_dad(net, idev))
cfg.ifa_flags &= ~IFA_F_OPTIMISTIC;
diff --git a/net/ipv6/ip6_input.c b/net/ipv6/ip6_input.c
index fa014d5..d432d00 100644
--- a/net/ipv6/ip6_input.c
+++ b/net/ipv6/ip6_input.c
@@ -80,8 +80,10 @@ static void ip6_sublist_rcv_finish(struct list_head *head)
{
struct sk_buff *skb, *next;
- list_for_each_entry_safe(skb, next, head, list)
+ list_for_each_entry_safe(skb, next, head, list) {
+ skb_list_del_init(skb);
dst_input(skb);
+ }
}
static void ip6_list_rcv_finish(struct net *net, struct sock *sk,
diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c
index e80d36c..857a89a 100644
--- a/net/ipv6/ip6mr.c
+++ b/net/ipv6/ip6mr.c
@@ -1148,8 +1148,8 @@ static int ip6mr_cache_unresolved(struct mr_table *mrt, mifi_t mifi,
* Create a new entry if allowable
*/
- if (atomic_read(&mrt->cache_resolve_queue_len) >= 10 ||
- (c = ip6mr_cache_alloc_unres()) == NULL) {
+ c = ip6mr_cache_alloc_unres();
+ if (!c) {
spin_unlock_bh(&mfc_unres_lock);
kfree_skb(skb);
diff --git a/net/ipv6/mcast.c b/net/ipv6/mcast.c
index 7f3f13c..eaa4c2cc 100644
--- a/net/ipv6/mcast.c
+++ b/net/ipv6/mcast.c
@@ -787,14 +787,15 @@ static void mld_del_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
if (pmc) {
im->idev = pmc->idev;
if (im->mca_sfmode == MCAST_INCLUDE) {
- im->mca_tomb = pmc->mca_tomb;
- im->mca_sources = pmc->mca_sources;
+ swap(im->mca_tomb, pmc->mca_tomb);
+ swap(im->mca_sources, pmc->mca_sources);
for (psf = im->mca_sources; psf; psf = psf->sf_next)
psf->sf_crcount = idev->mc_qrv;
} else {
im->mca_crcount = idev->mc_qrv;
}
in6_dev_put(pmc->idev);
+ ip6_mc_clear_src(pmc);
kfree(pmc);
}
spin_unlock_bh(&im->mca_lock);
diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c
index ed56b0c..7c6edb7 100644
--- a/net/mac80211/cfg.c
+++ b/net/mac80211/cfg.c
@@ -1549,6 +1549,11 @@ static int ieee80211_add_station(struct wiphy *wiphy, struct net_device *dev,
if (!is_valid_ether_addr(mac))
return -EINVAL;
+ if (params->sta_flags_set & BIT(NL80211_STA_FLAG_TDLS_PEER) &&
+ sdata->vif.type == NL80211_IFTYPE_STATION &&
+ !sdata->u.mgd.associated)
+ return -EINVAL;
+
sta = sta_info_alloc(sdata, mac, GFP_KERNEL);
if (!sta)
return -ENOMEM;
@@ -1556,10 +1561,6 @@ static int ieee80211_add_station(struct wiphy *wiphy, struct net_device *dev,
if (params->sta_flags_set & BIT(NL80211_STA_FLAG_TDLS_PEER))
sta->sta.tdls = true;
- if (sta->sta.tdls && sdata->vif.type == NL80211_IFTYPE_STATION &&
- !sdata->u.mgd.associated)
- return -EINVAL;
-
err = sta_apply_parameters(local, sta, params);
if (err) {
sta_info_free(local, sta);
diff --git a/net/mac80211/rx.c b/net/mac80211/rx.c
index 3c1ab87..768d14c 100644
--- a/net/mac80211/rx.c
+++ b/net/mac80211/rx.c
@@ -2447,11 +2447,13 @@ static void ieee80211_deliver_skb_to_local_stack(struct sk_buff *skb,
skb->protocol == cpu_to_be16(ETH_P_PREAUTH)) &&
sdata->control_port_over_nl80211)) {
struct ieee80211_rx_status *status = IEEE80211_SKB_RXCB(skb);
- bool noencrypt = status->flag & RX_FLAG_DECRYPTED;
+ bool noencrypt = !(status->flag & RX_FLAG_DECRYPTED);
cfg80211_rx_control_port(dev, skb, noencrypt);
dev_kfree_skb(skb);
} else {
+ memset(skb->cb, 0, sizeof(skb->cb));
+
/* deliver to local stack */
if (rx->napi)
napi_gro_receive(rx->napi, skb);
@@ -2546,8 +2548,6 @@ ieee80211_deliver_skb(struct ieee80211_rx_data *rx)
if (skb) {
skb->protocol = eth_type_trans(skb, dev);
- memset(skb->cb, 0, sizeof(skb->cb));
-
ieee80211_deliver_skb_to_local_stack(skb, rx);
}
diff --git a/net/mpls/mpls_iptunnel.c b/net/mpls/mpls_iptunnel.c
index d25e91d..44b6750 100644
--- a/net/mpls/mpls_iptunnel.c
+++ b/net/mpls/mpls_iptunnel.c
@@ -133,12 +133,12 @@ static int mpls_xmit(struct sk_buff *skb)
mpls_stats_inc_outucastpkts(out_dev, skb);
if (rt) {
- if (rt->rt_gw_family == AF_INET)
- err = neigh_xmit(NEIGH_ARP_TABLE, out_dev, &rt->rt_gw4,
- skb);
- else if (rt->rt_gw_family == AF_INET6)
+ if (rt->rt_gw_family == AF_INET6)
err = neigh_xmit(NEIGH_ND_TABLE, out_dev, &rt->rt_gw6,
skb);
+ else
+ err = neigh_xmit(NEIGH_ARP_TABLE, out_dev, &rt->rt_gw4,
+ skb);
} else if (rt6) {
if (ipv6_addr_v4mapped(&rt6->rt6i_gateway)) {
/* 6PE (RFC 4798) */
diff --git a/net/ncsi/ncsi-cmd.c b/net/ncsi/ncsi-cmd.c
index 5c3fad8..0187e65 100644
--- a/net/ncsi/ncsi-cmd.c
+++ b/net/ncsi/ncsi-cmd.c
@@ -54,7 +54,7 @@ static void ncsi_cmd_build_header(struct ncsi_pkt_hdr *h,
checksum = ncsi_calculate_checksum((unsigned char *)h,
sizeof(*h) + nca->payload);
pchecksum = (__be32 *)((void *)h + sizeof(struct ncsi_pkt_hdr) +
- nca->payload);
+ ALIGN(nca->payload, 4));
*pchecksum = htonl(checksum);
}
@@ -309,14 +309,21 @@ static struct ncsi_request *ncsi_alloc_command(struct ncsi_cmd_arg *nca)
int ncsi_xmit_cmd(struct ncsi_cmd_arg *nca)
{
- struct ncsi_request *nr;
- struct ethhdr *eh;
struct ncsi_cmd_handler *nch = NULL;
+ struct ncsi_request *nr;
+ unsigned char type;
+ struct ethhdr *eh;
int i, ret;
+ /* Use OEM generic handler for Netlink request */
+ if (nca->req_flags == NCSI_REQ_FLAG_NETLINK_DRIVEN)
+ type = NCSI_PKT_CMD_OEM;
+ else
+ type = nca->type;
+
/* Search for the handler */
for (i = 0; i < ARRAY_SIZE(ncsi_cmd_handlers); i++) {
- if (ncsi_cmd_handlers[i].type == nca->type) {
+ if (ncsi_cmd_handlers[i].type == type) {
if (ncsi_cmd_handlers[i].handler)
nch = &ncsi_cmd_handlers[i];
else
diff --git a/net/ncsi/ncsi-pkt.h b/net/ncsi/ncsi-pkt.h
index a8e9def..80938b3 100644
--- a/net/ncsi/ncsi-pkt.h
+++ b/net/ncsi/ncsi-pkt.h
@@ -387,6 +387,9 @@ struct ncsi_aen_hncdsc_pkt {
#define NCSI_PKT_CMD_OEM 0x50 /* OEM */
#define NCSI_PKT_CMD_PLDM 0x51 /* PLDM request over NCSI over RBT */
#define NCSI_PKT_CMD_GPUUID 0x52 /* Get package UUID */
+#define NCSI_PKT_CMD_QPNPR 0x56 /* Query Pending NC PLDM request */
+#define NCSI_PKT_CMD_SNPR 0x57 /* Send NC PLDM Reply */
+
/* NCSI packet responses */
#define NCSI_PKT_RSP_CIS (NCSI_PKT_CMD_CIS + 0x80)
@@ -419,6 +422,8 @@ struct ncsi_aen_hncdsc_pkt {
#define NCSI_PKT_RSP_OEM (NCSI_PKT_CMD_OEM + 0x80)
#define NCSI_PKT_RSP_PLDM (NCSI_PKT_CMD_PLDM + 0x80)
#define NCSI_PKT_RSP_GPUUID (NCSI_PKT_CMD_GPUUID + 0x80)
+#define NCSI_PKT_RSP_QPNPR (NCSI_PKT_CMD_QPNPR + 0x80)
+#define NCSI_PKT_RSP_SNPR (NCSI_PKT_CMD_SNPR + 0x80)
/* NCSI response code/reason */
#define NCSI_PKT_RSP_C_COMPLETED 0x0000 /* Command Completed */
diff --git a/net/ncsi/ncsi-rsp.c b/net/ncsi/ncsi-rsp.c
index 7581bf9..d5611f0 100644
--- a/net/ncsi/ncsi-rsp.c
+++ b/net/ncsi/ncsi-rsp.c
@@ -47,7 +47,8 @@ static int ncsi_validate_rsp_pkt(struct ncsi_request *nr,
if (ntohs(h->code) != NCSI_PKT_RSP_C_COMPLETED ||
ntohs(h->reason) != NCSI_PKT_RSP_R_NO_ERROR) {
netdev_dbg(nr->ndp->ndev.dev,
- "NCSI: non zero response/reason code\n");
+ "NCSI: non zero response/reason code %04xh, %04xh\n",
+ ntohs(h->code), ntohs(h->reason));
return -EPERM;
}
@@ -55,7 +56,7 @@ static int ncsi_validate_rsp_pkt(struct ncsi_request *nr,
* sender doesn't support checksum according to NCSI
* specification.
*/
- pchecksum = (__be32 *)((void *)(h + 1) + payload - 4);
+ pchecksum = (__be32 *)((void *)(h + 1) + ALIGN(payload, 4) - 4);
if (ntohl(*pchecksum) == 0)
return 0;
@@ -63,7 +64,9 @@ static int ncsi_validate_rsp_pkt(struct ncsi_request *nr,
sizeof(*h) + payload - 4);
if (*pchecksum != htonl(checksum)) {
- netdev_dbg(nr->ndp->ndev.dev, "NCSI: checksum mismatched\n");
+ netdev_dbg(nr->ndp->ndev.dev,
+ "NCSI: checksum mismatched; recd: %08x calc: %08x\n",
+ *pchecksum, htonl(checksum));
return -EINVAL;
}
@@ -1035,6 +1038,11 @@ static int ncsi_rsp_handler_gpuuid(struct ncsi_request *nr)
return 0;
}
+static int ncsi_rsp_handler_pldm(struct ncsi_request *nr)
+{
+ return 0;
+}
+
static int ncsi_rsp_handler_netlink(struct ncsi_request *nr)
{
struct ncsi_dev_priv *ndp = nr->ndp;
@@ -1083,13 +1091,15 @@ static struct ncsi_rsp_handler {
{ NCSI_PKT_RSP_GVI, 40, ncsi_rsp_handler_gvi },
{ NCSI_PKT_RSP_GC, 32, ncsi_rsp_handler_gc },
{ NCSI_PKT_RSP_GP, -1, ncsi_rsp_handler_gp },
- { NCSI_PKT_RSP_GCPS, 172, ncsi_rsp_handler_gcps },
- { NCSI_PKT_RSP_GNS, 172, ncsi_rsp_handler_gns },
- { NCSI_PKT_RSP_GNPTS, 172, ncsi_rsp_handler_gnpts },
+ { NCSI_PKT_RSP_GCPS, 204, ncsi_rsp_handler_gcps },
+ { NCSI_PKT_RSP_GNS, 32, ncsi_rsp_handler_gns },
+ { NCSI_PKT_RSP_GNPTS, 48, ncsi_rsp_handler_gnpts },
{ NCSI_PKT_RSP_GPS, 8, ncsi_rsp_handler_gps },
{ NCSI_PKT_RSP_OEM, -1, ncsi_rsp_handler_oem },
- { NCSI_PKT_RSP_PLDM, 0, NULL },
- { NCSI_PKT_RSP_GPUUID, 20, ncsi_rsp_handler_gpuuid }
+ { NCSI_PKT_RSP_PLDM, -1, ncsi_rsp_handler_pldm },
+ { NCSI_PKT_RSP_GPUUID, 20, ncsi_rsp_handler_gpuuid },
+ { NCSI_PKT_RSP_QPNPR, -1, ncsi_rsp_handler_pldm },
+ { NCSI_PKT_RSP_SNPR, -1, ncsi_rsp_handler_pldm }
};
int ncsi_rcv_rsp(struct sk_buff *skb, struct net_device *dev,
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index 81a8ef4..0c63120 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -73,8 +73,7 @@ struct conntrack_gc_work {
};
static __read_mostly struct kmem_cache *nf_conntrack_cachep;
-static __read_mostly spinlock_t nf_conntrack_locks_all_lock;
-static __read_mostly DEFINE_SPINLOCK(nf_conntrack_locks_all_lock);
+static DEFINE_SPINLOCK(nf_conntrack_locks_all_lock);
static __read_mostly bool nf_conntrack_locks_all;
/* every gc cycle scans at most 1/GC_MAX_BUCKETS_DIV part of table */
diff --git a/net/netfilter/nf_conntrack_ftp.c b/net/netfilter/nf_conntrack_ftp.c
index c57d234..9eca904 100644
--- a/net/netfilter/nf_conntrack_ftp.c
+++ b/net/netfilter/nf_conntrack_ftp.c
@@ -322,7 +322,7 @@ static int find_pattern(const char *data, size_t dlen,
i++;
}
- pr_debug("Skipped up to `%c'!\n", skip);
+ pr_debug("Skipped up to 0x%hhx delimiter!\n", skip);
*numoff = i;
*numlen = getnum(data + i, dlen - i, cmd, term, numoff);
diff --git a/net/netfilter/nf_conntrack_labels.c b/net/netfilter/nf_conntrack_labels.c
index d1c6b2a..5227925 100644
--- a/net/netfilter/nf_conntrack_labels.c
+++ b/net/netfilter/nf_conntrack_labels.c
@@ -11,7 +11,7 @@
#include <net/netfilter/nf_conntrack_ecache.h>
#include <net/netfilter/nf_conntrack_labels.h>
-static __read_mostly DEFINE_SPINLOCK(nf_connlabels_lock);
+static DEFINE_SPINLOCK(nf_connlabels_lock);
static int replace_u32(u32 *address, u32 mask, u32 new)
{
diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c
index d97f4ea..88d4127 100644
--- a/net/netfilter/nf_conntrack_standalone.c
+++ b/net/netfilter/nf_conntrack_standalone.c
@@ -1035,9 +1035,14 @@ static int nf_conntrack_standalone_init_sysctl(struct net *net)
table[NF_SYSCTL_CT_COUNT].data = &net->ct.count;
table[NF_SYSCTL_CT_CHECKSUM].data = &net->ct.sysctl_checksum;
table[NF_SYSCTL_CT_LOG_INVALID].data = &net->ct.sysctl_log_invalid;
+ table[NF_SYSCTL_CT_ACCT].data = &net->ct.sysctl_acct;
+ table[NF_SYSCTL_CT_HELPER].data = &net->ct.sysctl_auto_assign_helper;
#ifdef CONFIG_NF_CONNTRACK_EVENTS
table[NF_SYSCTL_CT_EVENTS].data = &net->ct.sysctl_events;
#endif
+#ifdef CONFIG_NF_CONNTRACK_TIMESTAMP
+ table[NF_SYSCTL_CT_TIMESTAMP].data = &net->ct.sysctl_tstamp;
+#endif
table[NF_SYSCTL_CT_PROTO_TIMEOUT_GENERIC].data = &nf_generic_pernet(net)->timeout;
table[NF_SYSCTL_CT_PROTO_TIMEOUT_ICMP].data = &nf_icmp_pernet(net)->timeout;
table[NF_SYSCTL_CT_PROTO_TIMEOUT_ICMPV6].data = &nf_icmpv6_pernet(net)->timeout;
diff --git a/net/netfilter/nf_flow_table_ip.c b/net/netfilter/nf_flow_table_ip.c
index d68c801..b9e7dd6 100644
--- a/net/netfilter/nf_flow_table_ip.c
+++ b/net/netfilter/nf_flow_table_ip.c
@@ -228,7 +228,6 @@ static unsigned int nf_flow_xmit_xfrm(struct sk_buff *skb,
{
skb_orphan(skb);
skb_dst_set_noref(skb, dst);
- skb->tstamp = 0;
dst_output(state->net, state->sk, skb);
return NF_STOLEN;
}
@@ -284,6 +283,7 @@ nf_flow_offload_ip_hook(void *priv, struct sk_buff *skb,
flow->timeout = (u32)jiffies + NF_FLOW_TIMEOUT;
iph = ip_hdr(skb);
ip_decrease_ttl(iph);
+ skb->tstamp = 0;
if (unlikely(dst_xfrm(&rt->dst))) {
memset(skb->cb, 0, sizeof(struct inet_skb_parm));
@@ -512,6 +512,7 @@ nf_flow_offload_ipv6_hook(void *priv, struct sk_buff *skb,
flow->timeout = (u32)jiffies + NF_FLOW_TIMEOUT;
ip6h = ipv6_hdr(skb);
ip6h->hop_limit--;
+ skb->tstamp = 0;
if (unlikely(dst_xfrm(&rt->dst))) {
memset(skb->cb, 0, sizeof(struct inet6_skb_parm));
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index 6d00bef..013d288 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -5131,6 +5131,41 @@ nft_obj_type_get(struct net *net, u32 objtype)
return ERR_PTR(-ENOENT);
}
+static int nf_tables_updobj(const struct nft_ctx *ctx,
+ const struct nft_object_type *type,
+ const struct nlattr *attr,
+ struct nft_object *obj)
+{
+ struct nft_object *newobj;
+ struct nft_trans *trans;
+ int err;
+
+ if (!obj->ops->update)
+ return -EOPNOTSUPP;
+
+ trans = nft_trans_alloc(ctx, NFT_MSG_NEWOBJ,
+ sizeof(struct nft_trans_obj));
+ if (!trans)
+ return -ENOMEM;
+
+ newobj = nft_obj_init(ctx, type, attr);
+ if (IS_ERR(newobj)) {
+ err = PTR_ERR(newobj);
+ goto err1;
+ }
+
+ nft_trans_obj(trans) = obj;
+ nft_trans_obj_update(trans) = true;
+ nft_trans_obj_newobj(trans) = newobj;
+ list_add_tail(&trans->list, &ctx->net->nft.commit_list);
+
+ return 0;
+err1:
+ kfree(trans);
+ kfree(newobj);
+ return err;
+}
+
static int nf_tables_newobj(struct net *net, struct sock *nlsk,
struct sk_buff *skb, const struct nlmsghdr *nlh,
const struct nlattr * const nla[],
@@ -5170,7 +5205,13 @@ static int nf_tables_newobj(struct net *net, struct sock *nlsk,
NL_SET_BAD_ATTR(extack, nla[NFTA_OBJ_NAME]);
return -EEXIST;
}
- return 0;
+ if (nlh->nlmsg_flags & NLM_F_REPLACE)
+ return -EOPNOTSUPP;
+
+ type = nft_obj_type_get(net, objtype);
+ nft_ctx_init(&ctx, net, skb, nlh, family, table, NULL, nla);
+
+ return nf_tables_updobj(&ctx, type, nla[NFTA_OBJ_DATA], obj);
}
nft_ctx_init(&ctx, net, skb, nlh, family, table, NULL, nla);
@@ -6431,6 +6472,19 @@ static void nft_chain_commit_update(struct nft_trans *trans)
}
}
+static void nft_obj_commit_update(struct nft_trans *trans)
+{
+ struct nft_object *newobj;
+ struct nft_object *obj;
+
+ obj = nft_trans_obj(trans);
+ newobj = nft_trans_obj_newobj(trans);
+
+ obj->ops->update(obj, newobj);
+
+ kfree(newobj);
+}
+
static void nft_commit_release(struct nft_trans *trans)
{
switch (trans->msg_type) {
@@ -6795,10 +6849,18 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb)
te->set->ndeact--;
break;
case NFT_MSG_NEWOBJ:
- nft_clear(net, nft_trans_obj(trans));
- nf_tables_obj_notify(&trans->ctx, nft_trans_obj(trans),
- NFT_MSG_NEWOBJ);
- nft_trans_destroy(trans);
+ if (nft_trans_obj_update(trans)) {
+ nft_obj_commit_update(trans);
+ nf_tables_obj_notify(&trans->ctx,
+ nft_trans_obj(trans),
+ NFT_MSG_NEWOBJ);
+ } else {
+ nft_clear(net, nft_trans_obj(trans));
+ nf_tables_obj_notify(&trans->ctx,
+ nft_trans_obj(trans),
+ NFT_MSG_NEWOBJ);
+ nft_trans_destroy(trans);
+ }
break;
case NFT_MSG_DELOBJ:
nft_obj_del(nft_trans_obj(trans));
@@ -6945,8 +7007,13 @@ static int __nf_tables_abort(struct net *net)
nft_trans_destroy(trans);
break;
case NFT_MSG_NEWOBJ:
- trans->ctx.table->use--;
- nft_obj_del(nft_trans_obj(trans));
+ if (nft_trans_obj_update(trans)) {
+ kfree(nft_trans_obj_newobj(trans));
+ nft_trans_destroy(trans);
+ } else {
+ trans->ctx.table->use--;
+ nft_obj_del(nft_trans_obj(trans));
+ }
break;
case NFT_MSG_DELOBJ:
trans->ctx.table->use++;
diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c
index d69e186..0ba020c 100644
--- a/net/netfilter/nfnetlink_log.c
+++ b/net/netfilter/nfnetlink_log.c
@@ -385,6 +385,57 @@ nfulnl_timer(struct timer_list *t)
instance_put(inst);
}
+static u32 nfulnl_get_bridge_size(const struct sk_buff *skb)
+{
+ u32 size = 0;
+
+ if (!skb_mac_header_was_set(skb))
+ return 0;
+
+ if (skb_vlan_tag_present(skb)) {
+ size += nla_total_size(0); /* nested */
+ size += nla_total_size(sizeof(u16)); /* id */
+ size += nla_total_size(sizeof(u16)); /* tag */
+ }
+
+ if (skb->network_header > skb->mac_header)
+ size += nla_total_size(skb->network_header - skb->mac_header);
+
+ return size;
+}
+
+static int nfulnl_put_bridge(struct nfulnl_instance *inst, const struct sk_buff *skb)
+{
+ if (!skb_mac_header_was_set(skb))
+ return 0;
+
+ if (skb_vlan_tag_present(skb)) {
+ struct nlattr *nest;
+
+ nest = nla_nest_start(inst->skb, NFULA_VLAN);
+ if (!nest)
+ goto nla_put_failure;
+
+ if (nla_put_be16(inst->skb, NFULA_VLAN_TCI, htons(skb->vlan_tci)) ||
+ nla_put_be16(inst->skb, NFULA_VLAN_PROTO, skb->vlan_proto))
+ goto nla_put_failure;
+
+ nla_nest_end(inst->skb, nest);
+ }
+
+ if (skb->mac_header < skb->network_header) {
+ int len = (int)(skb->network_header - skb->mac_header);
+
+ if (nla_put(inst->skb, NFULA_L2HDR, len, skb_mac_header(skb)))
+ goto nla_put_failure;
+ }
+
+ return 0;
+
+nla_put_failure:
+ return -1;
+}
+
/* This is an inline function, we don't really care about a long
* list of arguments */
static inline int
@@ -580,6 +631,10 @@ __build_packet_message(struct nfnl_log_net *log,
NFULA_CT, NFULA_CT_INFO) < 0)
goto nla_put_failure;
+ if ((pf == NFPROTO_NETDEV || pf == NFPROTO_BRIDGE) &&
+ nfulnl_put_bridge(inst, skb) < 0)
+ goto nla_put_failure;
+
if (data_len) {
struct nlattr *nla;
int size = nla_attr_size(data_len);
@@ -687,6 +742,8 @@ nfulnl_log_packet(struct net *net,
size += nfnl_ct->build_size(ct);
}
}
+ if (pf == NFPROTO_NETDEV || pf == NFPROTO_BRIDGE)
+ size += nfulnl_get_bridge_size(skb);
qthreshold = inst->qthreshold;
/* per-rule qthreshold overrides per-instance */
diff --git a/net/netfilter/nft_byteorder.c b/net/netfilter/nft_byteorder.c
index e063184..12bed3f 100644
--- a/net/netfilter/nft_byteorder.c
+++ b/net/netfilter/nft_byteorder.c
@@ -43,14 +43,15 @@ void nft_byteorder_eval(const struct nft_expr *expr,
switch (priv->op) {
case NFT_BYTEORDER_NTOH:
for (i = 0; i < priv->len / 8; i++) {
- src64 = get_unaligned((u64 *)&src[i]);
- put_unaligned_be64(src64, &dst[i]);
+ src64 = nft_reg_load64(&src[i]);
+ nft_reg_store64(&dst[i], be64_to_cpu(src64));
}
break;
case NFT_BYTEORDER_HTON:
for (i = 0; i < priv->len / 8; i++) {
- src64 = get_unaligned_be64(&src[i]);
- put_unaligned(src64, (u64 *)&dst[i]);
+ src64 = (__force __u64)
+ cpu_to_be64(nft_reg_load64(&src[i]));
+ nft_reg_store64(&dst[i], src64);
}
break;
}
diff --git a/net/netfilter/nft_dynset.c b/net/netfilter/nft_dynset.c
index 33833a0..8887295 100644
--- a/net/netfilter/nft_dynset.c
+++ b/net/netfilter/nft_dynset.c
@@ -84,6 +84,11 @@ void nft_dynset_eval(const struct nft_expr *expr,
const struct nft_expr *sexpr;
u64 timeout;
+ if (priv->op == NFT_DYNSET_OP_DELETE) {
+ set->ops->delete(set, ®s->data[priv->sreg_key]);
+ return;
+ }
+
if (set->ops->update(set, ®s->data[priv->sreg_key], nft_dynset_new,
expr, regs, &ext)) {
sexpr = NULL;
@@ -161,6 +166,7 @@ static int nft_dynset_init(const struct nft_ctx *ctx,
priv->op = ntohl(nla_get_be32(tb[NFTA_DYNSET_OP]));
switch (priv->op) {
case NFT_DYNSET_OP_ADD:
+ case NFT_DYNSET_OP_DELETE:
break;
case NFT_DYNSET_OP_UPDATE:
if (!(set->flags & NFT_SET_TIMEOUT))
diff --git a/net/netfilter/nft_flow_offload.c b/net/netfilter/nft_flow_offload.c
index 060a4ed..01705ad 100644
--- a/net/netfilter/nft_flow_offload.c
+++ b/net/netfilter/nft_flow_offload.c
@@ -149,6 +149,11 @@ static int nft_flow_offload_validate(const struct nft_ctx *ctx,
return nft_chain_validate_hooks(ctx->chain, hook_mask);
}
+static const struct nla_policy nft_flow_offload_policy[NFTA_FLOW_MAX + 1] = {
+ [NFTA_FLOW_TABLE_NAME] = { .type = NLA_STRING,
+ .len = NFT_NAME_MAXLEN - 1 },
+};
+
static int nft_flow_offload_init(const struct nft_ctx *ctx,
const struct nft_expr *expr,
const struct nlattr * const tb[])
@@ -207,6 +212,7 @@ static const struct nft_expr_ops nft_flow_offload_ops = {
static struct nft_expr_type nft_flow_offload_type __read_mostly = {
.name = "flow_offload",
.ops = &nft_flow_offload_ops,
+ .policy = nft_flow_offload_policy,
.maxattr = NFTA_FLOW_MAX,
.owner = THIS_MODULE,
};
diff --git a/net/netfilter/nft_meta.c b/net/netfilter/nft_meta.c
index f69afb9..317e3a9 100644
--- a/net/netfilter/nft_meta.c
+++ b/net/netfilter/nft_meta.c
@@ -26,8 +26,36 @@
#include <uapi/linux/netfilter_bridge.h> /* NF_BR_PRE_ROUTING */
+#define NFT_META_SECS_PER_MINUTE 60
+#define NFT_META_SECS_PER_HOUR 3600
+#define NFT_META_SECS_PER_DAY 86400
+#define NFT_META_DAYS_PER_WEEK 7
+
static DEFINE_PER_CPU(struct rnd_state, nft_prandom_state);
+static u8 nft_meta_weekday(unsigned long secs)
+{
+ unsigned int dse;
+ u8 wday;
+
+ secs -= NFT_META_SECS_PER_MINUTE * sys_tz.tz_minuteswest;
+ dse = secs / NFT_META_SECS_PER_DAY;
+ wday = (4 + dse) % NFT_META_DAYS_PER_WEEK;
+
+ return wday;
+}
+
+static u32 nft_meta_hour(unsigned long secs)
+{
+ struct tm tm;
+
+ time64_to_tm(secs, 0, &tm);
+
+ return tm.tm_hour * NFT_META_SECS_PER_HOUR
+ + tm.tm_min * NFT_META_SECS_PER_MINUTE
+ + tm.tm_sec;
+}
+
void nft_meta_get_eval(const struct nft_expr *expr,
struct nft_regs *regs,
const struct nft_pktinfo *pkt)
@@ -218,6 +246,15 @@ void nft_meta_get_eval(const struct nft_expr *expr,
goto err;
strncpy((char *)dest, out->rtnl_link_ops->kind, IFNAMSIZ);
break;
+ case NFT_META_TIME_NS:
+ nft_reg_store64(dest, ktime_get_real_ns());
+ break;
+ case NFT_META_TIME_DAY:
+ nft_reg_store8(dest, nft_meta_weekday(get_seconds()));
+ break;
+ case NFT_META_TIME_HOUR:
+ *dest = nft_meta_hour(get_seconds());
+ break;
default:
WARN_ON(1);
goto err;
@@ -330,6 +367,15 @@ int nft_meta_get_init(const struct nft_ctx *ctx,
len = sizeof(u8);
break;
#endif
+ case NFT_META_TIME_NS:
+ len = sizeof(u64);
+ break;
+ case NFT_META_TIME_DAY:
+ len = sizeof(u8);
+ break;
+ case NFT_META_TIME_HOUR:
+ len = sizeof(u32);
+ break;
default:
return -EOPNOTSUPP;
}
diff --git a/net/netfilter/nft_quota.c b/net/netfilter/nft_quota.c
index c8745d4..4413690 100644
--- a/net/netfilter/nft_quota.c
+++ b/net/netfilter/nft_quota.c
@@ -13,7 +13,7 @@
#include <net/netfilter/nf_tables.h>
struct nft_quota {
- u64 quota;
+ atomic64_t quota;
unsigned long flags;
atomic64_t consumed;
};
@@ -21,7 +21,8 @@ struct nft_quota {
static inline bool nft_overquota(struct nft_quota *priv,
const struct sk_buff *skb)
{
- return atomic64_add_return(skb->len, &priv->consumed) >= priv->quota;
+ return atomic64_add_return(skb->len, &priv->consumed) >=
+ atomic64_read(&priv->quota);
}
static inline bool nft_quota_invert(struct nft_quota *priv)
@@ -89,7 +90,7 @@ static int nft_quota_do_init(const struct nlattr * const tb[],
return -EOPNOTSUPP;
}
- priv->quota = quota;
+ atomic64_set(&priv->quota, quota);
priv->flags = flags;
atomic64_set(&priv->consumed, consumed);
@@ -105,10 +106,22 @@ static int nft_quota_obj_init(const struct nft_ctx *ctx,
return nft_quota_do_init(tb, priv);
}
+static void nft_quota_obj_update(struct nft_object *obj,
+ struct nft_object *newobj)
+{
+ struct nft_quota *newpriv = nft_obj_data(newobj);
+ struct nft_quota *priv = nft_obj_data(obj);
+ u64 newquota;
+
+ newquota = atomic64_read(&newpriv->quota);
+ atomic64_set(&priv->quota, newquota);
+ priv->flags = newpriv->flags;
+}
+
static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv,
bool reset)
{
- u64 consumed, consumed_cap;
+ u64 consumed, consumed_cap, quota;
u32 flags = priv->flags;
/* Since we inconditionally increment consumed quota for each packet
@@ -116,14 +129,15 @@ static int nft_quota_do_dump(struct sk_buff *skb, struct nft_quota *priv,
* userspace.
*/
consumed = atomic64_read(&priv->consumed);
- if (consumed >= priv->quota) {
- consumed_cap = priv->quota;
+ quota = atomic64_read(&priv->quota);
+ if (consumed >= quota) {
+ consumed_cap = quota;
flags |= NFT_QUOTA_F_DEPLETED;
} else {
consumed_cap = consumed;
}
- if (nla_put_be64(skb, NFTA_QUOTA_BYTES, cpu_to_be64(priv->quota),
+ if (nla_put_be64(skb, NFTA_QUOTA_BYTES, cpu_to_be64(quota),
NFTA_QUOTA_PAD) ||
nla_put_be64(skb, NFTA_QUOTA_CONSUMED, cpu_to_be64(consumed_cap),
NFTA_QUOTA_PAD) ||
@@ -155,6 +169,7 @@ static const struct nft_object_ops nft_quota_obj_ops = {
.init = nft_quota_obj_init,
.eval = nft_quota_obj_eval,
.dump = nft_quota_obj_dump,
+ .update = nft_quota_obj_update,
};
static struct nft_object_type nft_quota_obj_type __read_mostly = {
diff --git a/net/netfilter/nft_set_hash.c b/net/netfilter/nft_set_hash.c
index c490451..b331a3c 100644
--- a/net/netfilter/nft_set_hash.c
+++ b/net/netfilter/nft_set_hash.c
@@ -234,6 +234,24 @@ static void nft_rhash_remove(const struct net *net,
rhashtable_remove_fast(&priv->ht, &he->node, nft_rhash_params);
}
+static bool nft_rhash_delete(const struct nft_set *set,
+ const u32 *key)
+{
+ struct nft_rhash *priv = nft_set_priv(set);
+ struct nft_rhash_cmp_arg arg = {
+ .genmask = NFT_GENMASK_ANY,
+ .set = set,
+ .key = key,
+ };
+ struct nft_rhash_elem *he;
+
+ he = rhashtable_lookup(&priv->ht, &arg, nft_rhash_params);
+ if (he == NULL)
+ return false;
+
+ return rhashtable_remove_fast(&priv->ht, &he->node, nft_rhash_params) == 0;
+}
+
static void nft_rhash_walk(const struct nft_ctx *ctx, struct nft_set *set,
struct nft_set_iter *iter)
{
@@ -662,6 +680,7 @@ struct nft_set_type nft_set_rhash_type __read_mostly = {
.remove = nft_rhash_remove,
.lookup = nft_rhash_lookup,
.update = nft_rhash_update,
+ .delete = nft_rhash_delete,
.walk = nft_rhash_walk,
.get = nft_rhash_get,
},
diff --git a/net/netfilter/xt_nfacct.c b/net/netfilter/xt_nfacct.c
index d0ab1ad..5aab6df 100644
--- a/net/netfilter/xt_nfacct.c
+++ b/net/netfilter/xt_nfacct.c
@@ -54,25 +54,39 @@ nfacct_mt_destroy(const struct xt_mtdtor_param *par)
nfnl_acct_put(info->nfacct);
}
-static struct xt_match nfacct_mt_reg __read_mostly = {
- .name = "nfacct",
- .family = NFPROTO_UNSPEC,
- .checkentry = nfacct_mt_checkentry,
- .match = nfacct_mt,
- .destroy = nfacct_mt_destroy,
- .matchsize = sizeof(struct xt_nfacct_match_info),
- .usersize = offsetof(struct xt_nfacct_match_info, nfacct),
- .me = THIS_MODULE,
+static struct xt_match nfacct_mt_reg[] __read_mostly = {
+ {
+ .name = "nfacct",
+ .revision = 0,
+ .family = NFPROTO_UNSPEC,
+ .checkentry = nfacct_mt_checkentry,
+ .match = nfacct_mt,
+ .destroy = nfacct_mt_destroy,
+ .matchsize = sizeof(struct xt_nfacct_match_info),
+ .usersize = offsetof(struct xt_nfacct_match_info, nfacct),
+ .me = THIS_MODULE,
+ },
+ {
+ .name = "nfacct",
+ .revision = 1,
+ .family = NFPROTO_UNSPEC,
+ .checkentry = nfacct_mt_checkentry,
+ .match = nfacct_mt,
+ .destroy = nfacct_mt_destroy,
+ .matchsize = sizeof(struct xt_nfacct_match_info_v1),
+ .usersize = offsetof(struct xt_nfacct_match_info_v1, nfacct),
+ .me = THIS_MODULE,
+ },
};
static int __init nfacct_mt_init(void)
{
- return xt_register_match(&nfacct_mt_reg);
+ return xt_register_matches(nfacct_mt_reg, ARRAY_SIZE(nfacct_mt_reg));
}
static void __exit nfacct_mt_exit(void)
{
- xt_unregister_match(&nfacct_mt_reg);
+ xt_unregister_matches(nfacct_mt_reg, ARRAY_SIZE(nfacct_mt_reg));
}
module_init(nfacct_mt_init);
diff --git a/net/netfilter/xt_physdev.c b/net/netfilter/xt_physdev.c
index ead7c60..b92b22c 100644
--- a/net/netfilter/xt_physdev.c
+++ b/net/netfilter/xt_physdev.c
@@ -101,11 +101,9 @@ static int physdev_mt_check(const struct xt_mtchk_param *par)
if (info->bitmask & (XT_PHYSDEV_OP_OUT | XT_PHYSDEV_OP_ISOUT) &&
(!(info->bitmask & XT_PHYSDEV_OP_BRIDGED) ||
info->invert & XT_PHYSDEV_OP_BRIDGED) &&
- par->hook_mask & ((1 << NF_INET_LOCAL_OUT) |
- (1 << NF_INET_FORWARD) | (1 << NF_INET_POST_ROUTING))) {
+ par->hook_mask & (1 << NF_INET_LOCAL_OUT)) {
pr_info_ratelimited("--physdev-out and --physdev-is-out only supported in the FORWARD and POSTROUTING chains with bridged traffic\n");
- if (par->hook_mask & (1 << NF_INET_LOCAL_OUT))
- return -EINVAL;
+ return -EINVAL;
}
if (!brnf_probed) {
diff --git a/net/netlabel/netlabel_kapi.c b/net/netlabel/netlabel_kapi.c
index 2b0ef55..409a3ae 100644
--- a/net/netlabel/netlabel_kapi.c
+++ b/net/netlabel/netlabel_kapi.c
@@ -607,7 +607,7 @@ static struct netlbl_lsm_catmap *_netlbl_catmap_getnode(
*/
int netlbl_catmap_walk(struct netlbl_lsm_catmap *catmap, u32 offset)
{
- struct netlbl_lsm_catmap *iter = catmap;
+ struct netlbl_lsm_catmap *iter;
u32 idx;
u32 bit;
NETLBL_CATMAP_MAPTYPE bitmap;
diff --git a/net/openvswitch/conntrack.c b/net/openvswitch/conntrack.c
index 848c6eb..05249eb 100644
--- a/net/openvswitch/conntrack.c
+++ b/net/openvswitch/conntrack.c
@@ -67,6 +67,7 @@ struct ovs_conntrack_info {
struct md_mark mark;
struct md_labels labels;
char timeout[CTNL_TIMEOUT_NAME_MAX];
+ struct nf_ct_timeout *nf_ct_timeout;
#if IS_ENABLED(CONFIG_NF_NAT)
struct nf_nat_range2 range; /* Only present for SRC NAT and DST NAT. */
#endif
@@ -524,6 +525,11 @@ static int handle_fragments(struct net *net, struct sw_flow_key *key,
return -EPFNOSUPPORT;
}
+ /* The key extracted from the fragment that completed this datagram
+ * likely didn't have an L4 header, so regenerate it.
+ */
+ ovs_flow_key_update_l3l4(skb, key);
+
key->ip.frag = OVS_FRAG_TYPE_NONE;
skb_clear_hash(skb);
skb->ignore_df = 1;
@@ -697,6 +703,14 @@ static bool skb_nfct_cached(struct net *net,
if (help && rcu_access_pointer(help->helper) != info->helper)
return false;
}
+ if (info->nf_ct_timeout) {
+ struct nf_conn_timeout *timeout_ext;
+
+ timeout_ext = nf_ct_timeout_find(ct);
+ if (!timeout_ext || info->nf_ct_timeout !=
+ rcu_dereference(timeout_ext->timeout))
+ return false;
+ }
/* Force conntrack entry direction to the current packet? */
if (info->force && CTINFO2DIR(ctinfo) != IP_CT_DIR_ORIGINAL) {
/* Delete the conntrack entry if confirmed, else just release
@@ -1565,7 +1579,7 @@ static int parse_ct(const struct nlattr *attr, struct ovs_conntrack_info *info,
case OVS_CT_ATTR_TIMEOUT:
memcpy(info->timeout, nla_data(a), nla_len(a));
if (!memchr(info->timeout, '\0', nla_len(a))) {
- OVS_NLERR(log, "Invalid conntrack helper");
+ OVS_NLERR(log, "Invalid conntrack timeout");
return -EINVAL;
}
break;
@@ -1657,6 +1671,10 @@ int ovs_ct_copy_action(struct net *net, const struct nlattr *attr,
ct_info.timeout))
pr_info_ratelimited("Failed to associated timeout "
"policy `%s'\n", ct_info.timeout);
+ else
+ ct_info.nf_ct_timeout = rcu_dereference(
+ nf_ct_timeout_find(ct_info.ct)->timeout);
+
}
if (helper) {
diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c
index 65122bb..dde9d76 100644
--- a/net/openvswitch/datapath.c
+++ b/net/openvswitch/datapath.c
@@ -1545,10 +1545,34 @@ static void ovs_dp_reset_user_features(struct sk_buff *skb, struct genl_info *in
dp->user_features = 0;
}
-static void ovs_dp_change(struct datapath *dp, struct nlattr *a[])
+DEFINE_STATIC_KEY_FALSE(tc_recirc_sharing_support);
+
+static int ovs_dp_change(struct datapath *dp, struct nlattr *a[])
{
- if (a[OVS_DP_ATTR_USER_FEATURES])
- dp->user_features = nla_get_u32(a[OVS_DP_ATTR_USER_FEATURES]);
+ u32 user_features = 0;
+
+ if (a[OVS_DP_ATTR_USER_FEATURES]) {
+ user_features = nla_get_u32(a[OVS_DP_ATTR_USER_FEATURES]);
+
+ if (user_features & ~(OVS_DP_F_VPORT_PIDS |
+ OVS_DP_F_UNALIGNED |
+ OVS_DP_F_TC_RECIRC_SHARING))
+ return -EOPNOTSUPP;
+
+#if !IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ if (user_features & OVS_DP_F_TC_RECIRC_SHARING)
+ return -EOPNOTSUPP;
+#endif
+ }
+
+ dp->user_features = user_features;
+
+ if (dp->user_features & OVS_DP_F_TC_RECIRC_SHARING)
+ static_branch_enable(&tc_recirc_sharing_support);
+ else
+ static_branch_disable(&tc_recirc_sharing_support);
+
+ return 0;
}
static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info)
@@ -1610,7 +1634,9 @@ static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info)
parms.port_no = OVSP_LOCAL;
parms.upcall_portids = a[OVS_DP_ATTR_UPCALL_PID];
- ovs_dp_change(dp, a);
+ err = ovs_dp_change(dp, a);
+ if (err)
+ goto err_destroy_meters;
/* So far only local changes have been made, now need the lock. */
ovs_lock();
@@ -1736,7 +1762,9 @@ static int ovs_dp_cmd_set(struct sk_buff *skb, struct genl_info *info)
if (IS_ERR(dp))
goto err_unlock_free;
- ovs_dp_change(dp, info->attrs);
+ err = ovs_dp_change(dp, info->attrs);
+ if (err)
+ goto err_unlock_free;
err = ovs_dp_cmd_fill_info(dp, reply, info->snd_portid,
info->snd_seq, 0, OVS_DP_CMD_SET);
diff --git a/net/openvswitch/datapath.h b/net/openvswitch/datapath.h
index 751d34a..81e85dd 100644
--- a/net/openvswitch/datapath.h
+++ b/net/openvswitch/datapath.h
@@ -218,6 +218,8 @@ static inline struct datapath *get_dp(struct net *net, int dp_ifindex)
extern struct notifier_block ovs_dp_device_notifier;
extern struct genl_family dp_vport_genl_family;
+DECLARE_STATIC_KEY_FALSE(tc_recirc_sharing_support);
+
void ovs_dp_process_packet(struct sk_buff *skb, struct sw_flow_key *key);
void ovs_dp_detach_port(struct vport *);
int ovs_dp_upcall(struct datapath *, struct sk_buff *,
diff --git a/net/openvswitch/flow.c b/net/openvswitch/flow.c
index bc89e16..38147e6 100644
--- a/net/openvswitch/flow.c
+++ b/net/openvswitch/flow.c
@@ -523,78 +523,15 @@ static int parse_nsh(struct sk_buff *skb, struct sw_flow_key *key)
}
/**
- * key_extract - extracts a flow key from an Ethernet frame.
+ * key_extract_l3l4 - extracts L3/L4 header information.
* @skb: sk_buff that contains the frame, with skb->data pointing to the
- * Ethernet header
+ * L3 header
* @key: output flow key
*
- * The caller must ensure that skb->len >= ETH_HLEN.
- *
- * Returns 0 if successful, otherwise a negative errno value.
- *
- * Initializes @skb header fields as follows:
- *
- * - skb->mac_header: the L2 header.
- *
- * - skb->network_header: just past the L2 header, or just past the
- * VLAN header, to the first byte of the L2 payload.
- *
- * - skb->transport_header: If key->eth.type is ETH_P_IP or ETH_P_IPV6
- * on output, then just past the IP header, if one is present and
- * of a correct length, otherwise the same as skb->network_header.
- * For other key->eth.type values it is left untouched.
- *
- * - skb->protocol: the type of the data starting at skb->network_header.
- * Equals to key->eth.type.
*/
-static int key_extract(struct sk_buff *skb, struct sw_flow_key *key)
+static int key_extract_l3l4(struct sk_buff *skb, struct sw_flow_key *key)
{
int error;
- struct ethhdr *eth;
-
- /* Flags are always used as part of stats */
- key->tp.flags = 0;
-
- skb_reset_mac_header(skb);
-
- /* Link layer. */
- clear_vlan(key);
- if (ovs_key_mac_proto(key) == MAC_PROTO_NONE) {
- if (unlikely(eth_type_vlan(skb->protocol)))
- return -EINVAL;
-
- skb_reset_network_header(skb);
- key->eth.type = skb->protocol;
- } else {
- eth = eth_hdr(skb);
- ether_addr_copy(key->eth.src, eth->h_source);
- ether_addr_copy(key->eth.dst, eth->h_dest);
-
- __skb_pull(skb, 2 * ETH_ALEN);
- /* We are going to push all headers that we pull, so no need to
- * update skb->csum here.
- */
-
- if (unlikely(parse_vlan(skb, key)))
- return -ENOMEM;
-
- key->eth.type = parse_ethertype(skb);
- if (unlikely(key->eth.type == htons(0)))
- return -ENOMEM;
-
- /* Multiple tagged packets need to retain TPID to satisfy
- * skb_vlan_pop(), which will later shift the ethertype into
- * skb->protocol.
- */
- if (key->eth.cvlan.tci & htons(VLAN_CFI_MASK))
- skb->protocol = key->eth.cvlan.tpid;
- else
- skb->protocol = key->eth.type;
-
- skb_reset_network_header(skb);
- __skb_push(skb, skb->data - skb_mac_header(skb));
- }
- skb_reset_mac_len(skb);
/* Network layer. */
if (key->eth.type == htons(ETH_P_IP)) {
@@ -623,6 +560,7 @@ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key)
offset = nh->frag_off & htons(IP_OFFSET);
if (offset) {
key->ip.frag = OVS_FRAG_TYPE_LATER;
+ memset(&key->tp, 0, sizeof(key->tp));
return 0;
}
if (nh->frag_off & htons(IP_MF) ||
@@ -740,8 +678,10 @@ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key)
return error;
}
- if (key->ip.frag == OVS_FRAG_TYPE_LATER)
+ if (key->ip.frag == OVS_FRAG_TYPE_LATER) {
+ memset(&key->tp, 0, sizeof(key->tp));
return 0;
+ }
if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP)
key->ip.frag = OVS_FRAG_TYPE_FIRST;
@@ -788,6 +728,92 @@ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key)
return 0;
}
+/**
+ * key_extract - extracts a flow key from an Ethernet frame.
+ * @skb: sk_buff that contains the frame, with skb->data pointing to the
+ * Ethernet header
+ * @key: output flow key
+ *
+ * The caller must ensure that skb->len >= ETH_HLEN.
+ *
+ * Returns 0 if successful, otherwise a negative errno value.
+ *
+ * Initializes @skb header fields as follows:
+ *
+ * - skb->mac_header: the L2 header.
+ *
+ * - skb->network_header: just past the L2 header, or just past the
+ * VLAN header, to the first byte of the L2 payload.
+ *
+ * - skb->transport_header: If key->eth.type is ETH_P_IP or ETH_P_IPV6
+ * on output, then just past the IP header, if one is present and
+ * of a correct length, otherwise the same as skb->network_header.
+ * For other key->eth.type values it is left untouched.
+ *
+ * - skb->protocol: the type of the data starting at skb->network_header.
+ * Equals to key->eth.type.
+ */
+static int key_extract(struct sk_buff *skb, struct sw_flow_key *key)
+{
+ struct ethhdr *eth;
+
+ /* Flags are always used as part of stats */
+ key->tp.flags = 0;
+
+ skb_reset_mac_header(skb);
+
+ /* Link layer. */
+ clear_vlan(key);
+ if (ovs_key_mac_proto(key) == MAC_PROTO_NONE) {
+ if (unlikely(eth_type_vlan(skb->protocol)))
+ return -EINVAL;
+
+ skb_reset_network_header(skb);
+ key->eth.type = skb->protocol;
+ } else {
+ eth = eth_hdr(skb);
+ ether_addr_copy(key->eth.src, eth->h_source);
+ ether_addr_copy(key->eth.dst, eth->h_dest);
+
+ __skb_pull(skb, 2 * ETH_ALEN);
+ /* We are going to push all headers that we pull, so no need to
+ * update skb->csum here.
+ */
+
+ if (unlikely(parse_vlan(skb, key)))
+ return -ENOMEM;
+
+ key->eth.type = parse_ethertype(skb);
+ if (unlikely(key->eth.type == htons(0)))
+ return -ENOMEM;
+
+ /* Multiple tagged packets need to retain TPID to satisfy
+ * skb_vlan_pop(), which will later shift the ethertype into
+ * skb->protocol.
+ */
+ if (key->eth.cvlan.tci & htons(VLAN_CFI_MASK))
+ skb->protocol = key->eth.cvlan.tpid;
+ else
+ skb->protocol = key->eth.type;
+
+ skb_reset_network_header(skb);
+ __skb_push(skb, skb->data - skb_mac_header(skb));
+ }
+
+ skb_reset_mac_len(skb);
+
+ /* Fill out L3/L4 key info, if any */
+ return key_extract_l3l4(skb, key);
+}
+
+/* In the case of conntrack fragment handling it expects L3 headers,
+ * add a helper.
+ */
+int ovs_flow_key_update_l3l4(struct sk_buff *skb, struct sw_flow_key *key)
+{
+ return key_extract_l3l4(skb, key);
+}
+
int ovs_flow_key_update(struct sk_buff *skb, struct sw_flow_key *key)
{
int res;
@@ -816,6 +842,9 @@ static int key_extract_mac_proto(struct sk_buff *skb)
int ovs_flow_key_extract(const struct ip_tunnel_info *tun_info,
struct sk_buff *skb, struct sw_flow_key *key)
{
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ struct tc_skb_ext *tc_ext;
+#endif
int res, err;
/* Extract metadata from packet. */
@@ -848,7 +877,17 @@ int ovs_flow_key_extract(const struct ip_tunnel_info *tun_info,
if (res < 0)
return res;
key->mac_proto = res;
+
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ if (static_branch_unlikely(&tc_recirc_sharing_support)) {
+ tc_ext = skb_ext_find(skb, TC_SKB_EXT);
+ key->recirc_id = tc_ext ? tc_ext->chain : 0;
+ } else {
+ key->recirc_id = 0;
+ }
+#else
key->recirc_id = 0;
+#endif
err = key_extract(skb, key);
if (!err)
diff --git a/net/openvswitch/flow.h b/net/openvswitch/flow.h
index a5506e2..b830d5f 100644
--- a/net/openvswitch/flow.h
+++ b/net/openvswitch/flow.h
@@ -270,6 +270,7 @@ void ovs_flow_stats_clear(struct sw_flow *);
u64 ovs_flow_used_time(unsigned long flow_jiffies);
int ovs_flow_key_update(struct sk_buff *skb, struct sw_flow_key *key);
+int ovs_flow_key_update_l3l4(struct sk_buff *skb, struct sw_flow_key *key);
int ovs_flow_key_extract(const struct ip_tunnel_info *tun_info,
struct sk_buff *skb,
struct sw_flow_key *key);
diff --git a/net/psample/psample.c b/net/psample/psample.c
index 841f198..66e4b61 100644
--- a/net/psample/psample.c
+++ b/net/psample/psample.c
@@ -154,7 +154,7 @@ static void psample_group_destroy(struct psample_group *group)
{
psample_group_notify(group, PSAMPLE_CMD_DEL_GROUP);
list_del(&group->list);
- kfree(group);
+ kfree_rcu(group, rcu);
}
static struct psample_group *
diff --git a/net/rds/af_rds.c b/net/rds/af_rds.c
index 2977137..1a5bf3f 100644
--- a/net/rds/af_rds.c
+++ b/net/rds/af_rds.c
@@ -559,7 +559,7 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr,
ret = -EDESTADDRREQ;
break;
}
- if (IN_MULTICAST(ntohl(sin->sin_addr.s_addr)) ||
+ if (ipv4_is_multicast(sin->sin_addr.s_addr) ||
sin->sin_addr.s_addr == htonl(INADDR_BROADCAST)) {
ret = -EINVAL;
break;
@@ -593,7 +593,7 @@ static int rds_connect(struct socket *sock, struct sockaddr *uaddr,
addr4 = sin6->sin6_addr.s6_addr32[3];
if (addr4 == htonl(INADDR_ANY) ||
addr4 == htonl(INADDR_BROADCAST) ||
- IN_MULTICAST(ntohl(addr4))) {
+ ipv4_is_multicast(addr4)) {
ret = -EPROTOTYPE;
break;
}
diff --git a/net/rds/bind.c b/net/rds/bind.c
index 0f4398e..6dbb763 100644
--- a/net/rds/bind.c
+++ b/net/rds/bind.c
@@ -181,7 +181,7 @@ int rds_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
if (addr_len < sizeof(struct sockaddr_in) ||
sin->sin_addr.s_addr == htonl(INADDR_ANY) ||
sin->sin_addr.s_addr == htonl(INADDR_BROADCAST) ||
- IN_MULTICAST(ntohl(sin->sin_addr.s_addr)))
+ ipv4_is_multicast(sin->sin_addr.s_addr))
return -EINVAL;
ipv6_addr_set_v4mapped(sin->sin_addr.s_addr, &v6addr);
binding_addr = &v6addr;
@@ -206,7 +206,7 @@ int rds_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len)
addr4 = sin6->sin6_addr.s6_addr32[3];
if (addr4 == htonl(INADDR_ANY) ||
addr4 == htonl(INADDR_BROADCAST) ||
- IN_MULTICAST(ntohl(addr4)))
+ ipv4_is_multicast(addr4))
return -EINVAL;
}
/* The scope ID must be specified for link local address. */
diff --git a/net/rds/ib.c b/net/rds/ib.c
index ec05d91..45acab2 100644
--- a/net/rds/ib.c
+++ b/net/rds/ib.c
@@ -291,7 +291,7 @@ static int rds_ib_conn_info_visitor(struct rds_connection *conn,
void *buffer)
{
struct rds_info_rdma_connection *iinfo = buffer;
- struct rds_ib_connection *ic;
+ struct rds_ib_connection *ic = conn->c_transport_data;
/* We will only ever look at IB transports */
if (conn->c_trans != &rds_ib_transport)
@@ -301,15 +301,16 @@ static int rds_ib_conn_info_visitor(struct rds_connection *conn,
iinfo->src_addr = conn->c_laddr.s6_addr32[3];
iinfo->dst_addr = conn->c_faddr.s6_addr32[3];
- iinfo->tos = conn->c_tos;
+ if (ic) {
+ iinfo->tos = conn->c_tos;
+ iinfo->sl = ic->i_sl;
+ }
memset(&iinfo->src_gid, 0, sizeof(iinfo->src_gid));
memset(&iinfo->dst_gid, 0, sizeof(iinfo->dst_gid));
if (rds_conn_state(conn) == RDS_CONN_UP) {
struct rds_ib_device *rds_ibdev;
- ic = conn->c_transport_data;
-
rdma_read_gids(ic->i_cm_id, (union ib_gid *)&iinfo->src_gid,
(union ib_gid *)&iinfo->dst_gid);
@@ -329,7 +330,7 @@ static int rds6_ib_conn_info_visitor(struct rds_connection *conn,
void *buffer)
{
struct rds6_info_rdma_connection *iinfo6 = buffer;
- struct rds_ib_connection *ic;
+ struct rds_ib_connection *ic = conn->c_transport_data;
/* We will only ever look at IB transports */
if (conn->c_trans != &rds_ib_transport)
@@ -337,6 +338,10 @@ static int rds6_ib_conn_info_visitor(struct rds_connection *conn,
iinfo6->src_addr = conn->c_laddr;
iinfo6->dst_addr = conn->c_faddr;
+ if (ic) {
+ iinfo6->tos = conn->c_tos;
+ iinfo6->sl = ic->i_sl;
+ }
memset(&iinfo6->src_gid, 0, sizeof(iinfo6->src_gid));
memset(&iinfo6->dst_gid, 0, sizeof(iinfo6->dst_gid));
@@ -344,7 +349,6 @@ static int rds6_ib_conn_info_visitor(struct rds_connection *conn,
if (rds_conn_state(conn) == RDS_CONN_UP) {
struct rds_ib_device *rds_ibdev;
- ic = conn->c_transport_data;
rdma_read_gids(ic->i_cm_id, (union ib_gid *)&iinfo6->src_gid,
(union ib_gid *)&iinfo6->dst_gid);
rds_ibdev = ic->rds_ibdev;
diff --git a/net/rds/ib.h b/net/rds/ib.h
index 303c6ee..f2b558e 100644
--- a/net/rds/ib.h
+++ b/net/rds/ib.h
@@ -220,6 +220,7 @@ struct rds_ib_connection {
/* Send/Recv vectors */
int i_scq_vector;
int i_rcq_vector;
+ u8 i_sl;
};
/* This assumes that atomic_t is at least 32 bits */
diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index fddaa09..233f136 100644
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -152,6 +152,9 @@ void rds_ib_cm_connect_complete(struct rds_connection *conn, struct rdma_cm_even
RDS_PROTOCOL_MINOR(conn->c_version),
ic->i_flowctl ? ", flow control" : "");
+ /* receive sl from the peer */
+ ic->i_sl = ic->i_cm_id->route.path_rec->sl;
+
atomic_set(&ic->i_cq_quiesce, 0);
/* Init rings and fill recv. this needs to wait until protocol
diff --git a/net/rds/ib_recv.c b/net/rds/ib_recv.c
index 1a8a4a7..a0f99bb 100644
--- a/net/rds/ib_recv.c
+++ b/net/rds/ib_recv.c
@@ -1048,9 +1048,14 @@ int rds_ib_recv_init(void)
si_meminfo(&si);
rds_ib_sysctl_max_recv_allocation = si.totalram / 3 * PAGE_SIZE / RDS_FRAG_SIZE;
- rds_ib_incoming_slab = kmem_cache_create("rds_ib_incoming",
- sizeof(struct rds_ib_incoming),
- 0, SLAB_HWCACHE_ALIGN, NULL);
+ rds_ib_incoming_slab =
+ kmem_cache_create_usercopy("rds_ib_incoming",
+ sizeof(struct rds_ib_incoming),
+ 0, SLAB_HWCACHE_ALIGN,
+ offsetof(struct rds_ib_incoming,
+ ii_inc.i_usercopy),
+ sizeof(struct rds_inc_usercopy),
+ NULL);
if (!rds_ib_incoming_slab)
goto out;
diff --git a/net/rds/rdma_transport.c b/net/rds/rdma_transport.c
index 9986d60..5f741e5 100644
--- a/net/rds/rdma_transport.c
+++ b/net/rds/rdma_transport.c
@@ -43,6 +43,9 @@ static struct rdma_cm_id *rds_rdma_listen_id;
static struct rdma_cm_id *rds6_rdma_listen_id;
#endif
+/* Per IB specification 7.7.3, service level is a 4-bit field. */
+#define TOS_TO_SL(tos) ((tos) & 0xF)
+
static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id,
struct rdma_cm_event *event,
bool isv6)
@@ -97,10 +100,13 @@ static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id,
struct rds_ib_connection *ibic;
ibic = conn->c_transport_data;
- if (ibic && ibic->i_cm_id == cm_id)
+ if (ibic && ibic->i_cm_id == cm_id) {
+ cm_id->route.path_rec[0].sl =
+ TOS_TO_SL(conn->c_tos);
ret = trans->cm_initiate_connect(cm_id, isv6);
- else
+ } else {
rds_conn_drop(conn);
+ }
}
break;
diff --git a/net/rds/rds.h b/net/rds/rds.h
index ad605fd..53e8691 100644
--- a/net/rds/rds.h
+++ b/net/rds/rds.h
@@ -271,6 +271,12 @@ struct rds_ext_header_rdma_dest {
#define RDS_MSG_RX_END 2
#define RDS_MSG_RX_CMSG 3
+/* The following values are whitelisted for usercopy */
+struct rds_inc_usercopy {
+ rds_rdma_cookie_t rdma_cookie;
+ ktime_t rx_tstamp;
+};
+
struct rds_incoming {
refcount_t i_refcount;
struct list_head i_item;
@@ -280,8 +286,7 @@ struct rds_incoming {
unsigned long i_rx_jiffies;
struct in6_addr i_saddr;
- rds_rdma_cookie_t i_rdma_cookie;
- ktime_t i_rx_tstamp;
+ struct rds_inc_usercopy i_usercopy;
u64 i_rx_lat_trace[RDS_RX_MAX_TRACES];
};
diff --git a/net/rds/recv.c b/net/rds/recv.c
index 853de48..c840497 100644
--- a/net/rds/recv.c
+++ b/net/rds/recv.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2006, 2018 Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2006, 2019 Oracle and/or its affiliates. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -47,8 +47,8 @@ void rds_inc_init(struct rds_incoming *inc, struct rds_connection *conn,
INIT_LIST_HEAD(&inc->i_item);
inc->i_conn = conn;
inc->i_saddr = *saddr;
- inc->i_rdma_cookie = 0;
- inc->i_rx_tstamp = ktime_set(0, 0);
+ inc->i_usercopy.rdma_cookie = 0;
+ inc->i_usercopy.rx_tstamp = ktime_set(0, 0);
memset(inc->i_rx_lat_trace, 0, sizeof(inc->i_rx_lat_trace));
}
@@ -62,8 +62,8 @@ void rds_inc_path_init(struct rds_incoming *inc, struct rds_conn_path *cp,
inc->i_conn = cp->cp_conn;
inc->i_conn_path = cp;
inc->i_saddr = *saddr;
- inc->i_rdma_cookie = 0;
- inc->i_rx_tstamp = ktime_set(0, 0);
+ inc->i_usercopy.rdma_cookie = 0;
+ inc->i_usercopy.rx_tstamp = ktime_set(0, 0);
}
EXPORT_SYMBOL_GPL(rds_inc_path_init);
@@ -186,7 +186,7 @@ static void rds_recv_incoming_exthdrs(struct rds_incoming *inc, struct rds_sock
case RDS_EXTHDR_RDMA_DEST:
/* We ignore the size for now. We could stash it
* somewhere and use it for error checking. */
- inc->i_rdma_cookie = rds_rdma_make_cookie(
+ inc->i_usercopy.rdma_cookie = rds_rdma_make_cookie(
be32_to_cpu(buffer.rdma_dest.h_rdma_rkey),
be32_to_cpu(buffer.rdma_dest.h_rdma_offset));
@@ -380,7 +380,7 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
be32_to_cpu(inc->i_hdr.h_len),
inc->i_hdr.h_dport);
if (sock_flag(sk, SOCK_RCVTSTAMP))
- inc->i_rx_tstamp = ktime_get_real();
+ inc->i_usercopy.rx_tstamp = ktime_get_real();
rds_inc_addref(inc);
inc->i_rx_lat_trace[RDS_MSG_RX_END] = local_clock();
list_add_tail(&inc->i_item, &rs->rs_recv_queue);
@@ -540,16 +540,18 @@ static int rds_cmsg_recv(struct rds_incoming *inc, struct msghdr *msg,
{
int ret = 0;
- if (inc->i_rdma_cookie) {
+ if (inc->i_usercopy.rdma_cookie) {
ret = put_cmsg(msg, SOL_RDS, RDS_CMSG_RDMA_DEST,
- sizeof(inc->i_rdma_cookie), &inc->i_rdma_cookie);
+ sizeof(inc->i_usercopy.rdma_cookie),
+ &inc->i_usercopy.rdma_cookie);
if (ret)
goto out;
}
- if ((inc->i_rx_tstamp != 0) &&
+ if ((inc->i_usercopy.rx_tstamp != 0) &&
sock_flag(rds_rs_to_sk(rs), SOCK_RCVTSTAMP)) {
- struct __kernel_old_timeval tv = ns_to_kernel_old_timeval(inc->i_rx_tstamp);
+ struct __kernel_old_timeval tv =
+ ns_to_kernel_old_timeval(inc->i_usercopy.rx_tstamp);
if (!sock_flag(rds_rs_to_sk(rs), SOCK_TSTAMP_NEW)) {
ret = put_cmsg(msg, SOL_SOCKET, SO_TIMESTAMP_OLD,
@@ -811,6 +813,7 @@ void rds6_inc_info_copy(struct rds_incoming *inc,
minfo6.seq = be64_to_cpu(inc->i_hdr.h_sequence);
minfo6.len = be32_to_cpu(inc->i_hdr.h_len);
+ minfo6.tos = inc->i_conn->c_tos;
if (flip) {
minfo6.laddr = *daddr;
@@ -824,6 +827,8 @@ void rds6_inc_info_copy(struct rds_incoming *inc,
minfo6.fport = inc->i_hdr.h_dport;
}
+ minfo6.flags = 0;
+
rds_info_copy(iter, &minfo6, sizeof(minfo6));
}
#endif
diff --git a/net/rds/send.c b/net/rds/send.c
index 9ce552a..82dcd8b 100644
--- a/net/rds/send.c
+++ b/net/rds/send.c
@@ -1144,7 +1144,7 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len)
case AF_INET:
if (usin->sin_addr.s_addr == htonl(INADDR_ANY) ||
usin->sin_addr.s_addr == htonl(INADDR_BROADCAST) ||
- IN_MULTICAST(ntohl(usin->sin_addr.s_addr))) {
+ ipv4_is_multicast(usin->sin_addr.s_addr)) {
ret = -EINVAL;
goto out;
}
@@ -1175,7 +1175,7 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len)
addr4 = sin6->sin6_addr.s6_addr32[3];
if (addr4 == htonl(INADDR_ANY) ||
addr4 == htonl(INADDR_BROADCAST) ||
- IN_MULTICAST(ntohl(addr4))) {
+ ipv4_is_multicast(addr4)) {
ret = -EINVAL;
goto out;
}
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index 0dbbfd1..d72ddb6 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -862,7 +862,6 @@ static void rxrpc_sock_destructor(struct sock *sk)
static int rxrpc_release_sock(struct sock *sk)
{
struct rxrpc_sock *rx = rxrpc_sk(sk);
- struct rxrpc_net *rxnet = rxrpc_net(sock_net(&rx->sk));
_enter("%p{%d,%d}", sk, sk->sk_state, refcount_read(&sk->sk_refcnt));
@@ -898,8 +897,6 @@ static int rxrpc_release_sock(struct sock *sk)
rxrpc_release_calls_on_socket(rx);
flush_workqueue(rxrpc_workqueue);
rxrpc_purge_queue(&sk->sk_receive_queue);
- rxrpc_queue_work(&rxnet->service_conn_reaper);
- rxrpc_queue_work(&rxnet->client_conn_reaper);
rxrpc_unuse_local(rx->local);
rx->local = NULL;
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index fa5b030..1091bf3 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -185,11 +185,17 @@ struct rxrpc_host_header {
* - max 48 bytes (struct sk_buff::cb)
*/
struct rxrpc_skb_priv {
- union {
- u8 nr_jumbo; /* Number of jumbo subpackets */
- };
+ atomic_t nr_ring_pins; /* Number of rxtx ring pins */
+ u8 nr_subpackets; /* Number of subpackets */
+ u8 rx_flags; /* Received packet flags */
+#define RXRPC_SKB_INCL_LAST 0x01 /* - Includes last packet */
+#define RXRPC_SKB_TX_BUFFER 0x02 /* - Is transmit buffer */
union {
int remain; /* amount of space remaining for next write */
+
+ /* List of requested ACKs on subpackets */
+ unsigned long rx_req_ack[(RXRPC_MAX_NR_JUMBO + BITS_PER_LONG - 1) /
+ BITS_PER_LONG];
};
struct rxrpc_host_header hdr; /* RxRPC packet header from this packet */
@@ -617,8 +623,7 @@ struct rxrpc_call {
#define RXRPC_TX_ANNO_LAST 0x04
#define RXRPC_TX_ANNO_RESENT 0x08
-#define RXRPC_RX_ANNO_JUMBO 0x3f /* Jumbo subpacket number + 1 if not zero */
-#define RXRPC_RX_ANNO_JLAST 0x40 /* Set if last element of a jumbo packet */
+#define RXRPC_RX_ANNO_SUBPACKET 0x3f /* Subpacket number in jumbogram */
#define RXRPC_RX_ANNO_VERIFIED 0x80 /* Set if verified and decrypted */
rxrpc_seq_t tx_hard_ack; /* Dead slot in buffer; the first transmitted but
* not hard-ACK'd packet follows this.
@@ -909,6 +914,7 @@ void rxrpc_disconnect_client_call(struct rxrpc_call *);
void rxrpc_put_client_conn(struct rxrpc_connection *);
void rxrpc_discard_expired_client_conns(struct work_struct *);
void rxrpc_destroy_all_client_connections(struct rxrpc_net *);
+void rxrpc_clean_up_local_conns(struct rxrpc_local *);
/*
* conn_event.c
@@ -1109,6 +1115,7 @@ void rxrpc_kernel_data_consumed(struct rxrpc_call *, struct sk_buff *);
void rxrpc_packet_destructor(struct sk_buff *);
void rxrpc_new_skb(struct sk_buff *, enum rxrpc_skb_trace);
void rxrpc_see_skb(struct sk_buff *, enum rxrpc_skb_trace);
+void rxrpc_eaten_skb(struct sk_buff *, enum rxrpc_skb_trace);
void rxrpc_get_skb(struct sk_buff *, enum rxrpc_skb_trace);
void rxrpc_free_skb(struct sk_buff *, enum rxrpc_skb_trace);
void rxrpc_purge_queue(struct sk_buff_head *);
diff --git a/net/rxrpc/call_event.c b/net/rxrpc/call_event.c
index c767679b..cedbbb3 100644
--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -199,7 +199,7 @@ static void rxrpc_resend(struct rxrpc_call *call, unsigned long now_j)
continue;
skb = call->rxtx_buffer[ix];
- rxrpc_see_skb(skb, rxrpc_skb_tx_seen);
+ rxrpc_see_skb(skb, rxrpc_skb_seen);
if (anno_type == RXRPC_TX_ANNO_UNACK) {
if (ktime_after(skb->tstamp, max_age)) {
@@ -255,18 +255,18 @@ static void rxrpc_resend(struct rxrpc_call *call, unsigned long now_j)
continue;
skb = call->rxtx_buffer[ix];
- rxrpc_get_skb(skb, rxrpc_skb_tx_got);
+ rxrpc_get_skb(skb, rxrpc_skb_got);
spin_unlock_bh(&call->lock);
if (rxrpc_send_data_packet(call, skb, true) < 0) {
- rxrpc_free_skb(skb, rxrpc_skb_tx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
return;
}
if (rxrpc_is_client_call(call))
rxrpc_expose_client_call(call);
- rxrpc_free_skb(skb, rxrpc_skb_tx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
spin_lock_bh(&call->lock);
/* We need to clear the retransmit state, but there are two
diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c
index 60cbc81..32d8dc6 100644
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -422,6 +422,19 @@ void rxrpc_get_call(struct rxrpc_call *call, enum rxrpc_call_trace op)
}
/*
+ * Clean up the RxTx skb ring.
+ */
+static void rxrpc_cleanup_ring(struct rxrpc_call *call)
+{
+ int i;
+
+ for (i = 0; i < RXRPC_RXTX_BUFF_SIZE; i++) {
+ rxrpc_free_skb(call->rxtx_buffer[i], rxrpc_skb_cleaned);
+ call->rxtx_buffer[i] = NULL;
+ }
+}
+
+/*
* Detach a call from its owning socket.
*/
void rxrpc_release_call(struct rxrpc_sock *rx, struct rxrpc_call *call)
@@ -429,7 +442,6 @@ void rxrpc_release_call(struct rxrpc_sock *rx, struct rxrpc_call *call)
const void *here = __builtin_return_address(0);
struct rxrpc_connection *conn = call->conn;
bool put = false;
- int i;
_enter("{%d,%d}", call->debug_id, atomic_read(&call->usage));
@@ -481,13 +493,7 @@ void rxrpc_release_call(struct rxrpc_sock *rx, struct rxrpc_call *call)
conn->security->free_call_crypto(call);
}
- for (i = 0; i < RXRPC_RXTX_BUFF_SIZE; i++) {
- rxrpc_free_skb(call->rxtx_buffer[i],
- (call->tx_phase ? rxrpc_skb_tx_cleaned :
- rxrpc_skb_rx_cleaned));
- call->rxtx_buffer[i] = NULL;
- }
-
+ rxrpc_cleanup_ring(call);
_leave("");
}
@@ -570,8 +576,6 @@ static void rxrpc_rcu_destroy_call(struct rcu_head *rcu)
*/
void rxrpc_cleanup_call(struct rxrpc_call *call)
{
- int i;
-
_net("DESTROY CALL %d", call->debug_id);
memset(&call->sock_node, 0xcd, sizeof(call->sock_node));
@@ -582,13 +586,8 @@ void rxrpc_cleanup_call(struct rxrpc_call *call)
ASSERT(test_bit(RXRPC_CALL_RELEASED, &call->flags));
ASSERTCMP(call->conn, ==, NULL);
- /* Clean up the Rx/Tx buffer */
- for (i = 0; i < RXRPC_RXTX_BUFF_SIZE; i++)
- rxrpc_free_skb(call->rxtx_buffer[i],
- (call->tx_phase ? rxrpc_skb_tx_cleaned :
- rxrpc_skb_rx_cleaned));
-
- rxrpc_free_skb(call->tx_pending, rxrpc_skb_tx_cleaned);
+ rxrpc_cleanup_ring(call);
+ rxrpc_free_skb(call->tx_pending, rxrpc_skb_cleaned);
call_rcu(&call->rcu, rxrpc_rcu_destroy_call);
}
diff --git a/net/rxrpc/conn_client.c b/net/rxrpc/conn_client.c
index aea82f9..3f1da1b 100644
--- a/net/rxrpc/conn_client.c
+++ b/net/rxrpc/conn_client.c
@@ -1162,3 +1162,47 @@ void rxrpc_destroy_all_client_connections(struct rxrpc_net *rxnet)
_leave("");
}
+
+/*
+ * Clean up the client connections on a local endpoint.
+ */
+void rxrpc_clean_up_local_conns(struct rxrpc_local *local)
+{
+ struct rxrpc_connection *conn, *tmp;
+ struct rxrpc_net *rxnet = local->rxnet;
+ unsigned int nr_active;
+ LIST_HEAD(graveyard);
+
+ _enter("");
+
+ spin_lock(&rxnet->client_conn_cache_lock);
+ nr_active = rxnet->nr_active_client_conns;
+
+ list_for_each_entry_safe(conn, tmp, &rxnet->idle_client_conns,
+ cache_link) {
+ if (conn->params.local == local) {
+ ASSERTCMP(conn->cache_state, ==, RXRPC_CONN_CLIENT_IDLE);
+
+ trace_rxrpc_client(conn, -1, rxrpc_client_discard);
+ if (!test_and_clear_bit(RXRPC_CONN_EXPOSED, &conn->flags))
+ BUG();
+ conn->cache_state = RXRPC_CONN_CLIENT_INACTIVE;
+ list_move(&conn->cache_link, &graveyard);
+ nr_active--;
+ }
+ }
+
+ rxnet->nr_active_client_conns = nr_active;
+ spin_unlock(&rxnet->client_conn_cache_lock);
+ ASSERTCMP(nr_active, >=, 0);
+
+ while (!list_empty(&graveyard)) {
+ conn = list_entry(graveyard.next,
+ struct rxrpc_connection, cache_link);
+ list_del_init(&conn->cache_link);
+
+ rxrpc_put_connection(conn);
+ }
+
+ _leave(" [culled]");
+}
diff --git a/net/rxrpc/conn_event.c b/net/rxrpc/conn_event.c
index df6624c..a1ceef4 100644
--- a/net/rxrpc/conn_event.c
+++ b/net/rxrpc/conn_event.c
@@ -472,7 +472,7 @@ void rxrpc_process_connection(struct work_struct *work)
/* go through the conn-level event packets, releasing the ref on this
* connection that each one has when we've finished with it */
while ((skb = skb_dequeue(&conn->rx_queue))) {
- rxrpc_see_skb(skb, rxrpc_skb_rx_seen);
+ rxrpc_see_skb(skb, rxrpc_skb_seen);
ret = rxrpc_process_event(conn, skb, &abort_code);
switch (ret) {
case -EPROTO:
@@ -484,7 +484,7 @@ void rxrpc_process_connection(struct work_struct *work)
goto requeue_and_leave;
case -ECONNABORTED:
default:
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
break;
}
}
@@ -501,6 +501,6 @@ void rxrpc_process_connection(struct work_struct *work)
protocol_error:
if (rxrpc_abort_connection(conn, ret, abort_code) < 0)
goto requeue_and_leave;
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
goto out;
}
diff --git a/net/rxrpc/conn_object.c b/net/rxrpc/conn_object.c
index 434ef39..ed05b69 100644
--- a/net/rxrpc/conn_object.c
+++ b/net/rxrpc/conn_object.c
@@ -398,7 +398,7 @@ void rxrpc_service_connection_reaper(struct work_struct *work)
if (conn->state == RXRPC_CONN_SERVICE_PREALLOC)
continue;
- if (rxnet->live) {
+ if (rxnet->live && !conn->params.local->dead) {
idle_timestamp = READ_ONCE(conn->idle_timestamp);
expire_at = idle_timestamp + rxrpc_connection_expiry * HZ;
if (conn->params.local->service_closed)
diff --git a/net/rxrpc/input.c b/net/rxrpc/input.c
index dd47d46..d122c53 100644
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -233,7 +233,7 @@ static bool rxrpc_rotate_tx_window(struct rxrpc_call *call, rxrpc_seq_t to,
ix = call->tx_hard_ack & RXRPC_RXTX_BUFF_MASK;
skb = call->rxtx_buffer[ix];
annotation = call->rxtx_annotations[ix];
- rxrpc_see_skb(skb, rxrpc_skb_tx_rotated);
+ rxrpc_see_skb(skb, rxrpc_skb_rotated);
call->rxtx_buffer[ix] = NULL;
call->rxtx_annotations[ix] = 0;
skb->next = list;
@@ -258,7 +258,7 @@ static bool rxrpc_rotate_tx_window(struct rxrpc_call *call, rxrpc_seq_t to,
skb = list;
list = skb->next;
skb_mark_not_on_list(skb);
- rxrpc_free_skb(skb, rxrpc_skb_tx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
}
return rot_last;
@@ -347,7 +347,7 @@ static bool rxrpc_receiving_reply(struct rxrpc_call *call)
}
/*
- * Scan a jumbo packet to validate its structure and to work out how many
+ * Scan a data packet to validate its structure and to work out how many
* subpackets it contains.
*
* A jumbo packet is a collection of consecutive packets glued together with
@@ -358,16 +358,21 @@ static bool rxrpc_receiving_reply(struct rxrpc_call *call)
* the last are RXRPC_JUMBO_DATALEN in size. The last subpacket may be of any
* size.
*/
-static bool rxrpc_validate_jumbo(struct sk_buff *skb)
+static bool rxrpc_validate_data(struct sk_buff *skb)
{
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
unsigned int offset = sizeof(struct rxrpc_wire_header);
unsigned int len = skb->len;
- int nr_jumbo = 1;
u8 flags = sp->hdr.flags;
- do {
- nr_jumbo++;
+ for (;;) {
+ if (flags & RXRPC_REQUEST_ACK)
+ __set_bit(sp->nr_subpackets, sp->rx_req_ack);
+ sp->nr_subpackets++;
+
+ if (!(flags & RXRPC_JUMBO_PACKET))
+ break;
+
if (len - offset < RXRPC_JUMBO_SUBPKTLEN)
goto protocol_error;
if (flags & RXRPC_LAST_PACKET)
@@ -376,9 +381,10 @@ static bool rxrpc_validate_jumbo(struct sk_buff *skb)
if (skb_copy_bits(skb, offset, &flags, 1) < 0)
goto protocol_error;
offset += sizeof(struct rxrpc_jumbo_header);
- } while (flags & RXRPC_JUMBO_PACKET);
+ }
- sp->nr_jumbo = nr_jumbo;
+ if (flags & RXRPC_LAST_PACKET)
+ sp->rx_flags |= RXRPC_SKB_INCL_LAST;
return true;
protocol_error:
@@ -399,10 +405,10 @@ static bool rxrpc_validate_jumbo(struct sk_buff *skb)
* (that information is encoded in the ACK packet).
*/
static void rxrpc_input_dup_data(struct rxrpc_call *call, rxrpc_seq_t seq,
- u8 annotation, bool *_jumbo_bad)
+ bool is_jumbo, bool *_jumbo_bad)
{
/* Discard normal packets that are duplicates. */
- if (annotation == 0)
+ if (is_jumbo)
return;
/* Skip jumbo subpackets that are duplicates. When we've had three or
@@ -416,29 +422,30 @@ static void rxrpc_input_dup_data(struct rxrpc_call *call, rxrpc_seq_t seq,
}
/*
- * Process a DATA packet, adding the packet to the Rx ring.
+ * Process a DATA packet, adding the packet to the Rx ring. The caller's
+ * packet ref must be passed on or discarded.
*/
static void rxrpc_input_data(struct rxrpc_call *call, struct sk_buff *skb)
{
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
enum rxrpc_call_state state;
- unsigned int offset = sizeof(struct rxrpc_wire_header);
- unsigned int ix;
+ unsigned int j;
rxrpc_serial_t serial = sp->hdr.serial, ack_serial = 0;
- rxrpc_seq_t seq = sp->hdr.seq, hard_ack;
- bool immediate_ack = false, jumbo_bad = false, queued;
- u16 len;
- u8 ack = 0, flags, annotation = 0;
+ rxrpc_seq_t seq0 = sp->hdr.seq, hard_ack;
+ bool immediate_ack = false, jumbo_bad = false;
+ u8 ack = 0;
_enter("{%u,%u},{%u,%u}",
- call->rx_hard_ack, call->rx_top, skb->len, seq);
+ call->rx_hard_ack, call->rx_top, skb->len, seq0);
- _proto("Rx DATA %%%u { #%u f=%02x }",
- sp->hdr.serial, seq, sp->hdr.flags);
+ _proto("Rx DATA %%%u { #%u f=%02x n=%u }",
+ sp->hdr.serial, seq0, sp->hdr.flags, sp->nr_subpackets);
state = READ_ONCE(call->state);
- if (state >= RXRPC_CALL_COMPLETE)
+ if (state >= RXRPC_CALL_COMPLETE) {
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
return;
+ }
if (call->state == RXRPC_CALL_SERVER_RECV_REQUEST) {
unsigned long timo = READ_ONCE(call->next_req_timo);
@@ -463,137 +470,137 @@ static void rxrpc_input_data(struct rxrpc_call *call, struct sk_buff *skb)
!rxrpc_receiving_reply(call))
goto unlock;
- call->ackr_prev_seq = seq;
-
+ call->ackr_prev_seq = seq0;
hard_ack = READ_ONCE(call->rx_hard_ack);
- if (after(seq, hard_ack + call->rx_winsize)) {
- ack = RXRPC_ACK_EXCEEDS_WINDOW;
- ack_serial = serial;
- goto ack;
- }
- flags = sp->hdr.flags;
- if (flags & RXRPC_JUMBO_PACKET) {
+ if (sp->nr_subpackets > 1) {
if (call->nr_jumbo_bad > 3) {
ack = RXRPC_ACK_NOSPACE;
ack_serial = serial;
goto ack;
}
- annotation = 1;
}
-next_subpacket:
- queued = false;
- ix = seq & RXRPC_RXTX_BUFF_MASK;
- len = skb->len;
- if (flags & RXRPC_JUMBO_PACKET)
- len = RXRPC_JUMBO_DATALEN;
+ for (j = 0; j < sp->nr_subpackets; j++) {
+ rxrpc_serial_t serial = sp->hdr.serial + j;
+ rxrpc_seq_t seq = seq0 + j;
+ unsigned int ix = seq & RXRPC_RXTX_BUFF_MASK;
+ bool terminal = (j == sp->nr_subpackets - 1);
+ bool last = terminal && (sp->rx_flags & RXRPC_SKB_INCL_LAST);
+ u8 flags, annotation = j;
- if (flags & RXRPC_LAST_PACKET) {
- if (test_bit(RXRPC_CALL_RX_LAST, &call->flags) &&
- seq != call->rx_top) {
- rxrpc_proto_abort("LSN", call, seq);
- goto unlock;
+ _proto("Rx DATA+%u %%%u { #%x t=%u l=%u }",
+ j, serial, seq, terminal, last);
+
+ if (last) {
+ if (test_bit(RXRPC_CALL_RX_LAST, &call->flags) &&
+ seq != call->rx_top) {
+ rxrpc_proto_abort("LSN", call, seq);
+ goto unlock;
+ }
+ } else {
+ if (test_bit(RXRPC_CALL_RX_LAST, &call->flags) &&
+ after_eq(seq, call->rx_top)) {
+ rxrpc_proto_abort("LSA", call, seq);
+ goto unlock;
+ }
}
- } else {
- if (test_bit(RXRPC_CALL_RX_LAST, &call->flags) &&
- after_eq(seq, call->rx_top)) {
- rxrpc_proto_abort("LSA", call, seq);
- goto unlock;
- }
- }
- trace_rxrpc_rx_data(call->debug_id, seq, serial, flags, annotation);
- if (before_eq(seq, hard_ack)) {
- ack = RXRPC_ACK_DUPLICATE;
- ack_serial = serial;
- goto skip;
- }
+ flags = 0;
+ if (last)
+ flags |= RXRPC_LAST_PACKET;
+ if (!terminal)
+ flags |= RXRPC_JUMBO_PACKET;
+ if (test_bit(j, sp->rx_req_ack))
+ flags |= RXRPC_REQUEST_ACK;
+ trace_rxrpc_rx_data(call->debug_id, seq, serial, flags, annotation);
- if (flags & RXRPC_REQUEST_ACK && !ack) {
- ack = RXRPC_ACK_REQUESTED;
- ack_serial = serial;
- }
-
- if (call->rxtx_buffer[ix]) {
- rxrpc_input_dup_data(call, seq, annotation, &jumbo_bad);
- if (ack != RXRPC_ACK_DUPLICATE) {
+ if (before_eq(seq, hard_ack)) {
ack = RXRPC_ACK_DUPLICATE;
ack_serial = serial;
+ continue;
}
- immediate_ack = true;
- goto skip;
- }
- /* Queue the packet. We use a couple of memory barriers here as need
- * to make sure that rx_top is perceived to be set after the buffer
- * pointer and that the buffer pointer is set after the annotation and
- * the skb data.
- *
- * Barriers against rxrpc_recvmsg_data() and rxrpc_rotate_rx_window()
- * and also rxrpc_fill_out_ack().
- */
- rxrpc_get_skb(skb, rxrpc_skb_rx_got);
- call->rxtx_annotations[ix] = annotation;
- smp_wmb();
- call->rxtx_buffer[ix] = skb;
- if (after(seq, call->rx_top)) {
- smp_store_release(&call->rx_top, seq);
- } else if (before(seq, call->rx_top)) {
- /* Send an immediate ACK if we fill in a hole */
- if (!ack) {
- ack = RXRPC_ACK_DELAY;
- ack_serial = serial;
+ if (call->rxtx_buffer[ix]) {
+ rxrpc_input_dup_data(call, seq, sp->nr_subpackets > 1,
+ &jumbo_bad);
+ if (ack != RXRPC_ACK_DUPLICATE) {
+ ack = RXRPC_ACK_DUPLICATE;
+ ack_serial = serial;
+ }
+ immediate_ack = true;
+ continue;
}
- immediate_ack = true;
- }
- if (flags & RXRPC_LAST_PACKET) {
- set_bit(RXRPC_CALL_RX_LAST, &call->flags);
- trace_rxrpc_receive(call, rxrpc_receive_queue_last, serial, seq);
- } else {
- trace_rxrpc_receive(call, rxrpc_receive_queue, serial, seq);
- }
- queued = true;
- if (after_eq(seq, call->rx_expect_next)) {
- if (after(seq, call->rx_expect_next)) {
- _net("OOS %u > %u", seq, call->rx_expect_next);
- ack = RXRPC_ACK_OUT_OF_SEQUENCE;
- ack_serial = serial;
- }
- call->rx_expect_next = seq + 1;
- }
-
-skip:
- offset += len;
- if (flags & RXRPC_JUMBO_PACKET) {
- if (skb_copy_bits(skb, offset, &flags, 1) < 0) {
- rxrpc_proto_abort("XJF", call, seq);
- goto unlock;
- }
- offset += sizeof(struct rxrpc_jumbo_header);
- seq++;
- serial++;
- annotation++;
- if (flags & RXRPC_JUMBO_PACKET)
- annotation |= RXRPC_RX_ANNO_JLAST;
if (after(seq, hard_ack + call->rx_winsize)) {
ack = RXRPC_ACK_EXCEEDS_WINDOW;
ack_serial = serial;
- if (!jumbo_bad) {
- call->nr_jumbo_bad++;
- jumbo_bad = true;
+ if (flags & RXRPC_JUMBO_PACKET) {
+ if (!jumbo_bad) {
+ call->nr_jumbo_bad++;
+ jumbo_bad = true;
+ }
}
+
goto ack;
}
- _proto("Rx DATA Jumbo %%%u", serial);
- goto next_subpacket;
- }
+ if (flags & RXRPC_REQUEST_ACK && !ack) {
+ ack = RXRPC_ACK_REQUESTED;
+ ack_serial = serial;
+ }
- if (queued && flags & RXRPC_LAST_PACKET && !ack) {
- ack = RXRPC_ACK_DELAY;
- ack_serial = serial;
+ /* Queue the packet. We use a couple of memory barriers here as need
+ * to make sure that rx_top is perceived to be set after the buffer
+ * pointer and that the buffer pointer is set after the annotation and
+ * the skb data.
+ *
+ * Barriers against rxrpc_recvmsg_data() and rxrpc_rotate_rx_window()
+ * and also rxrpc_fill_out_ack().
+ */
+ if (!terminal)
+ rxrpc_get_skb(skb, rxrpc_skb_got);
+ call->rxtx_annotations[ix] = annotation;
+ smp_wmb();
+ call->rxtx_buffer[ix] = skb;
+ if (after(seq, call->rx_top)) {
+ smp_store_release(&call->rx_top, seq);
+ } else if (before(seq, call->rx_top)) {
+ /* Send an immediate ACK if we fill in a hole */
+ if (!ack) {
+ ack = RXRPC_ACK_DELAY;
+ ack_serial = serial;
+ }
+ immediate_ack = true;
+ }
+
+ if (terminal) {
+ /* From this point on, we're not allowed to touch the
+ * packet any longer as its ref now belongs to the Rx
+ * ring.
+ */
+ skb = NULL;
+ }
+
+ if (last) {
+ set_bit(RXRPC_CALL_RX_LAST, &call->flags);
+ if (!ack) {
+ ack = RXRPC_ACK_DELAY;
+ ack_serial = serial;
+ }
+ trace_rxrpc_receive(call, rxrpc_receive_queue_last, serial, seq);
+ } else {
+ trace_rxrpc_receive(call, rxrpc_receive_queue, serial, seq);
+ }
+
+ if (after_eq(seq, call->rx_expect_next)) {
+ if (after(seq, call->rx_expect_next)) {
+ _net("OOS %u > %u", seq, call->rx_expect_next);
+ ack = RXRPC_ACK_OUT_OF_SEQUENCE;
+ ack_serial = serial;
+ }
+ call->rx_expect_next = seq + 1;
+ }
}
ack:
@@ -606,13 +613,14 @@ static void rxrpc_input_data(struct rxrpc_call *call, struct sk_buff *skb)
false, true,
rxrpc_propose_ack_input_data);
- if (sp->hdr.seq == READ_ONCE(call->rx_hard_ack) + 1) {
+ if (seq0 == READ_ONCE(call->rx_hard_ack) + 1) {
trace_rxrpc_notify_socket(call->debug_id, serial);
rxrpc_notify_socket(call);
}
unlock:
spin_unlock(&call->input_lock);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
_leave(" [queued]");
}
@@ -1021,7 +1029,7 @@ static void rxrpc_input_call_packet(struct rxrpc_call *call,
switch (sp->hdr.type) {
case RXRPC_PACKET_TYPE_DATA:
rxrpc_input_data(call, skb);
- break;
+ goto no_free;
case RXRPC_PACKET_TYPE_ACK:
rxrpc_input_ack(call, skb);
@@ -1048,6 +1056,8 @@ static void rxrpc_input_call_packet(struct rxrpc_call *call,
break;
}
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
+no_free:
_leave("");
}
@@ -1109,7 +1119,7 @@ static void rxrpc_post_packet_to_local(struct rxrpc_local *local,
skb_queue_tail(&local->event_queue, skb);
rxrpc_queue_local(local);
} else {
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
}
}
@@ -1124,7 +1134,7 @@ static void rxrpc_reject_packet(struct rxrpc_local *local, struct sk_buff *skb)
skb_queue_tail(&local->reject_queue, skb);
rxrpc_queue_local(local);
} else {
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
}
}
@@ -1188,7 +1198,7 @@ int rxrpc_input_packet(struct sock *udp_sk, struct sk_buff *skb)
if (skb->tstamp == 0)
skb->tstamp = ktime_get_real();
- rxrpc_new_skb(skb, rxrpc_skb_rx_received);
+ rxrpc_new_skb(skb, rxrpc_skb_received);
skb_pull(skb, sizeof(struct udphdr));
@@ -1205,7 +1215,7 @@ int rxrpc_input_packet(struct sock *udp_sk, struct sk_buff *skb)
static int lose;
if ((lose++ & 7) == 7) {
trace_rxrpc_rx_lose(sp);
- rxrpc_free_skb(skb, rxrpc_skb_rx_lost);
+ rxrpc_free_skb(skb, rxrpc_skb_lost);
return 0;
}
}
@@ -1237,9 +1247,26 @@ int rxrpc_input_packet(struct sock *udp_sk, struct sk_buff *skb)
if (sp->hdr.callNumber == 0 ||
sp->hdr.seq == 0)
goto bad_message;
- if (sp->hdr.flags & RXRPC_JUMBO_PACKET &&
- !rxrpc_validate_jumbo(skb))
+ if (!rxrpc_validate_data(skb))
goto bad_message;
+
+ /* Unshare the packet so that it can be modified for in-place
+ * decryption.
+ */
+ if (sp->hdr.securityIndex != 0) {
+ struct sk_buff *nskb = skb_unshare(skb, GFP_ATOMIC);
+ if (!nskb) {
+ rxrpc_eaten_skb(skb, rxrpc_skb_unshared_nomem);
+ goto out;
+ }
+
+ if (nskb != skb) {
+ rxrpc_eaten_skb(skb, rxrpc_skb_received);
+ rxrpc_new_skb(skb, rxrpc_skb_unshared);
+ skb = nskb;
+ sp = rxrpc_skb(skb);
+ }
+ }
break;
case RXRPC_PACKET_TYPE_CHALLENGE:
@@ -1373,11 +1400,14 @@ int rxrpc_input_packet(struct sock *udp_sk, struct sk_buff *skb)
mutex_unlock(&call->user_mutex);
}
+ /* Process a call packet; this either discards or passes on the ref
+ * elsewhere.
+ */
rxrpc_input_call_packet(call, skb);
- goto discard;
+ goto out;
discard:
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
out:
trace_rxrpc_rx_done(0, 0);
return 0;
diff --git a/net/rxrpc/local_event.c b/net/rxrpc/local_event.c
index e93a78f..3ce6d62 100644
--- a/net/rxrpc/local_event.c
+++ b/net/rxrpc/local_event.c
@@ -90,7 +90,7 @@ void rxrpc_process_local_events(struct rxrpc_local *local)
if (skb) {
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
- rxrpc_see_skb(skb, rxrpc_skb_rx_seen);
+ rxrpc_see_skb(skb, rxrpc_skb_seen);
_debug("{%d},{%u}", local->debug_id, sp->hdr.type);
switch (sp->hdr.type) {
@@ -108,7 +108,7 @@ void rxrpc_process_local_events(struct rxrpc_local *local)
break;
}
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
}
_leave("");
diff --git a/net/rxrpc/local_object.c b/net/rxrpc/local_object.c
index 72a6e12..3658726 100644
--- a/net/rxrpc/local_object.c
+++ b/net/rxrpc/local_object.c
@@ -426,11 +426,14 @@ static void rxrpc_local_destroyer(struct rxrpc_local *local)
_enter("%d", local->debug_id);
+ local->dead = true;
+
mutex_lock(&rxnet->local_mutex);
list_del_init(&local->link);
mutex_unlock(&rxnet->local_mutex);
- ASSERT(RB_EMPTY_ROOT(&local->client_conns));
+ rxrpc_clean_up_local_conns(local);
+ rxrpc_service_connection_reaper(&rxnet->service_conn_reaper);
ASSERT(!local->service);
if (socket) {
diff --git a/net/rxrpc/output.c b/net/rxrpc/output.c
index 369e516..935bb60 100644
--- a/net/rxrpc/output.c
+++ b/net/rxrpc/output.c
@@ -565,7 +565,7 @@ void rxrpc_reject_packets(struct rxrpc_local *local)
memset(&whdr, 0, sizeof(whdr));
while ((skb = skb_dequeue(&local->reject_queue))) {
- rxrpc_see_skb(skb, rxrpc_skb_rx_seen);
+ rxrpc_see_skb(skb, rxrpc_skb_seen);
sp = rxrpc_skb(skb);
switch (skb->mark) {
@@ -581,7 +581,7 @@ void rxrpc_reject_packets(struct rxrpc_local *local)
ioc = 2;
break;
default:
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
continue;
}
@@ -606,7 +606,7 @@ void rxrpc_reject_packets(struct rxrpc_local *local)
rxrpc_tx_point_reject);
}
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
}
_leave("");
diff --git a/net/rxrpc/peer_event.c b/net/rxrpc/peer_event.c
index 7666ec7..c97ebdc 100644
--- a/net/rxrpc/peer_event.c
+++ b/net/rxrpc/peer_event.c
@@ -163,11 +163,11 @@ void rxrpc_error_report(struct sock *sk)
_leave("UDP socket errqueue empty");
return;
}
- rxrpc_new_skb(skb, rxrpc_skb_rx_received);
+ rxrpc_new_skb(skb, rxrpc_skb_received);
serr = SKB_EXT_ERR(skb);
if (!skb->len && serr->ee.ee_origin == SO_EE_ORIGIN_TIMESTAMPING) {
_leave("UDP empty message");
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
return;
}
@@ -177,7 +177,7 @@ void rxrpc_error_report(struct sock *sk)
peer = NULL;
if (!peer) {
rcu_read_unlock();
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
_leave(" [no peer]");
return;
}
@@ -189,7 +189,7 @@ void rxrpc_error_report(struct sock *sk)
serr->ee.ee_code == ICMP_FRAG_NEEDED)) {
rxrpc_adjust_mtu(peer, serr);
rcu_read_unlock();
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
rxrpc_put_peer(peer);
_leave(" [MTU update]");
return;
@@ -197,7 +197,7 @@ void rxrpc_error_report(struct sock *sk)
rxrpc_store_error(peer, serr);
rcu_read_unlock();
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
rxrpc_put_peer(peer);
_leave("");
diff --git a/net/rxrpc/protocol.h b/net/rxrpc/protocol.h
index 99ce322..49bb972 100644
--- a/net/rxrpc/protocol.h
+++ b/net/rxrpc/protocol.h
@@ -89,6 +89,15 @@ struct rxrpc_jumbo_header {
#define RXRPC_JUMBO_DATALEN 1412 /* non-terminal jumbo packet data length */
#define RXRPC_JUMBO_SUBPKTLEN (RXRPC_JUMBO_DATALEN + sizeof(struct rxrpc_jumbo_header))
+/*
+ * The maximum number of subpackets that can possibly fit in a UDP packet is:
+ *
+ * ((max_IP - IP_hdr - UDP_hdr) / RXRPC_JUMBO_SUBPKTLEN) + 1
+ * = ((65535 - 28 - 28) / 1416) + 1
+ * = 46 non-terminal packets and 1 terminal packet.
+ */
+#define RXRPC_MAX_NR_JUMBO 47
+
/*****************************************************************************/
/*
* on-the-wire Rx ACK packet data payload
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index 9a7e1bc..3b0becb 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -177,7 +177,8 @@ static void rxrpc_rotate_rx_window(struct rxrpc_call *call)
struct sk_buff *skb;
rxrpc_serial_t serial;
rxrpc_seq_t hard_ack, top;
- u8 flags;
+ bool last = false;
+ u8 subpacket;
int ix;
_enter("%d", call->debug_id);
@@ -189,23 +190,25 @@ static void rxrpc_rotate_rx_window(struct rxrpc_call *call)
hard_ack++;
ix = hard_ack & RXRPC_RXTX_BUFF_MASK;
skb = call->rxtx_buffer[ix];
- rxrpc_see_skb(skb, rxrpc_skb_rx_rotated);
+ rxrpc_see_skb(skb, rxrpc_skb_rotated);
sp = rxrpc_skb(skb);
- flags = sp->hdr.flags;
- serial = sp->hdr.serial;
- if (call->rxtx_annotations[ix] & RXRPC_RX_ANNO_JUMBO)
- serial += (call->rxtx_annotations[ix] & RXRPC_RX_ANNO_JUMBO) - 1;
+
+ subpacket = call->rxtx_annotations[ix] & RXRPC_RX_ANNO_SUBPACKET;
+ serial = sp->hdr.serial + subpacket;
+
+ if (subpacket == sp->nr_subpackets - 1 &&
+ sp->rx_flags & RXRPC_SKB_INCL_LAST)
+ last = true;
call->rxtx_buffer[ix] = NULL;
call->rxtx_annotations[ix] = 0;
/* Barrier against rxrpc_input_data(). */
smp_store_release(&call->rx_hard_ack, hard_ack);
- rxrpc_free_skb(skb, rxrpc_skb_rx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
- _debug("%u,%u,%02x", hard_ack, top, flags);
trace_rxrpc_receive(call, rxrpc_receive_rotate, serial, hard_ack);
- if (flags & RXRPC_LAST_PACKET) {
+ if (last) {
rxrpc_end_rx_phase(call, serial);
} else {
/* Check to see if there's an ACK that needs sending. */
@@ -233,18 +236,19 @@ static int rxrpc_verify_packet(struct rxrpc_call *call, struct sk_buff *skb,
struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
rxrpc_seq_t seq = sp->hdr.seq;
u16 cksum = sp->hdr.cksum;
+ u8 subpacket = annotation & RXRPC_RX_ANNO_SUBPACKET;
_enter("");
/* For all but the head jumbo subpacket, the security checksum is in a
* jumbo header immediately prior to the data.
*/
- if ((annotation & RXRPC_RX_ANNO_JUMBO) > 1) {
+ if (subpacket > 0) {
__be16 tmp;
if (skb_copy_bits(skb, offset - 2, &tmp, 2) < 0)
BUG();
cksum = ntohs(tmp);
- seq += (annotation & RXRPC_RX_ANNO_JUMBO) - 1;
+ seq += subpacket;
}
return call->conn->security->verify_packet(call, skb, offset, len,
@@ -265,19 +269,18 @@ static int rxrpc_locate_data(struct rxrpc_call *call, struct sk_buff *skb,
u8 *_annotation,
unsigned int *_offset, unsigned int *_len)
{
+ struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
unsigned int offset = sizeof(struct rxrpc_wire_header);
unsigned int len;
int ret;
u8 annotation = *_annotation;
+ u8 subpacket = annotation & RXRPC_RX_ANNO_SUBPACKET;
/* Locate the subpacket */
+ offset += subpacket * RXRPC_JUMBO_SUBPKTLEN;
len = skb->len - offset;
- if ((annotation & RXRPC_RX_ANNO_JUMBO) > 0) {
- offset += (((annotation & RXRPC_RX_ANNO_JUMBO) - 1) *
- RXRPC_JUMBO_SUBPKTLEN);
- len = (annotation & RXRPC_RX_ANNO_JLAST) ?
- skb->len - offset : RXRPC_JUMBO_SUBPKTLEN;
- }
+ if (subpacket < sp->nr_subpackets - 1)
+ len = RXRPC_JUMBO_DATALEN;
if (!(annotation & RXRPC_RX_ANNO_VERIFIED)) {
ret = rxrpc_verify_packet(call, skb, annotation, offset, len);
@@ -303,6 +306,7 @@ static int rxrpc_recvmsg_data(struct socket *sock, struct rxrpc_call *call,
{
struct rxrpc_skb_priv *sp;
struct sk_buff *skb;
+ rxrpc_serial_t serial;
rxrpc_seq_t hard_ack, top, seq;
size_t remain;
bool last;
@@ -336,12 +340,15 @@ static int rxrpc_recvmsg_data(struct socket *sock, struct rxrpc_call *call,
break;
}
smp_rmb();
- rxrpc_see_skb(skb, rxrpc_skb_rx_seen);
+ rxrpc_see_skb(skb, rxrpc_skb_seen);
sp = rxrpc_skb(skb);
- if (!(flags & MSG_PEEK))
+ if (!(flags & MSG_PEEK)) {
+ serial = sp->hdr.serial;
+ serial += call->rxtx_annotations[ix] & RXRPC_RX_ANNO_SUBPACKET;
trace_rxrpc_receive(call, rxrpc_receive_front,
- sp->hdr.serial, seq);
+ serial, seq);
+ }
if (msg)
sock_recv_timestamp(msg, sock->sk, skb);
diff --git a/net/rxrpc/rxkad.c b/net/rxrpc/rxkad.c
index dbb109d..8d8aa3c 100644
--- a/net/rxrpc/rxkad.c
+++ b/net/rxrpc/rxkad.c
@@ -223,10 +223,8 @@ static int rxkad_secure_packet_encrypt(const struct rxrpc_call *call,
struct rxrpc_skb_priv *sp;
struct rxrpc_crypt iv;
struct scatterlist sg[16];
- struct sk_buff *trailer;
unsigned int len;
u16 check;
- int nsg;
int err;
sp = rxrpc_skb(skb);
@@ -250,15 +248,14 @@ static int rxkad_secure_packet_encrypt(const struct rxrpc_call *call,
crypto_skcipher_encrypt(req);
/* we want to encrypt the skbuff in-place */
- nsg = skb_cow_data(skb, 0, &trailer);
- err = -ENOMEM;
- if (nsg < 0 || nsg > 16)
+ err = -EMSGSIZE;
+ if (skb_shinfo(skb)->nr_frags > 16)
goto out;
len = data_size + call->conn->size_align - 1;
len &= ~(call->conn->size_align - 1);
- sg_init_table(sg, nsg);
+ sg_init_table(sg, ARRAY_SIZE(sg));
err = skb_to_sgvec(skb, sg, 0, len);
if (unlikely(err < 0))
goto out;
@@ -359,11 +356,10 @@ static int rxkad_verify_packet_1(struct rxrpc_call *call, struct sk_buff *skb,
struct rxkad_level1_hdr sechdr;
struct rxrpc_crypt iv;
struct scatterlist sg[16];
- struct sk_buff *trailer;
bool aborted;
u32 data_size, buf;
u16 check;
- int nsg, ret;
+ int ret;
_enter("");
@@ -376,11 +372,7 @@ static int rxkad_verify_packet_1(struct rxrpc_call *call, struct sk_buff *skb,
/* Decrypt the skbuff in-place. TODO: We really want to decrypt
* directly into the target buffer.
*/
- nsg = skb_cow_data(skb, 0, &trailer);
- if (nsg < 0 || nsg > 16)
- goto nomem;
-
- sg_init_table(sg, nsg);
+ sg_init_table(sg, ARRAY_SIZE(sg));
ret = skb_to_sgvec(skb, sg, offset, 8);
if (unlikely(ret < 0))
return ret;
@@ -428,10 +420,6 @@ static int rxkad_verify_packet_1(struct rxrpc_call *call, struct sk_buff *skb,
if (aborted)
rxrpc_send_abort_packet(call);
return -EPROTO;
-
-nomem:
- _leave(" = -ENOMEM");
- return -ENOMEM;
}
/*
@@ -446,7 +434,6 @@ static int rxkad_verify_packet_2(struct rxrpc_call *call, struct sk_buff *skb,
struct rxkad_level2_hdr sechdr;
struct rxrpc_crypt iv;
struct scatterlist _sg[4], *sg;
- struct sk_buff *trailer;
bool aborted;
u32 data_size, buf;
u16 check;
@@ -463,12 +450,11 @@ static int rxkad_verify_packet_2(struct rxrpc_call *call, struct sk_buff *skb,
/* Decrypt the skbuff in-place. TODO: We really want to decrypt
* directly into the target buffer.
*/
- nsg = skb_cow_data(skb, 0, &trailer);
- if (nsg < 0)
- goto nomem;
-
sg = _sg;
- if (unlikely(nsg > 4)) {
+ nsg = skb_shinfo(skb)->nr_frags;
+ if (nsg <= 4) {
+ nsg = 4;
+ } else {
sg = kmalloc_array(nsg, sizeof(*sg), GFP_NOIO);
if (!sg)
goto nomem;
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index bae1443..6a1547b2 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -176,7 +176,7 @@ static int rxrpc_queue_packet(struct rxrpc_sock *rx, struct rxrpc_call *call,
skb->tstamp = ktime_get_real();
ix = seq & RXRPC_RXTX_BUFF_MASK;
- rxrpc_get_skb(skb, rxrpc_skb_tx_got);
+ rxrpc_get_skb(skb, rxrpc_skb_got);
call->rxtx_annotations[ix] = annotation;
smp_wmb();
call->rxtx_buffer[ix] = skb;
@@ -248,7 +248,7 @@ static int rxrpc_queue_packet(struct rxrpc_sock *rx, struct rxrpc_call *call,
}
out:
- rxrpc_free_skb(skb, rxrpc_skb_tx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
_leave(" = %d", ret);
return ret;
}
@@ -289,7 +289,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
skb = call->tx_pending;
call->tx_pending = NULL;
- rxrpc_see_skb(skb, rxrpc_skb_tx_seen);
+ rxrpc_see_skb(skb, rxrpc_skb_seen);
copied = 0;
do {
@@ -336,7 +336,9 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
if (!skb)
goto maybe_error;
- rxrpc_new_skb(skb, rxrpc_skb_tx_new);
+ sp = rxrpc_skb(skb);
+ sp->rx_flags |= RXRPC_SKB_TX_BUFFER;
+ rxrpc_new_skb(skb, rxrpc_skb_new);
_debug("ALLOC SEND %p", skb);
@@ -346,7 +348,6 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
skb_reserve(skb, call->conn->security_size);
skb->len += call->conn->security_size;
- sp = rxrpc_skb(skb);
sp->remain = chunk;
if (sp->remain > skb_tailroom(skb))
sp->remain = skb_tailroom(skb);
@@ -439,7 +440,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
return ret;
call_terminated:
- rxrpc_free_skb(skb, rxrpc_skb_tx_freed);
+ rxrpc_free_skb(skb, rxrpc_skb_freed);
_leave(" = %d", call->error);
return call->error;
diff --git a/net/rxrpc/skbuff.c b/net/rxrpc/skbuff.c
index 9ad5045..0348d2b 100644
--- a/net/rxrpc/skbuff.c
+++ b/net/rxrpc/skbuff.c
@@ -14,7 +14,8 @@
#include <net/af_rxrpc.h>
#include "ar-internal.h"
-#define select_skb_count(op) (op >= rxrpc_skb_tx_cleaned ? &rxrpc_n_tx_skbs : &rxrpc_n_rx_skbs)
+#define is_tx_skb(skb) (rxrpc_skb(skb)->rx_flags & RXRPC_SKB_TX_BUFFER)
+#define select_skb_count(skb) (is_tx_skb(skb) ? &rxrpc_n_tx_skbs : &rxrpc_n_rx_skbs)
/*
* Note the allocation or reception of a socket buffer.
@@ -22,8 +23,9 @@
void rxrpc_new_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
{
const void *here = __builtin_return_address(0);
- int n = atomic_inc_return(select_skb_count(op));
- trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n, here);
+ int n = atomic_inc_return(select_skb_count(skb));
+ trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n,
+ rxrpc_skb(skb)->rx_flags, here);
}
/*
@@ -33,8 +35,9 @@ void rxrpc_see_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
{
const void *here = __builtin_return_address(0);
if (skb) {
- int n = atomic_read(select_skb_count(op));
- trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n, here);
+ int n = atomic_read(select_skb_count(skb));
+ trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n,
+ rxrpc_skb(skb)->rx_flags, here);
}
}
@@ -44,12 +47,23 @@ void rxrpc_see_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
void rxrpc_get_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
{
const void *here = __builtin_return_address(0);
- int n = atomic_inc_return(select_skb_count(op));
- trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n, here);
+ int n = atomic_inc_return(select_skb_count(skb));
+ trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n,
+ rxrpc_skb(skb)->rx_flags, here);
skb_get(skb);
}
/*
+ * Note the dropping of a ref on a socket buffer by the core.
+ */
+void rxrpc_eaten_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
+{
+ const void *here = __builtin_return_address(0);
+ int n = atomic_inc_return(&rxrpc_n_rx_skbs);
+ trace_rxrpc_skb(skb, op, 0, n, 0, here);
+}
+
+/*
* Note the destruction of a socket buffer.
*/
void rxrpc_free_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
@@ -58,8 +72,9 @@ void rxrpc_free_skb(struct sk_buff *skb, enum rxrpc_skb_trace op)
if (skb) {
int n;
CHECK_SLAB_OKAY(&skb->users);
- n = atomic_dec_return(select_skb_count(op));
- trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n, here);
+ n = atomic_dec_return(select_skb_count(skb));
+ trace_rxrpc_skb(skb, op, refcount_read(&skb->users), n,
+ rxrpc_skb(skb)->rx_flags, here);
kfree_skb(skb);
}
}
@@ -72,9 +87,10 @@ void rxrpc_purge_queue(struct sk_buff_head *list)
const void *here = __builtin_return_address(0);
struct sk_buff *skb;
while ((skb = skb_dequeue((list))) != NULL) {
- int n = atomic_dec_return(select_skb_count(rxrpc_skb_rx_purged));
- trace_rxrpc_skb(skb, rxrpc_skb_rx_purged,
- refcount_read(&skb->users), n, here);
+ int n = atomic_dec_return(select_skb_count(skb));
+ trace_rxrpc_skb(skb, rxrpc_skb_purged,
+ refcount_read(&skb->users), n,
+ rxrpc_skb(skb)->rx_flags, here);
kfree_skb(skb);
}
}
diff --git a/net/sched/Kconfig b/net/sched/Kconfig
index afd2ba1..b3faafe 100644
--- a/net/sched/Kconfig
+++ b/net/sched/Kconfig
@@ -963,6 +963,19 @@
tristate "Support to encoding decoding skb tcindex on IFE action"
depends on NET_ACT_IFE
+config NET_TC_SKB_EXT
+ bool "TC recirculation support"
+ depends on NET_CLS_ACT
+ default y if NET_CLS_ACT
+ select SKB_EXTENSIONS
+
+ help
+ Say Y here to allow tc chain misses to continue in OvS datapath in
+ the correct recirc_id, and hardware chain misses to continue in
+ the correct chain in tc software datapath.
+
+ Say N here if you won't be using tc<->ovs offload or tc chains offload.
+
endif # NET_SCHED
config NET_SCH_FIFO
diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
index fd1f7e7..04b7bd4 100644
--- a/net/sched/act_bpf.c
+++ b/net/sched/act_bpf.c
@@ -422,7 +422,7 @@ static __net_init int bpf_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, bpf_net_id);
- return tc_action_net_init(tn, &act_bpf_ops);
+ return tc_action_net_init(net, tn, &act_bpf_ops);
}
static void __net_exit bpf_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_connmark.c b/net/sched/act_connmark.c
index 32ac04d..2b43cac 100644
--- a/net/sched/act_connmark.c
+++ b/net/sched/act_connmark.c
@@ -231,7 +231,7 @@ static __net_init int connmark_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, connmark_net_id);
- return tc_action_net_init(tn, &act_connmark_ops);
+ return tc_action_net_init(net, tn, &act_connmark_ops);
}
static void __net_exit connmark_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_csum.c b/net/sched/act_csum.c
index 9b92882..d3cfad8 100644
--- a/net/sched/act_csum.c
+++ b/net/sched/act_csum.c
@@ -714,7 +714,7 @@ static __net_init int csum_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, csum_net_id);
- return tc_action_net_init(tn, &act_csum_ops);
+ return tc_action_net_init(net, tn, &act_csum_ops);
}
static void __net_exit csum_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c
index 33a1a74..cdd6f38 100644
--- a/net/sched/act_ct.c
+++ b/net/sched/act_ct.c
@@ -939,7 +939,7 @@ static __net_init int ct_init_net(struct net *net)
tn->labels = true;
}
- return tc_action_net_init(&tn->tn, &act_ct_ops);
+ return tc_action_net_init(net, &tn->tn, &act_ct_ops);
}
static void __net_exit ct_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_ctinfo.c b/net/sched/act_ctinfo.c
index 06ef74b..0dbcfd1 100644
--- a/net/sched/act_ctinfo.c
+++ b/net/sched/act_ctinfo.c
@@ -376,7 +376,7 @@ static __net_init int ctinfo_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, ctinfo_net_id);
- return tc_action_net_init(tn, &act_ctinfo_ops);
+ return tc_action_net_init(net, tn, &act_ctinfo_ops);
}
static void __net_exit ctinfo_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_gact.c b/net/sched/act_gact.c
index 8f0140c..324f1d1 100644
--- a/net/sched/act_gact.c
+++ b/net/sched/act_gact.c
@@ -278,7 +278,7 @@ static __net_init int gact_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, gact_net_id);
- return tc_action_net_init(tn, &act_gact_ops);
+ return tc_action_net_init(net, tn, &act_gact_ops);
}
static void __net_exit gact_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_ife.c b/net/sched/act_ife.c
index 92ee853..3a31e24 100644
--- a/net/sched/act_ife.c
+++ b/net/sched/act_ife.c
@@ -890,7 +890,7 @@ static __net_init int ife_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, ife_net_id);
- return tc_action_net_init(tn, &act_ife_ops);
+ return tc_action_net_init(net, tn, &act_ife_ops);
}
static void __net_exit ife_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_ipt.c b/net/sched/act_ipt.c
index ce2c30a..214a03d 100644
--- a/net/sched/act_ipt.c
+++ b/net/sched/act_ipt.c
@@ -61,12 +61,13 @@ static int ipt_init_target(struct net *net, struct xt_entry_target *t,
return 0;
}
-static void ipt_destroy_target(struct xt_entry_target *t)
+static void ipt_destroy_target(struct xt_entry_target *t, struct net *net)
{
struct xt_tgdtor_param par = {
.target = t->u.kernel.target,
.targinfo = t->data,
.family = NFPROTO_IPV4,
+ .net = net,
};
if (par.target->destroy != NULL)
par.target->destroy(&par);
@@ -78,7 +79,7 @@ static void tcf_ipt_release(struct tc_action *a)
struct tcf_ipt *ipt = to_ipt(a);
if (ipt->tcfi_t) {
- ipt_destroy_target(ipt->tcfi_t);
+ ipt_destroy_target(ipt->tcfi_t, a->idrinfo->net);
kfree(ipt->tcfi_t);
}
kfree(ipt->tcfi_tname);
@@ -180,7 +181,7 @@ static int __tcf_ipt_init(struct net *net, unsigned int id, struct nlattr *nla,
spin_lock_bh(&ipt->tcf_lock);
if (ret != ACT_P_CREATED) {
- ipt_destroy_target(ipt->tcfi_t);
+ ipt_destroy_target(ipt->tcfi_t, net);
kfree(ipt->tcfi_tname);
kfree(ipt->tcfi_t);
}
@@ -350,7 +351,7 @@ static __net_init int ipt_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, ipt_net_id);
- return tc_action_net_init(tn, &act_ipt_ops);
+ return tc_action_net_init(net, tn, &act_ipt_ops);
}
static void __net_exit ipt_exit_net(struct list_head *net_list)
@@ -399,7 +400,7 @@ static __net_init int xt_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, xt_net_id);
- return tc_action_net_init(tn, &act_xt_ops);
+ return tc_action_net_init(net, tn, &act_xt_ops);
}
static void __net_exit xt_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
index be3f88d..9d1bf50 100644
--- a/net/sched/act_mirred.c
+++ b/net/sched/act_mirred.c
@@ -453,7 +453,7 @@ static __net_init int mirred_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, mirred_net_id);
- return tc_action_net_init(tn, &act_mirred_ops);
+ return tc_action_net_init(net, tn, &act_mirred_ops);
}
static void __net_exit mirred_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_mpls.c b/net/sched/act_mpls.c
index 0f299e3..e168df0 100644
--- a/net/sched/act_mpls.c
+++ b/net/sched/act_mpls.c
@@ -375,7 +375,7 @@ static __net_init int mpls_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, mpls_net_id);
- return tc_action_net_init(tn, &act_mpls_ops);
+ return tc_action_net_init(net, tn, &act_mpls_ops);
}
static void __net_exit mpls_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_nat.c b/net/sched/act_nat.c
index 7b858c1..ea4c535 100644
--- a/net/sched/act_nat.c
+++ b/net/sched/act_nat.c
@@ -327,7 +327,7 @@ static __net_init int nat_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, nat_net_id);
- return tc_action_net_init(tn, &act_nat_ops);
+ return tc_action_net_init(net, tn, &act_nat_ops);
}
static void __net_exit nat_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_pedit.c b/net/sched/act_pedit.c
index 17360c6..cdfaa79 100644
--- a/net/sched/act_pedit.c
+++ b/net/sched/act_pedit.c
@@ -498,7 +498,7 @@ static __net_init int pedit_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, pedit_net_id);
- return tc_action_net_init(tn, &act_pedit_ops);
+ return tc_action_net_init(net, tn, &act_pedit_ops);
}
static void __net_exit pedit_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_police.c b/net/sched/act_police.c
index 49cec3e..89c04c5 100644
--- a/net/sched/act_police.c
+++ b/net/sched/act_police.c
@@ -40,6 +40,8 @@ static const struct nla_policy police_policy[TCA_POLICE_MAX + 1] = {
[TCA_POLICE_PEAKRATE] = { .len = TC_RTAB_SIZE },
[TCA_POLICE_AVRATE] = { .type = NLA_U32 },
[TCA_POLICE_RESULT] = { .type = NLA_U32 },
+ [TCA_POLICE_RATE64] = { .type = NLA_U64 },
+ [TCA_POLICE_PEAKRATE64] = { .type = NLA_U64 },
};
static int tcf_police_init(struct net *net, struct nlattr *nla,
@@ -58,6 +60,7 @@ static int tcf_police_init(struct net *net, struct nlattr *nla,
struct tcf_police_params *new;
bool exists = false;
u32 index;
+ u64 rate64, prate64;
if (nla == NULL)
return -EINVAL;
@@ -155,14 +158,18 @@ static int tcf_police_init(struct net *net, struct nlattr *nla,
}
if (R_tab) {
new->rate_present = true;
- psched_ratecfg_precompute(&new->rate, &R_tab->rate, 0);
+ rate64 = tb[TCA_POLICE_RATE64] ?
+ nla_get_u64(tb[TCA_POLICE_RATE64]) : 0;
+ psched_ratecfg_precompute(&new->rate, &R_tab->rate, rate64);
qdisc_put_rtab(R_tab);
} else {
new->rate_present = false;
}
if (P_tab) {
new->peak_present = true;
- psched_ratecfg_precompute(&new->peak, &P_tab->rate, 0);
+ prate64 = tb[TCA_POLICE_PEAKRATE64] ?
+ nla_get_u64(tb[TCA_POLICE_PEAKRATE64]) : 0;
+ psched_ratecfg_precompute(&new->peak, &P_tab->rate, prate64);
qdisc_put_rtab(P_tab);
} else {
new->peak_present = false;
@@ -313,10 +320,22 @@ static int tcf_police_dump(struct sk_buff *skb, struct tc_action *a,
lockdep_is_held(&police->tcf_lock));
opt.mtu = p->tcfp_mtu;
opt.burst = PSCHED_NS2TICKS(p->tcfp_burst);
- if (p->rate_present)
+ if (p->rate_present) {
psched_ratecfg_getrate(&opt.rate, &p->rate);
- if (p->peak_present)
+ if ((police->params->rate.rate_bytes_ps >= (1ULL << 32)) &&
+ nla_put_u64_64bit(skb, TCA_POLICE_RATE64,
+ police->params->rate.rate_bytes_ps,
+ TCA_POLICE_PAD))
+ goto nla_put_failure;
+ }
+ if (p->peak_present) {
psched_ratecfg_getrate(&opt.peakrate, &p->peak);
+ if ((police->params->peak.rate_bytes_ps >= (1ULL << 32)) &&
+ nla_put_u64_64bit(skb, TCA_POLICE_PEAKRATE64,
+ police->params->peak.rate_bytes_ps,
+ TCA_POLICE_PAD))
+ goto nla_put_failure;
+ }
if (nla_put(skb, TCA_POLICE_TBF, sizeof(opt), &opt))
goto nla_put_failure;
if (p->tcfp_result &&
@@ -371,7 +390,7 @@ static __net_init int police_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, police_net_id);
- return tc_action_net_init(tn, &act_police_ops);
+ return tc_action_net_init(net, tn, &act_police_ops);
}
static void __net_exit police_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_sample.c b/net/sched/act_sample.c
index 595308d..1022912 100644
--- a/net/sched/act_sample.c
+++ b/net/sched/act_sample.c
@@ -102,13 +102,17 @@ static int tcf_sample_init(struct net *net, struct nlattr *nla,
goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch);
s->rate = rate;
s->psample_group_num = psample_group_num;
- RCU_INIT_POINTER(s->psample_group, psample_group);
+ rcu_swap_protected(s->psample_group, psample_group,
+ lockdep_is_held(&s->tcf_lock));
if (tb[TCA_SAMPLE_TRUNC_SIZE]) {
s->truncate = true;
s->trunc_size = nla_get_u32(tb[TCA_SAMPLE_TRUNC_SIZE]);
}
spin_unlock_bh(&s->tcf_lock);
+
+ if (psample_group)
+ psample_group_put(psample_group);
if (goto_ch)
tcf_chain_put_by_act(goto_ch);
@@ -265,7 +269,7 @@ static __net_init int sample_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, sample_net_id);
- return tc_action_net_init(tn, &act_sample_ops);
+ return tc_action_net_init(net, tn, &act_sample_ops);
}
static void __net_exit sample_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_simple.c b/net/sched/act_simple.c
index 33aefa2..6120e56 100644
--- a/net/sched/act_simple.c
+++ b/net/sched/act_simple.c
@@ -232,7 +232,7 @@ static __net_init int simp_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, simp_net_id);
- return tc_action_net_init(tn, &act_simp_ops);
+ return tc_action_net_init(net, tn, &act_simp_ops);
}
static void __net_exit simp_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_skbedit.c b/net/sched/act_skbedit.c
index 37dced0..6a8d333 100644
--- a/net/sched/act_skbedit.c
+++ b/net/sched/act_skbedit.c
@@ -336,7 +336,7 @@ static __net_init int skbedit_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, skbedit_net_id);
- return tc_action_net_init(tn, &act_skbedit_ops);
+ return tc_action_net_init(net, tn, &act_skbedit_ops);
}
static void __net_exit skbedit_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_skbmod.c b/net/sched/act_skbmod.c
index 7da3518..888437f97b 100644
--- a/net/sched/act_skbmod.c
+++ b/net/sched/act_skbmod.c
@@ -287,7 +287,7 @@ static __net_init int skbmod_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, skbmod_net_id);
- return tc_action_net_init(tn, &act_skbmod_ops);
+ return tc_action_net_init(net, tn, &act_skbmod_ops);
}
static void __net_exit skbmod_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c
index 6d0debd..2f83a79 100644
--- a/net/sched/act_tunnel_key.c
+++ b/net/sched/act_tunnel_key.c
@@ -600,7 +600,7 @@ static __net_init int tunnel_key_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, tunnel_key_net_id);
- return tc_action_net_init(tn, &act_tunnel_key_ops);
+ return tc_action_net_init(net, tn, &act_tunnel_key_ops);
}
static void __net_exit tunnel_key_exit_net(struct list_head *net_list)
diff --git a/net/sched/act_vlan.c b/net/sched/act_vlan.c
index a3c9eea..08aaf71 100644
--- a/net/sched/act_vlan.c
+++ b/net/sched/act_vlan.c
@@ -301,6 +301,19 @@ static int tcf_vlan_walker(struct net *net, struct sk_buff *skb,
return tcf_generic_walker(tn, skb, cb, type, ops, extack);
}
+static void tcf_vlan_stats_update(struct tc_action *a, u64 bytes, u32 packets,
+ u64 lastuse, bool hw)
+{
+ struct tcf_vlan *v = to_vlan(a);
+ struct tcf_t *tm = &v->tcf_tm;
+
+ _bstats_cpu_update(this_cpu_ptr(a->cpu_bstats), bytes, packets);
+ if (hw)
+ _bstats_cpu_update(this_cpu_ptr(a->cpu_bstats_hw),
+ bytes, packets);
+ tm->lastuse = max_t(u64, tm->lastuse, lastuse);
+}
+
static int tcf_vlan_search(struct net *net, struct tc_action **a, u32 index)
{
struct tc_action_net *tn = net_generic(net, vlan_net_id);
@@ -325,6 +338,7 @@ static struct tc_action_ops act_vlan_ops = {
.init = tcf_vlan_init,
.cleanup = tcf_vlan_cleanup,
.walk = tcf_vlan_walker,
+ .stats_update = tcf_vlan_stats_update,
.get_fill_size = tcf_vlan_get_fill_size,
.lookup = tcf_vlan_search,
.size = sizeof(struct tcf_vlan),
@@ -334,7 +348,7 @@ static __net_init int vlan_init_net(struct net *net)
{
struct tc_action_net *tn = net_generic(net, vlan_net_id);
- return tc_action_net_init(tn, &act_vlan_ops);
+ return tc_action_net_init(net, tn, &act_vlan_ops);
}
static void __net_exit vlan_exit_net(struct list_head *net_list)
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index e0d8b45..05c4fe1 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -568,9 +568,11 @@ static void tc_indr_block_ing_cmd(struct net_device *dev,
bo.block = &block->flow_block;
+ down_write(&block->cb_lock);
cb(dev, cb_priv, TC_SETUP_BLOCK, &bo);
tcf_block_setup(block, &bo);
+ up_write(&block->cb_lock);
}
static struct tcf_block *tc_dev_ingress_block(struct net_device *dev)
@@ -627,7 +629,7 @@ static void tc_indr_block_call(struct tcf_block *block,
static bool tcf_block_offload_in_use(struct tcf_block *block)
{
- return block->offloadcnt;
+ return atomic_read(&block->offloadcnt);
}
static int tcf_block_offload_cmd(struct tcf_block *block,
@@ -661,6 +663,7 @@ static int tcf_block_offload_bind(struct tcf_block *block, struct Qdisc *q,
struct net_device *dev = q->dev_queue->dev;
int err;
+ down_write(&block->cb_lock);
if (!dev->netdev_ops->ndo_setup_tc)
goto no_offload_dev_inc;
@@ -669,24 +672,31 @@ static int tcf_block_offload_bind(struct tcf_block *block, struct Qdisc *q,
*/
if (!tc_can_offload(dev) && tcf_block_offload_in_use(block)) {
NL_SET_ERR_MSG(extack, "Bind to offloaded block failed as dev has offload disabled");
- return -EOPNOTSUPP;
+ err = -EOPNOTSUPP;
+ goto err_unlock;
}
err = tcf_block_offload_cmd(block, dev, ei, FLOW_BLOCK_BIND, extack);
if (err == -EOPNOTSUPP)
goto no_offload_dev_inc;
if (err)
- return err;
+ goto err_unlock;
tc_indr_block_call(block, dev, ei, FLOW_BLOCK_BIND, extack);
+ up_write(&block->cb_lock);
return 0;
no_offload_dev_inc:
- if (tcf_block_offload_in_use(block))
- return -EOPNOTSUPP;
+ if (tcf_block_offload_in_use(block)) {
+ err = -EOPNOTSUPP;
+ goto err_unlock;
+ }
+ err = 0;
block->nooffloaddevcnt++;
tc_indr_block_call(block, dev, ei, FLOW_BLOCK_BIND, extack);
- return 0;
+err_unlock:
+ up_write(&block->cb_lock);
+ return err;
}
static void tcf_block_offload_unbind(struct tcf_block *block, struct Qdisc *q,
@@ -695,6 +705,7 @@ static void tcf_block_offload_unbind(struct tcf_block *block, struct Qdisc *q,
struct net_device *dev = q->dev_queue->dev;
int err;
+ down_write(&block->cb_lock);
tc_indr_block_call(block, dev, ei, FLOW_BLOCK_UNBIND, NULL);
if (!dev->netdev_ops->ndo_setup_tc)
@@ -702,10 +713,12 @@ static void tcf_block_offload_unbind(struct tcf_block *block, struct Qdisc *q,
err = tcf_block_offload_cmd(block, dev, ei, FLOW_BLOCK_UNBIND, NULL);
if (err == -EOPNOTSUPP)
goto no_offload_dev_dec;
+ up_write(&block->cb_lock);
return;
no_offload_dev_dec:
WARN_ON(block->nooffloaddevcnt-- == 0);
+ up_write(&block->cb_lock);
}
static int
@@ -820,6 +833,7 @@ static struct tcf_block *tcf_block_create(struct net *net, struct Qdisc *q,
return ERR_PTR(-ENOMEM);
}
mutex_init(&block->lock);
+ init_rwsem(&block->cb_lock);
flow_block_init(&block->flow_block);
INIT_LIST_HEAD(&block->chain_list);
INIT_LIST_HEAD(&block->owner_list);
@@ -1355,6 +1369,8 @@ tcf_block_playback_offloads(struct tcf_block *block, flow_setup_cb_t *cb,
struct tcf_proto *tp, *tp_prev;
int err;
+ lockdep_assert_held(&block->cb_lock);
+
for (chain = __tcf_get_next_chain(block, NULL);
chain;
chain_prev = chain,
@@ -1393,6 +1409,8 @@ static int tcf_block_bind(struct tcf_block *block,
struct flow_block_cb *block_cb, *next;
int err, i = 0;
+ lockdep_assert_held(&block->cb_lock);
+
list_for_each_entry(block_cb, &bo->cb_list, list) {
err = tcf_block_playback_offloads(block, block_cb->cb,
block_cb->cb_priv, true,
@@ -1400,6 +1418,8 @@ static int tcf_block_bind(struct tcf_block *block,
bo->extack);
if (err)
goto err_unroll;
+ if (!bo->unlocked_driver_cb)
+ block->lockeddevcnt++;
i++;
}
@@ -1415,6 +1435,8 @@ static int tcf_block_bind(struct tcf_block *block,
block_cb->cb_priv, false,
tcf_block_offload_in_use(block),
NULL);
+ if (!bo->unlocked_driver_cb)
+ block->lockeddevcnt--;
}
flow_block_cb_free(block_cb);
}
@@ -1427,6 +1449,8 @@ static void tcf_block_unbind(struct tcf_block *block,
{
struct flow_block_cb *block_cb, *next;
+ lockdep_assert_held(&block->cb_lock);
+
list_for_each_entry_safe(block_cb, next, &bo->cb_list, list) {
tcf_block_playback_offloads(block, block_cb->cb,
block_cb->cb_priv, false,
@@ -1434,6 +1458,8 @@ static void tcf_block_unbind(struct tcf_block *block,
NULL);
list_del(&block_cb->list);
flow_block_cb_free(block_cb);
+ if (!bo->unlocked_driver_cb)
+ block->lockeddevcnt--;
}
}
@@ -1488,6 +1514,18 @@ int tcf_classify(struct sk_buff *skb, const struct tcf_proto *tp,
goto reset;
} else if (unlikely(TC_ACT_EXT_CMP(err, TC_ACT_GOTO_CHAIN))) {
first_tp = res->goto_tp;
+
+#if IS_ENABLED(CONFIG_NET_TC_SKB_EXT)
+ {
+ struct tc_skb_ext *ext;
+
+ ext = skb_ext_add(skb, TC_SKB_EXT);
+ if (WARN_ON_ONCE(!ext))
+ return TC_ACT_SHOT;
+
+ ext->chain = err & TC_ACT_EXT_VAL_MASK;
+ }
+#endif
goto reset;
}
#endif
@@ -2980,17 +3018,61 @@ int tcf_exts_dump_stats(struct sk_buff *skb, struct tcf_exts *exts)
}
EXPORT_SYMBOL(tcf_exts_dump_stats);
-int tc_setup_cb_call(struct tcf_block *block, enum tc_setup_type type,
- void *type_data, bool err_stop)
+static void tcf_block_offload_inc(struct tcf_block *block, u32 *flags)
+{
+ if (*flags & TCA_CLS_FLAGS_IN_HW)
+ return;
+ *flags |= TCA_CLS_FLAGS_IN_HW;
+ atomic_inc(&block->offloadcnt);
+}
+
+static void tcf_block_offload_dec(struct tcf_block *block, u32 *flags)
+{
+ if (!(*flags & TCA_CLS_FLAGS_IN_HW))
+ return;
+ *flags &= ~TCA_CLS_FLAGS_IN_HW;
+ atomic_dec(&block->offloadcnt);
+}
+
+static void tc_cls_offload_cnt_update(struct tcf_block *block,
+ struct tcf_proto *tp, u32 *cnt,
+ u32 *flags, u32 diff, bool add)
+{
+ lockdep_assert_held(&block->cb_lock);
+
+ spin_lock(&tp->lock);
+ if (add) {
+ if (!*cnt)
+ tcf_block_offload_inc(block, flags);
+ *cnt += diff;
+ } else {
+ *cnt -= diff;
+ if (!*cnt)
+ tcf_block_offload_dec(block, flags);
+ }
+ spin_unlock(&tp->lock);
+}
+
+static void
+tc_cls_offload_cnt_reset(struct tcf_block *block, struct tcf_proto *tp,
+ u32 *cnt, u32 *flags)
+{
+ lockdep_assert_held(&block->cb_lock);
+
+ spin_lock(&tp->lock);
+ tcf_block_offload_dec(block, flags);
+ *cnt = 0;
+ spin_unlock(&tp->lock);
+}
+
+static int
+__tc_setup_cb_call(struct tcf_block *block, enum tc_setup_type type,
+ void *type_data, bool err_stop)
{
struct flow_block_cb *block_cb;
int ok_count = 0;
int err;
- /* Make sure all netdevs sharing this block are offload-capable. */
- if (block->nooffloaddevcnt && err_stop)
- return -EOPNOTSUPP;
-
list_for_each_entry(block_cb, &block->flow_block.cb_list, list) {
err = block_cb->cb(type, type_data, block_cb->cb_priv);
if (err) {
@@ -3002,17 +3084,235 @@ int tc_setup_cb_call(struct tcf_block *block, enum tc_setup_type type,
}
return ok_count;
}
+
+int tc_setup_cb_call(struct tcf_block *block, enum tc_setup_type type,
+ void *type_data, bool err_stop, bool rtnl_held)
+{
+ bool take_rtnl = READ_ONCE(block->lockeddevcnt) && !rtnl_held;
+ int ok_count;
+
+retry:
+ if (take_rtnl)
+ rtnl_lock();
+ down_read(&block->cb_lock);
+ /* Need to obtain rtnl lock if block is bound to devs that require it.
+ * In block bind code cb_lock is obtained while holding rtnl, so we must
+ * obtain the locks in same order here.
+ */
+ if (!rtnl_held && !take_rtnl && block->lockeddevcnt) {
+ up_read(&block->cb_lock);
+ take_rtnl = true;
+ goto retry;
+ }
+
+ ok_count = __tc_setup_cb_call(block, type, type_data, err_stop);
+
+ up_read(&block->cb_lock);
+ if (take_rtnl)
+ rtnl_unlock();
+ return ok_count;
+}
EXPORT_SYMBOL(tc_setup_cb_call);
+/* Non-destructive filter add. If filter that wasn't already in hardware is
+ * successfully offloaded, increment block offloads counter. On failure,
+ * previously offloaded filter is considered to be intact and offloads counter
+ * is not decremented.
+ */
+
+int tc_setup_cb_add(struct tcf_block *block, struct tcf_proto *tp,
+ enum tc_setup_type type, void *type_data, bool err_stop,
+ u32 *flags, unsigned int *in_hw_count, bool rtnl_held)
+{
+ bool take_rtnl = READ_ONCE(block->lockeddevcnt) && !rtnl_held;
+ int ok_count;
+
+retry:
+ if (take_rtnl)
+ rtnl_lock();
+ down_read(&block->cb_lock);
+ /* Need to obtain rtnl lock if block is bound to devs that require it.
+ * In block bind code cb_lock is obtained while holding rtnl, so we must
+ * obtain the locks in same order here.
+ */
+ if (!rtnl_held && !take_rtnl && block->lockeddevcnt) {
+ up_read(&block->cb_lock);
+ take_rtnl = true;
+ goto retry;
+ }
+
+ /* Make sure all netdevs sharing this block are offload-capable. */
+ if (block->nooffloaddevcnt && err_stop) {
+ ok_count = -EOPNOTSUPP;
+ goto err_unlock;
+ }
+
+ ok_count = __tc_setup_cb_call(block, type, type_data, err_stop);
+ if (ok_count < 0)
+ goto err_unlock;
+
+ if (tp->ops->hw_add)
+ tp->ops->hw_add(tp, type_data);
+ if (ok_count > 0)
+ tc_cls_offload_cnt_update(block, tp, in_hw_count, flags,
+ ok_count, true);
+err_unlock:
+ up_read(&block->cb_lock);
+ if (take_rtnl)
+ rtnl_unlock();
+ return ok_count < 0 ? ok_count : 0;
+}
+EXPORT_SYMBOL(tc_setup_cb_add);
+
+/* Destructive filter replace. If filter that wasn't already in hardware is
+ * successfully offloaded, increment block offload counter. On failure,
+ * previously offloaded filter is considered to be destroyed and offload counter
+ * is decremented.
+ */
+
+int tc_setup_cb_replace(struct tcf_block *block, struct tcf_proto *tp,
+ enum tc_setup_type type, void *type_data, bool err_stop,
+ u32 *old_flags, unsigned int *old_in_hw_count,
+ u32 *new_flags, unsigned int *new_in_hw_count,
+ bool rtnl_held)
+{
+ bool take_rtnl = READ_ONCE(block->lockeddevcnt) && !rtnl_held;
+ int ok_count;
+
+retry:
+ if (take_rtnl)
+ rtnl_lock();
+ down_read(&block->cb_lock);
+ /* Need to obtain rtnl lock if block is bound to devs that require it.
+ * In block bind code cb_lock is obtained while holding rtnl, so we must
+ * obtain the locks in same order here.
+ */
+ if (!rtnl_held && !take_rtnl && block->lockeddevcnt) {
+ up_read(&block->cb_lock);
+ take_rtnl = true;
+ goto retry;
+ }
+
+ /* Make sure all netdevs sharing this block are offload-capable. */
+ if (block->nooffloaddevcnt && err_stop) {
+ ok_count = -EOPNOTSUPP;
+ goto err_unlock;
+ }
+
+ tc_cls_offload_cnt_reset(block, tp, old_in_hw_count, old_flags);
+ if (tp->ops->hw_del)
+ tp->ops->hw_del(tp, type_data);
+
+ ok_count = __tc_setup_cb_call(block, type, type_data, err_stop);
+ if (ok_count < 0)
+ goto err_unlock;
+
+ if (tp->ops->hw_add)
+ tp->ops->hw_add(tp, type_data);
+ if (ok_count > 0)
+ tc_cls_offload_cnt_update(block, tp, new_in_hw_count,
+ new_flags, ok_count, true);
+err_unlock:
+ up_read(&block->cb_lock);
+ if (take_rtnl)
+ rtnl_unlock();
+ return ok_count < 0 ? ok_count : 0;
+}
+EXPORT_SYMBOL(tc_setup_cb_replace);
+
+/* Destroy filter and decrement block offload counter, if filter was previously
+ * offloaded.
+ */
+
+int tc_setup_cb_destroy(struct tcf_block *block, struct tcf_proto *tp,
+ enum tc_setup_type type, void *type_data, bool err_stop,
+ u32 *flags, unsigned int *in_hw_count, bool rtnl_held)
+{
+ bool take_rtnl = READ_ONCE(block->lockeddevcnt) && !rtnl_held;
+ int ok_count;
+
+retry:
+ if (take_rtnl)
+ rtnl_lock();
+ down_read(&block->cb_lock);
+ /* Need to obtain rtnl lock if block is bound to devs that require it.
+ * In block bind code cb_lock is obtained while holding rtnl, so we must
+ * obtain the locks in same order here.
+ */
+ if (!rtnl_held && !take_rtnl && block->lockeddevcnt) {
+ up_read(&block->cb_lock);
+ take_rtnl = true;
+ goto retry;
+ }
+
+ ok_count = __tc_setup_cb_call(block, type, type_data, err_stop);
+
+ tc_cls_offload_cnt_reset(block, tp, in_hw_count, flags);
+ if (tp->ops->hw_del)
+ tp->ops->hw_del(tp, type_data);
+
+ up_read(&block->cb_lock);
+ if (take_rtnl)
+ rtnl_unlock();
+ return ok_count < 0 ? ok_count : 0;
+}
+EXPORT_SYMBOL(tc_setup_cb_destroy);
+
+int tc_setup_cb_reoffload(struct tcf_block *block, struct tcf_proto *tp,
+ bool add, flow_setup_cb_t *cb,
+ enum tc_setup_type type, void *type_data,
+ void *cb_priv, u32 *flags, unsigned int *in_hw_count)
+{
+ int err = cb(type, type_data, cb_priv);
+
+ if (err) {
+ if (add && tc_skip_sw(*flags))
+ return err;
+ } else {
+ tc_cls_offload_cnt_update(block, tp, in_hw_count, flags, 1,
+ add);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(tc_setup_cb_reoffload);
+
+void tc_cleanup_flow_action(struct flow_action *flow_action)
+{
+ struct flow_action_entry *entry;
+ int i;
+
+ flow_action_for_each(i, entry, flow_action) {
+ switch (entry->id) {
+ case FLOW_ACTION_REDIRECT:
+ case FLOW_ACTION_MIRRED:
+ case FLOW_ACTION_REDIRECT_INGRESS:
+ case FLOW_ACTION_MIRRED_INGRESS:
+ if (entry->dev)
+ dev_put(entry->dev);
+ break;
+ case FLOW_ACTION_TUNNEL_ENCAP:
+ kfree(entry->tunnel);
+ break;
+ default:
+ break;
+ }
+ }
+}
+EXPORT_SYMBOL(tc_cleanup_flow_action);
+
int tc_setup_flow_action(struct flow_action *flow_action,
- const struct tcf_exts *exts)
+ const struct tcf_exts *exts, bool rtnl_held)
{
const struct tc_action *act;
- int i, j, k;
+ int i, j, k, err = 0;
if (!exts)
return 0;
+ if (!rtnl_held)
+ rtnl_lock();
+
j = 0;
tcf_exts_for_each_action(i, act, exts) {
struct flow_action_entry *entry;
@@ -3030,15 +3330,23 @@ int tc_setup_flow_action(struct flow_action *flow_action,
} else if (is_tcf_mirred_egress_redirect(act)) {
entry->id = FLOW_ACTION_REDIRECT;
entry->dev = tcf_mirred_dev(act);
+ if (entry->dev)
+ dev_hold(entry->dev);
} else if (is_tcf_mirred_egress_mirror(act)) {
entry->id = FLOW_ACTION_MIRRED;
entry->dev = tcf_mirred_dev(act);
+ if (entry->dev)
+ dev_hold(entry->dev);
} else if (is_tcf_mirred_ingress_redirect(act)) {
entry->id = FLOW_ACTION_REDIRECT_INGRESS;
entry->dev = tcf_mirred_dev(act);
+ if (entry->dev)
+ dev_hold(entry->dev);
} else if (is_tcf_mirred_ingress_mirror(act)) {
entry->id = FLOW_ACTION_MIRRED_INGRESS;
entry->dev = tcf_mirred_dev(act);
+ if (entry->dev)
+ dev_hold(entry->dev);
} else if (is_tcf_vlan(act)) {
switch (tcf_vlan_action(act)) {
case TCA_VLAN_ACT_PUSH:
@@ -3057,11 +3365,16 @@ int tc_setup_flow_action(struct flow_action *flow_action,
entry->vlan.prio = tcf_vlan_push_prio(act);
break;
default:
+ err = -EOPNOTSUPP;
goto err_out;
}
} else if (is_tcf_tunnel_set(act)) {
entry->id = FLOW_ACTION_TUNNEL_ENCAP;
- entry->tunnel = tcf_tunnel_info(act);
+ entry->tunnel = tcf_tunnel_info_copy(act);
+ if (!entry->tunnel) {
+ err = -ENOMEM;
+ goto err_out;
+ }
} else if (is_tcf_tunnel_release(act)) {
entry->id = FLOW_ACTION_TUNNEL_DECAP;
} else if (is_tcf_pedit(act)) {
@@ -3074,6 +3387,7 @@ int tc_setup_flow_action(struct flow_action *flow_action,
entry->id = FLOW_ACTION_ADD;
break;
default:
+ err = -EOPNOTSUPP;
goto err_out;
}
entry->mangle.htype = tcf_pedit_htype(act, k);
@@ -3132,15 +3446,22 @@ int tc_setup_flow_action(struct flow_action *flow_action,
entry->id = FLOW_ACTION_PTYPE;
entry->ptype = tcf_skbedit_ptype(act);
} else {
+ err = -EOPNOTSUPP;
goto err_out;
}
if (!is_tcf_pedit(act))
j++;
}
- return 0;
+
err_out:
- return -EOPNOTSUPP;
+ if (!rtnl_held)
+ rtnl_unlock();
+
+ if (err)
+ tc_cleanup_flow_action(flow_action);
+
+ return err;
}
EXPORT_SYMBOL(tc_setup_flow_action);
diff --git a/net/sched/cls_bpf.c b/net/sched/cls_bpf.c
index 3f7a9c0..bf10bda 100644
--- a/net/sched/cls_bpf.c
+++ b/net/sched/cls_bpf.c
@@ -163,17 +163,19 @@ static int cls_bpf_offload_cmd(struct tcf_proto *tp, struct cls_bpf_prog *prog,
cls_bpf.exts_integrated = obj->exts_integrated;
if (oldprog)
- tcf_block_offload_dec(block, &oldprog->gen_flags);
+ err = tc_setup_cb_replace(block, tp, TC_SETUP_CLSBPF, &cls_bpf,
+ skip_sw, &oldprog->gen_flags,
+ &oldprog->in_hw_count,
+ &prog->gen_flags, &prog->in_hw_count,
+ true);
+ else
+ err = tc_setup_cb_add(block, tp, TC_SETUP_CLSBPF, &cls_bpf,
+ skip_sw, &prog->gen_flags,
+ &prog->in_hw_count, true);
- err = tc_setup_cb_call(block, TC_SETUP_CLSBPF, &cls_bpf, skip_sw);
- if (prog) {
- if (err < 0) {
- cls_bpf_offload_cmd(tp, oldprog, prog, extack);
- return err;
- } else if (err > 0) {
- prog->in_hw_count = err;
- tcf_block_offload_inc(block, &prog->gen_flags);
- }
+ if (prog && err) {
+ cls_bpf_offload_cmd(tp, oldprog, prog, extack);
+ return err;
}
if (prog && skip_sw && !(prog->gen_flags & TCA_CLS_FLAGS_IN_HW))
@@ -230,7 +232,7 @@ static void cls_bpf_offload_update_stats(struct tcf_proto *tp,
cls_bpf.name = prog->bpf_name;
cls_bpf.exts_integrated = prog->exts_integrated;
- tc_setup_cb_call(block, TC_SETUP_CLSBPF, &cls_bpf, false);
+ tc_setup_cb_call(block, TC_SETUP_CLSBPF, &cls_bpf, false, true);
}
static int cls_bpf_init(struct tcf_proto *tp)
@@ -673,15 +675,11 @@ static int cls_bpf_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb
cls_bpf.name = prog->bpf_name;
cls_bpf.exts_integrated = prog->exts_integrated;
- err = cb(TC_SETUP_CLSBPF, &cls_bpf, cb_priv);
- if (err) {
- if (add && tc_skip_sw(prog->gen_flags))
- return err;
- continue;
- }
-
- tc_cls_offload_cnt_update(block, &prog->in_hw_count,
- &prog->gen_flags, add);
+ err = tc_setup_cb_reoffload(block, tp, add, cb, TC_SETUP_CLSBPF,
+ &cls_bpf, cb_priv, &prog->gen_flags,
+ &prog->in_hw_count);
+ if (err)
+ return err;
}
return 0;
diff --git a/net/sched/cls_flower.c b/net/sched/cls_flower.c
index 0541237..74221e3 100644
--- a/net/sched/cls_flower.c
+++ b/net/sched/cls_flower.c
@@ -412,41 +412,27 @@ static void fl_hw_destroy_filter(struct tcf_proto *tp, struct cls_fl_filter *f,
struct tcf_block *block = tp->chain->block;
struct flow_cls_offload cls_flower = {};
- if (!rtnl_held)
- rtnl_lock();
-
tc_cls_common_offload_init(&cls_flower.common, tp, f->flags, extack);
cls_flower.command = FLOW_CLS_DESTROY;
cls_flower.cookie = (unsigned long) f;
- tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false);
- spin_lock(&tp->lock);
- list_del_init(&f->hw_list);
- tcf_block_offload_dec(block, &f->flags);
- spin_unlock(&tp->lock);
+ tc_setup_cb_destroy(block, tp, TC_SETUP_CLSFLOWER, &cls_flower, false,
+ &f->flags, &f->in_hw_count, rtnl_held);
- if (!rtnl_held)
- rtnl_unlock();
}
static int fl_hw_replace_filter(struct tcf_proto *tp,
struct cls_fl_filter *f, bool rtnl_held,
struct netlink_ext_ack *extack)
{
- struct cls_fl_head *head = fl_head_dereference(tp);
struct tcf_block *block = tp->chain->block;
struct flow_cls_offload cls_flower = {};
bool skip_sw = tc_skip_sw(f->flags);
int err = 0;
- if (!rtnl_held)
- rtnl_lock();
-
cls_flower.rule = flow_rule_alloc(tcf_exts_num_actions(&f->exts));
- if (!cls_flower.rule) {
- err = -ENOMEM;
- goto errout;
- }
+ if (!cls_flower.rule)
+ return -ENOMEM;
tc_cls_common_offload_init(&cls_flower.common, tp, f->flags, extack);
cls_flower.command = FLOW_CLS_REPLACE;
@@ -456,43 +442,31 @@ static int fl_hw_replace_filter(struct tcf_proto *tp,
cls_flower.rule->match.key = &f->mkey;
cls_flower.classid = f->res.classid;
- err = tc_setup_flow_action(&cls_flower.rule->action, &f->exts);
+ err = tc_setup_flow_action(&cls_flower.rule->action, &f->exts,
+ rtnl_held);
if (err) {
kfree(cls_flower.rule);
- if (skip_sw)
+ if (skip_sw) {
NL_SET_ERR_MSG_MOD(extack, "Failed to setup flow action");
- else
- err = 0;
- goto errout;
+ return err;
+ }
+ return 0;
}
- err = tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, skip_sw);
+ err = tc_setup_cb_add(block, tp, TC_SETUP_CLSFLOWER, &cls_flower,
+ skip_sw, &f->flags, &f->in_hw_count, rtnl_held);
+ tc_cleanup_flow_action(&cls_flower.rule->action);
kfree(cls_flower.rule);
- if (err < 0) {
- fl_hw_destroy_filter(tp, f, true, NULL);
- goto errout;
- } else if (err > 0) {
- f->in_hw_count = err;
- err = 0;
- spin_lock(&tp->lock);
- tcf_block_offload_inc(block, &f->flags);
- spin_unlock(&tp->lock);
+ if (err) {
+ fl_hw_destroy_filter(tp, f, rtnl_held, NULL);
+ return err;
}
- if (skip_sw && !(f->flags & TCA_CLS_FLAGS_IN_HW)) {
- err = -EINVAL;
- goto errout;
- }
+ if (skip_sw && !(f->flags & TCA_CLS_FLAGS_IN_HW))
+ return -EINVAL;
- spin_lock(&tp->lock);
- list_add(&f->hw_list, &head->hw_filters);
- spin_unlock(&tp->lock);
-errout:
- if (!rtnl_held)
- rtnl_unlock();
-
- return err;
+ return 0;
}
static void fl_hw_update_stats(struct tcf_proto *tp, struct cls_fl_filter *f,
@@ -501,22 +475,17 @@ static void fl_hw_update_stats(struct tcf_proto *tp, struct cls_fl_filter *f,
struct tcf_block *block = tp->chain->block;
struct flow_cls_offload cls_flower = {};
- if (!rtnl_held)
- rtnl_lock();
-
tc_cls_common_offload_init(&cls_flower.common, tp, f->flags, NULL);
cls_flower.command = FLOW_CLS_STATS;
cls_flower.cookie = (unsigned long) f;
cls_flower.classid = f->res.classid;
- tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false);
+ tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false,
+ rtnl_held);
tcf_exts_stats_update(&f->exts, cls_flower.stats.bytes,
cls_flower.stats.pkts,
cls_flower.stats.lastused);
-
- if (!rtnl_held)
- rtnl_unlock();
}
static void __fl_put(struct cls_fl_filter *f)
@@ -1831,7 +1800,8 @@ static int fl_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
cls_flower.rule->match.mask = &f->mask->key;
cls_flower.rule->match.key = &f->mkey;
- err = tc_setup_flow_action(&cls_flower.rule->action, &f->exts);
+ err = tc_setup_flow_action(&cls_flower.rule->action, &f->exts,
+ true);
if (err) {
kfree(cls_flower.rule);
if (tc_skip_sw(f->flags)) {
@@ -1844,21 +1814,17 @@ static int fl_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
cls_flower.classid = f->res.classid;
- err = cb(TC_SETUP_CLSFLOWER, &cls_flower, cb_priv);
+ err = tc_setup_cb_reoffload(block, tp, add, cb,
+ TC_SETUP_CLSFLOWER, &cls_flower,
+ cb_priv, &f->flags,
+ &f->in_hw_count);
+ tc_cleanup_flow_action(&cls_flower.rule->action);
kfree(cls_flower.rule);
if (err) {
- if (add && tc_skip_sw(f->flags)) {
- __fl_put(f);
- return err;
- }
- goto next_flow;
+ __fl_put(f);
+ return err;
}
-
- spin_lock(&tp->lock);
- tc_cls_offload_cnt_update(block, &f->in_hw_count, &f->flags,
- add);
- spin_unlock(&tp->lock);
next_flow:
__fl_put(f);
}
@@ -1866,6 +1832,30 @@ static int fl_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
return 0;
}
+static void fl_hw_add(struct tcf_proto *tp, void *type_data)
+{
+ struct flow_cls_offload *cls_flower = type_data;
+ struct cls_fl_filter *f =
+ (struct cls_fl_filter *) cls_flower->cookie;
+ struct cls_fl_head *head = fl_head_dereference(tp);
+
+ spin_lock(&tp->lock);
+ list_add(&f->hw_list, &head->hw_filters);
+ spin_unlock(&tp->lock);
+}
+
+static void fl_hw_del(struct tcf_proto *tp, void *type_data)
+{
+ struct flow_cls_offload *cls_flower = type_data;
+ struct cls_fl_filter *f =
+ (struct cls_fl_filter *) cls_flower->cookie;
+
+ spin_lock(&tp->lock);
+ if (!list_empty(&f->hw_list))
+ list_del_init(&f->hw_list);
+ spin_unlock(&tp->lock);
+}
+
static int fl_hw_create_tmplt(struct tcf_chain *chain,
struct fl_flow_tmplt *tmplt)
{
@@ -1886,7 +1876,7 @@ static int fl_hw_create_tmplt(struct tcf_chain *chain,
/* We don't care if driver (any of them) fails to handle this
* call. It serves just as a hint for it.
*/
- tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false);
+ tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false, true);
kfree(cls_flower.rule);
return 0;
@@ -1902,7 +1892,7 @@ static void fl_hw_destroy_tmplt(struct tcf_chain *chain,
cls_flower.command = FLOW_CLS_TMPLT_DESTROY;
cls_flower.cookie = (unsigned long) tmplt;
- tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false);
+ tc_setup_cb_call(block, TC_SETUP_CLSFLOWER, &cls_flower, false, true);
}
static void *fl_tmplt_create(struct net *net, struct tcf_chain *chain,
@@ -2526,6 +2516,8 @@ static struct tcf_proto_ops cls_fl_ops __read_mostly = {
.delete = fl_delete,
.walk = fl_walk,
.reoffload = fl_reoffload,
+ .hw_add = fl_hw_add,
+ .hw_del = fl_hw_del,
.dump = fl_dump,
.bind_class = fl_bind_class,
.tmplt_create = fl_tmplt_create,
diff --git a/net/sched/cls_matchall.c b/net/sched/cls_matchall.c
index 455ea27..7fc2eb6 100644
--- a/net/sched/cls_matchall.c
+++ b/net/sched/cls_matchall.c
@@ -75,8 +75,8 @@ static void mall_destroy_hw_filter(struct tcf_proto *tp,
cls_mall.command = TC_CLSMATCHALL_DESTROY;
cls_mall.cookie = cookie;
- tc_setup_cb_call(block, TC_SETUP_CLSMATCHALL, &cls_mall, false);
- tcf_block_offload_dec(block, &head->flags);
+ tc_setup_cb_destroy(block, tp, TC_SETUP_CLSMATCHALL, &cls_mall, false,
+ &head->flags, &head->in_hw_count, true);
}
static int mall_replace_hw_filter(struct tcf_proto *tp,
@@ -97,7 +97,7 @@ static int mall_replace_hw_filter(struct tcf_proto *tp,
cls_mall.command = TC_CLSMATCHALL_REPLACE;
cls_mall.cookie = cookie;
- err = tc_setup_flow_action(&cls_mall.rule->action, &head->exts);
+ err = tc_setup_flow_action(&cls_mall.rule->action, &head->exts, true);
if (err) {
kfree(cls_mall.rule);
mall_destroy_hw_filter(tp, head, cookie, NULL);
@@ -109,15 +109,14 @@ static int mall_replace_hw_filter(struct tcf_proto *tp,
return err;
}
- err = tc_setup_cb_call(block, TC_SETUP_CLSMATCHALL, &cls_mall, skip_sw);
+ err = tc_setup_cb_add(block, tp, TC_SETUP_CLSMATCHALL, &cls_mall,
+ skip_sw, &head->flags, &head->in_hw_count, true);
+ tc_cleanup_flow_action(&cls_mall.rule->action);
kfree(cls_mall.rule);
- if (err < 0) {
+ if (err) {
mall_destroy_hw_filter(tp, head, cookie, NULL);
return err;
- } else if (err > 0) {
- head->in_hw_count = err;
- tcf_block_offload_inc(block, &head->flags);
}
if (skip_sw && !(head->flags & TCA_CLS_FLAGS_IN_HW))
@@ -302,7 +301,7 @@ static int mall_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
TC_CLSMATCHALL_REPLACE : TC_CLSMATCHALL_DESTROY;
cls_mall.cookie = (unsigned long)head;
- err = tc_setup_flow_action(&cls_mall.rule->action, &head->exts);
+ err = tc_setup_flow_action(&cls_mall.rule->action, &head->exts, true);
if (err) {
kfree(cls_mall.rule);
if (add && tc_skip_sw(head->flags)) {
@@ -312,16 +311,14 @@ static int mall_reoffload(struct tcf_proto *tp, bool add, flow_setup_cb_t *cb,
return 0;
}
- err = cb(TC_SETUP_CLSMATCHALL, &cls_mall, cb_priv);
+ err = tc_setup_cb_reoffload(block, tp, add, cb, TC_SETUP_CLSMATCHALL,
+ &cls_mall, cb_priv, &head->flags,
+ &head->in_hw_count);
+ tc_cleanup_flow_action(&cls_mall.rule->action);
kfree(cls_mall.rule);
- if (err) {
- if (add && tc_skip_sw(head->flags))
- return err;
- return 0;
- }
-
- tc_cls_offload_cnt_update(block, &head->in_hw_count, &head->flags, add);
+ if (err)
+ return err;
return 0;
}
@@ -337,7 +334,7 @@ static void mall_stats_hw_filter(struct tcf_proto *tp,
cls_mall.command = TC_CLSMATCHALL_STATS;
cls_mall.cookie = cookie;
- tc_setup_cb_call(block, TC_SETUP_CLSMATCHALL, &cls_mall, false);
+ tc_setup_cb_call(block, TC_SETUP_CLSMATCHALL, &cls_mall, false, true);
tcf_exts_stats_update(&head->exts, cls_mall.stats.bytes,
cls_mall.stats.pkts, cls_mall.stats.lastused);
diff --git a/net/sched/cls_u32.c b/net/sched/cls_u32.c
index 8614088..a0e6fac 100644
--- a/net/sched/cls_u32.c
+++ b/net/sched/cls_u32.c
@@ -480,7 +480,7 @@ static void u32_clear_hw_hnode(struct tcf_proto *tp, struct tc_u_hnode *h,
cls_u32.hnode.handle = h->handle;
cls_u32.hnode.prio = h->prio;
- tc_setup_cb_call(block, TC_SETUP_CLSU32, &cls_u32, false);
+ tc_setup_cb_call(block, TC_SETUP_CLSU32, &cls_u32, false, true);
}
static int u32_replace_hw_hnode(struct tcf_proto *tp, struct tc_u_hnode *h,
@@ -498,7 +498,7 @@ static int u32_replace_hw_hnode(struct tcf_proto *tp, struct tc_u_hnode *h,
cls_u32.hnode.handle = h->handle;
cls_u32.hnode.prio = h->prio;
- err = tc_setup_cb_call(block, TC_SETUP_CLSU32, &cls_u32, skip_sw);
+ err = tc_setup_cb_call(block, TC_SETUP_CLSU32, &cls_u32, skip_sw, true);
if (err < 0) {
u32_clear_hw_hnode(tp, h, NULL);
return err;
@@ -522,8 +522,8 @@ static void u32_remove_hw_knode(struct tcf_proto *tp, struct tc_u_knode *n,
cls_u32.command = TC_CLSU32_DELETE_KNODE;
cls_u32.knode.handle = n->handle;
- tc_setup_cb_call(block, TC_SETUP_CLSU32, &cls_u32, false);
- tcf_block_offload_dec(block, &n->flags);
+ tc_setup_cb_destroy(block, tp, TC_SETUP_CLSU32, &cls_u32, false,
+ &n->flags, &n->in_hw_count, true);
}
static int u32_replace_hw_knode(struct tcf_proto *tp, struct tc_u_knode *n,
@@ -552,13 +552,11 @@ static int u32_replace_hw_knode(struct tcf_proto *tp, struct tc_u_knode *n,
if (n->ht_down)
cls_u32.knode.link_handle = ht->handle;
- err = tc_setup_cb_call(block, TC_SETUP_CLSU32, &cls_u32, skip_sw);
- if (err < 0) {
+ err = tc_setup_cb_add(block, tp, TC_SETUP_CLSU32, &cls_u32, skip_sw,
+ &n->flags, &n->in_hw_count, true);
+ if (err) {
u32_remove_hw_knode(tp, n, NULL);
return err;
- } else if (err > 0) {
- n->in_hw_count = err;
- tcf_block_offload_inc(block, &n->flags);
}
if (skip_sw && !(n->flags & TCA_CLS_FLAGS_IN_HW))
@@ -1201,14 +1199,11 @@ static int u32_reoffload_knode(struct tcf_proto *tp, struct tc_u_knode *n,
cls_u32.knode.link_handle = ht->handle;
}
- err = cb(TC_SETUP_CLSU32, &cls_u32, cb_priv);
- if (err) {
- if (add && tc_skip_sw(n->flags))
- return err;
- return 0;
- }
-
- tc_cls_offload_cnt_update(block, &n->in_hw_count, &n->flags, add);
+ err = tc_setup_cb_reoffload(block, tp, add, cb, TC_SETUP_CLSU32,
+ &cls_u32, cb_priv, &n->flags,
+ &n->in_hw_count);
+ if (err)
+ return err;
return 0;
}
diff --git a/net/sched/sch_cbs.c b/net/sched/sch_cbs.c
index 732e109..93b58fd 100644
--- a/net/sched/sch_cbs.c
+++ b/net/sched/sch_cbs.c
@@ -181,11 +181,6 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
s64 credits;
int len;
- if (atomic64_read(&q->port_rate) == -1) {
- WARN_ONCE(1, "cbs: dequeue() called with unknown port rate.");
- return NULL;
- }
-
if (q->credits < 0) {
credits = timediff_to_credits(now - q->last, q->idleslope);
@@ -303,11 +298,19 @@ static int cbs_enable_offload(struct net_device *dev, struct cbs_sched_data *q,
static void cbs_set_port_rate(struct net_device *dev, struct cbs_sched_data *q)
{
struct ethtool_link_ksettings ecmd;
- int port_rate = -1;
+ int speed = SPEED_10;
+ int port_rate;
+ int err;
- if (!__ethtool_get_link_ksettings(dev, &ecmd) &&
- ecmd.base.speed != SPEED_UNKNOWN)
- port_rate = ecmd.base.speed * 1000 * BYTES_PER_KBIT;
+ err = __ethtool_get_link_ksettings(dev, &ecmd);
+ if (err < 0)
+ goto skip;
+
+ if (ecmd.base.speed != SPEED_UNKNOWN)
+ speed = ecmd.base.speed;
+
+skip:
+ port_rate = speed * 1000 * BYTES_PER_KBIT;
atomic64_set(&q->port_rate, port_rate);
netdev_dbg(dev, "cbs: set %s's port_rate to: %lld, linkspeed: %d\n",
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 11c03cf..137db1c 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -624,8 +624,12 @@ static int pfifo_fast_enqueue(struct sk_buff *skb, struct Qdisc *qdisc,
err = skb_array_produce(q, skb);
- if (unlikely(err))
- return qdisc_drop_cpu(skb, qdisc, to_free);
+ if (unlikely(err)) {
+ if (qdisc_is_percpu_stats(qdisc))
+ return qdisc_drop_cpu(skb, qdisc, to_free);
+ else
+ return qdisc_drop(skb, qdisc, to_free);
+ }
qdisc_update_stats_at_enqueue(qdisc, pkt_len);
return NET_XMIT_SUCCESS;
@@ -688,11 +692,14 @@ static void pfifo_fast_reset(struct Qdisc *qdisc)
kfree_skb(skb);
}
- for_each_possible_cpu(i) {
- struct gnet_stats_queue *q = per_cpu_ptr(qdisc->cpu_qstats, i);
+ if (qdisc_is_percpu_stats(qdisc)) {
+ for_each_possible_cpu(i) {
+ struct gnet_stats_queue *q;
- q->backlog = 0;
- q->qlen = 0;
+ q = per_cpu_ptr(qdisc->cpu_qstats, i);
+ q->backlog = 0;
+ q->qlen = 0;
+ }
}
}
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 540bde0..84b863e 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -477,11 +477,6 @@ static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
u32 gate_mask;
int i;
- if (atomic64_read(&q->picos_per_byte) == -1) {
- WARN_ONCE(1, "taprio: dequeue() called with unknown picos per byte.");
- return NULL;
- }
-
rcu_read_lock();
entry = rcu_dereference(q->current_entry);
/* if there's no entry, it means that the schedule didn't
@@ -954,12 +949,20 @@ static void taprio_set_picos_per_byte(struct net_device *dev,
struct taprio_sched *q)
{
struct ethtool_link_ksettings ecmd;
- int picos_per_byte = -1;
+ int speed = SPEED_10;
+ int picos_per_byte;
+ int err;
- if (!__ethtool_get_link_ksettings(dev, &ecmd) &&
- ecmd.base.speed != SPEED_UNKNOWN)
- picos_per_byte = div64_s64(NSEC_PER_SEC * 1000LL * 8,
- ecmd.base.speed * 1000 * 1000);
+ err = __ethtool_get_link_ksettings(dev, &ecmd);
+ if (err < 0)
+ goto skip;
+
+ if (ecmd.base.speed != SPEED_UNKNOWN)
+ speed = ecmd.base.speed;
+
+skip:
+ picos_per_byte = div64_s64(NSEC_PER_SEC * 1000LL * 8,
+ speed * 1000 * 1000);
atomic64_set(&q->picos_per_byte, picos_per_byte);
netdev_dbg(dev, "taprio: set %s's picos_per_byte to: %lld, linkspeed: %d\n",
@@ -1245,6 +1248,10 @@ static int taprio_init(struct Qdisc *sch, struct nlattr *opt,
*/
q->clockid = -1;
+ spin_lock(&taprio_list_lock);
+ list_add(&q->taprio_list, &taprio_list);
+ spin_unlock(&taprio_list_lock);
+
if (sch->parent != TC_H_ROOT)
return -EOPNOTSUPP;
@@ -1262,10 +1269,6 @@ static int taprio_init(struct Qdisc *sch, struct nlattr *opt,
if (!opt)
return -EINVAL;
- spin_lock(&taprio_list_lock);
- list_add(&q->taprio_list, &taprio_list);
- spin_unlock(&taprio_list_lock);
-
for (i = 0; i < dev->num_tx_queues; i++) {
struct netdev_queue *dev_queue;
struct Qdisc *qdisc;
diff --git a/net/sctp/endpointola.c b/net/sctp/endpointola.c
index 75a407d..ea53049 100644
--- a/net/sctp/endpointola.c
+++ b/net/sctp/endpointola.c
@@ -106,6 +106,7 @@ static struct sctp_endpoint *sctp_endpoint_init(struct sctp_endpoint *ep,
*/
ep->prsctp_enable = net->sctp.prsctp_enable;
ep->reconf_enable = net->sctp.reconf_enable;
+ ep->ecn_enable = net->sctp.ecn_enable;
/* Remember who we are attached to. */
ep->base.sk = sk;
diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c
index 2d47adc..b48ffe8 100644
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -1254,6 +1254,9 @@ static int __net_init sctp_defaults_init(struct net *net)
/* Disable AUTH by default. */
net->sctp.auth_enable = 0;
+ /* Enable ECN by default. */
+ net->sctp.ecn_enable = 1;
+
/* Set SCOPE policy to enabled */
net->sctp.scope_policy = SCTP_SCOPE_POLICY_ENABLE;
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 338278f..e41ed2e 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -244,7 +244,9 @@ struct sctp_chunk *sctp_make_init(const struct sctp_association *asoc,
chunksize = sizeof(init) + addrs_len;
chunksize += SCTP_PAD4(SCTP_SAT_LEN(num_types));
- chunksize += sizeof(ecap_param);
+
+ if (asoc->ep->ecn_enable)
+ chunksize += sizeof(ecap_param);
if (asoc->ep->prsctp_enable)
chunksize += sizeof(prsctp_param);
@@ -335,7 +337,8 @@ struct sctp_chunk *sctp_make_init(const struct sctp_association *asoc,
sctp_addto_chunk(retval, sizeof(sat), &sat);
sctp_addto_chunk(retval, num_types * sizeof(__u16), &types);
- sctp_addto_chunk(retval, sizeof(ecap_param), &ecap_param);
+ if (asoc->ep->ecn_enable)
+ sctp_addto_chunk(retval, sizeof(ecap_param), &ecap_param);
/* Add the supported extensions parameter. Be nice and add this
* fist before addiding the parameters for the extensions themselves
@@ -2597,8 +2600,13 @@ static int sctp_process_param(struct sctp_association *asoc,
break;
case SCTP_PARAM_ECN_CAPABLE:
- asoc->peer.ecn_capable = 1;
- break;
+ if (asoc->ep->ecn_enable) {
+ asoc->peer.ecn_capable = 1;
+ break;
+ }
+ /* Fall Through */
+ goto fall_through;
+
case SCTP_PARAM_ADAPTATION_LAYER_IND:
asoc->peer.adaptation_ind = ntohl(param.aind->adaptation_ind);
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 82bc252..3e50a97 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -4560,6 +4560,34 @@ static int sctp_setsockopt_auth_supported(struct sock *sk,
return retval;
}
+static int sctp_setsockopt_ecn_supported(struct sock *sk,
+ char __user *optval,
+ unsigned int optlen)
+{
+ struct sctp_assoc_value params;
+ struct sctp_association *asoc;
+ int retval = -EINVAL;
+
+ if (optlen != sizeof(params))
+ goto out;
+
+ if (copy_from_user(¶ms, optval, optlen)) {
+ retval = -EFAULT;
+ goto out;
+ }
+
+ asoc = sctp_id2assoc(sk, params.assoc_id);
+ if (!asoc && params.assoc_id != SCTP_FUTURE_ASSOC &&
+ sctp_style(sk, UDP))
+ goto out;
+
+ sctp_sk(sk)->ep->ecn_enable = !!params.assoc_value;
+ retval = 0;
+
+out:
+ return retval;
+}
+
/* API 6.2 setsockopt(), getsockopt()
*
* Applications use setsockopt() and getsockopt() to set or retrieve
@@ -4766,6 +4794,9 @@ static int sctp_setsockopt(struct sock *sk, int level, int optname,
case SCTP_AUTH_SUPPORTED:
retval = sctp_setsockopt_auth_supported(sk, optval, optlen);
break;
+ case SCTP_ECN_SUPPORTED:
+ retval = sctp_setsockopt_ecn_supported(sk, optval, optlen);
+ break;
default:
retval = -ENOPROTOOPT;
break;
@@ -7828,6 +7859,45 @@ static int sctp_getsockopt_auth_supported(struct sock *sk, int len,
return retval;
}
+static int sctp_getsockopt_ecn_supported(struct sock *sk, int len,
+ char __user *optval,
+ int __user *optlen)
+{
+ struct sctp_assoc_value params;
+ struct sctp_association *asoc;
+ int retval = -EFAULT;
+
+ if (len < sizeof(params)) {
+ retval = -EINVAL;
+ goto out;
+ }
+
+ len = sizeof(params);
+ if (copy_from_user(¶ms, optval, len))
+ goto out;
+
+ asoc = sctp_id2assoc(sk, params.assoc_id);
+ if (!asoc && params.assoc_id != SCTP_FUTURE_ASSOC &&
+ sctp_style(sk, UDP)) {
+ retval = -EINVAL;
+ goto out;
+ }
+
+ params.assoc_value = asoc ? asoc->peer.ecn_capable
+ : sctp_sk(sk)->ep->ecn_enable;
+
+ if (put_user(len, optlen))
+ goto out;
+
+ if (copy_to_user(optval, ¶ms, len))
+ goto out;
+
+ retval = 0;
+
+out:
+ return retval;
+}
+
static int sctp_getsockopt(struct sock *sk, int level, int optname,
char __user *optval, int __user *optlen)
{
@@ -8037,6 +8107,9 @@ static int sctp_getsockopt(struct sock *sk, int level, int optname,
retval = sctp_getsockopt_auth_supported(sk, len, optval,
optlen);
break;
+ case SCTP_ECN_SUPPORTED:
+ retval = sctp_getsockopt_ecn_supported(sk, len, optval, optlen);
+ break;
default:
retval = -ENOPROTOOPT;
break;
diff --git a/net/sctp/sysctl.c b/net/sctp/sysctl.c
index 1250751..238cf17 100644
--- a/net/sctp/sysctl.c
+++ b/net/sctp/sysctl.c
@@ -278,6 +278,13 @@ static struct ctl_table sctp_net_table[] = {
.proc_handler = proc_dointvec,
},
{
+ .procname = "ecn_enable",
+ .data = &init_net.sctp.ecn_enable,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec,
+ },
+ {
.procname = "addr_scope_policy",
.data = &init_net.sctp.scope_policy,
.maxlen = sizeof(int),
diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c
index f0de323..6c8f09c 100644
--- a/net/smc/smc_tx.c
+++ b/net/smc/smc_tx.c
@@ -76,13 +76,11 @@ static int smc_tx_wait(struct smc_sock *smc, int flags)
DEFINE_WAIT_FUNC(wait, woken_wake_function);
struct smc_connection *conn = &smc->conn;
struct sock *sk = &smc->sk;
- bool noblock;
long timeo;
int rc = 0;
/* similar to sk_stream_wait_memory */
timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT);
- noblock = timeo ? false : true;
add_wait_queue(sk_sleep(sk), &wait);
while (1) {
sk_set_bit(SOCKWQ_ASYNC_NOSPACE, sk);
@@ -97,8 +95,8 @@ static int smc_tx_wait(struct smc_sock *smc, int flags)
break;
}
if (!timeo) {
- if (noblock)
- set_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
+ /* ensure EPOLLOUT is subsequently generated */
+ set_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
rc = -EAGAIN;
break;
}
diff --git a/net/sunrpc/auth_gss/gss_krb5_keys.c b/net/sunrpc/auth_gss/gss_krb5_keys.c
index 550fdf1..3b7f721 100644
--- a/net/sunrpc/auth_gss/gss_krb5_keys.c
+++ b/net/sunrpc/auth_gss/gss_krb5_keys.c
@@ -228,14 +228,11 @@ u32 krb5_derive_key(const struct gss_krb5_enctype *gk5e,
ret = 0;
err_free_raw:
- memset(rawkey, 0, keybytes);
- kfree(rawkey);
+ kzfree(rawkey);
err_free_out:
- memset(outblockdata, 0, blocksize);
- kfree(outblockdata);
+ kzfree(outblockdata);
err_free_in:
- memset(inblockdata, 0, blocksize);
- kfree(inblockdata);
+ kzfree(inblockdata);
err_free_cipher:
crypto_free_sync_skcipher(cipher);
err_return:
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index d8679b60..a07b516 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -1970,6 +1970,7 @@ call_bind(struct rpc_task *task)
static void
call_bind_status(struct rpc_task *task)
{
+ struct rpc_xprt *xprt = task->tk_rqstp->rq_xprt;
int status = -EIO;
if (rpc_task_transmitted(task)) {
@@ -1977,14 +1978,15 @@ call_bind_status(struct rpc_task *task)
return;
}
- if (task->tk_status >= 0) {
- dprint_status(task);
+ dprint_status(task);
+ trace_rpc_bind_status(task);
+ if (task->tk_status >= 0)
+ goto out_next;
+ if (xprt_bound(xprt)) {
task->tk_status = 0;
- task->tk_action = call_connect;
- return;
+ goto out_next;
}
- trace_rpc_bind_status(task);
switch (task->tk_status) {
case -ENOMEM:
dprintk("RPC: %5u rpcbind out of memory\n", task->tk_pid);
@@ -2003,6 +2005,9 @@ call_bind_status(struct rpc_task *task)
task->tk_rebind_retry--;
rpc_delay(task, 3*HZ);
goto retry_timeout;
+ case -ENOBUFS:
+ rpc_delay(task, HZ >> 2);
+ goto retry_timeout;
case -EAGAIN:
goto retry_timeout;
case -ETIMEDOUT:
@@ -2026,7 +2031,6 @@ call_bind_status(struct rpc_task *task)
case -ENETDOWN:
case -EHOSTUNREACH:
case -ENETUNREACH:
- case -ENOBUFS:
case -EPIPE:
dprintk("RPC: %5u remote rpcbind unreachable: %d\n",
task->tk_pid, task->tk_status);
@@ -2043,7 +2047,9 @@ call_bind_status(struct rpc_task *task)
rpc_call_rpcerror(task, status);
return;
-
+out_next:
+ task->tk_action = call_connect;
+ return;
retry_timeout:
task->tk_status = 0;
task->tk_action = call_bind;
@@ -2090,6 +2096,7 @@ call_connect(struct rpc_task *task)
static void
call_connect_status(struct rpc_task *task)
{
+ struct rpc_xprt *xprt = task->tk_rqstp->rq_xprt;
struct rpc_clnt *clnt = task->tk_client;
int status = task->tk_status;
@@ -2099,8 +2106,17 @@ call_connect_status(struct rpc_task *task)
}
dprint_status(task);
-
trace_rpc_connect_status(task);
+
+ if (task->tk_status == 0) {
+ clnt->cl_stats->netreconn++;
+ goto out_next;
+ }
+ if (xprt_connected(xprt)) {
+ task->tk_status = 0;
+ goto out_next;
+ }
+
task->tk_status = 0;
switch (status) {
case -ECONNREFUSED:
@@ -2117,8 +2133,6 @@ call_connect_status(struct rpc_task *task)
case -ENETDOWN:
case -ENETUNREACH:
case -EHOSTUNREACH:
- case -EADDRINUSE:
- case -ENOBUFS:
case -EPIPE:
xprt_conditional_disconnect(task->tk_rqstp->rq_xprt,
task->tk_rqstp->rq_connect_cookie);
@@ -2127,17 +2141,20 @@ call_connect_status(struct rpc_task *task)
/* retry with existing socket, after a delay */
rpc_delay(task, 3*HZ);
/* fall through */
+ case -EADDRINUSE:
case -ENOTCONN:
case -EAGAIN:
case -ETIMEDOUT:
goto out_retry;
- case 0:
- clnt->cl_stats->netreconn++;
- task->tk_action = call_transmit;
- return;
+ case -ENOBUFS:
+ rpc_delay(task, HZ >> 2);
+ goto out_retry;
}
rpc_call_rpcerror(task, status);
return;
+out_next:
+ task->tk_action = call_transmit;
+ return;
out_retry:
/* Check for timeouts before looping back to call_bind */
task->tk_action = call_bind;
@@ -2365,7 +2382,7 @@ call_status(struct rpc_task *task)
case -ECONNABORTED:
case -ENOTCONN:
rpc_force_rebind(clnt);
- /* fall through */
+ break;
case -EADDRINUSE:
rpc_delay(task, 3*HZ);
/* fall through */
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index 783748d..2e71f54 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -1408,13 +1408,6 @@ xprt_request_transmit(struct rpc_rqst *req, struct rpc_task *snd_task)
status = -EBADMSG;
goto out_dequeue;
}
- if (task->tk_ops->rpc_call_prepare_transmit) {
- task->tk_ops->rpc_call_prepare_transmit(task,
- task->tk_calldata);
- status = task->tk_status;
- if (status < 0)
- goto out_dequeue;
- }
if (RPC_SIGNALLED(task)) {
status = -ERESTARTSYS;
goto out_dequeue;
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index a470df7..f959487 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -61,7 +61,7 @@ static void tls_device_free_ctx(struct tls_context *ctx)
if (ctx->rx_conf == TLS_HW)
kfree(tls_offload_ctx_rx(ctx));
- tls_ctx_free(ctx);
+ tls_ctx_free(NULL, ctx);
}
static void tls_device_gc_task(struct work_struct *work)
@@ -122,13 +122,10 @@ static struct net_device *get_netdev_for_sock(struct sock *sk)
static void destroy_record(struct tls_record_info *record)
{
- int nr_frags = record->num_frags;
- skb_frag_t *frag;
+ int i;
- while (nr_frags-- > 0) {
- frag = &record->frags[nr_frags];
- __skb_frag_unref(frag);
- }
+ for (i = 0; i < record->num_frags; i++)
+ __skb_frag_unref(&record->frags[i]);
kfree(record);
}
@@ -159,12 +156,8 @@ static void tls_icsk_clean_acked(struct sock *sk, u32 acked_seq)
spin_lock_irqsave(&ctx->lock, flags);
info = ctx->retransmit_hint;
- if (info && !before(acked_seq, info->end_seq)) {
+ if (info && !before(acked_seq, info->end_seq))
ctx->retransmit_hint = NULL;
- list_del(&info->list);
- destroy_record(info);
- deleted_records++;
- }
list_for_each_entry_safe(info, temp, &ctx->records_list, list) {
if (before(acked_seq, info->end_seq))
@@ -263,33 +256,15 @@ static int tls_push_record(struct sock *sk,
struct tls_context *ctx,
struct tls_offload_context_tx *offload_ctx,
struct tls_record_info *record,
- struct page_frag *pfrag,
- int flags,
- unsigned char record_type)
+ int flags)
{
struct tls_prot_info *prot = &ctx->prot_info;
struct tcp_sock *tp = tcp_sk(sk);
- struct page_frag dummy_tag_frag;
skb_frag_t *frag;
int i;
- /* fill prepend */
- frag = &record->frags[0];
- tls_fill_prepend(ctx,
- skb_frag_address(frag),
- record->len - prot->prepend_size,
- record_type,
- prot->version);
-
- /* HW doesn't care about the data in the tag, because it fills it. */
- dummy_tag_frag.page = skb_frag_page(frag);
- dummy_tag_frag.offset = 0;
-
- tls_append_frag(record, &dummy_tag_frag, prot->tag_size);
record->end_seq = tp->write_seq + record->len;
- spin_lock_irq(&offload_ctx->lock);
- list_add_tail(&record->list, &offload_ctx->records_list);
- spin_unlock_irq(&offload_ctx->lock);
+ list_add_tail_rcu(&record->list, &offload_ctx->records_list);
offload_ctx->open_record = NULL;
if (test_bit(TLS_TX_SYNC_SCHED, &ctx->flags))
@@ -311,6 +286,38 @@ static int tls_push_record(struct sock *sk,
return tls_push_sg(sk, ctx, offload_ctx->sg_tx_data, 0, flags);
}
+static int tls_device_record_close(struct sock *sk,
+ struct tls_context *ctx,
+ struct tls_record_info *record,
+ struct page_frag *pfrag,
+ unsigned char record_type)
+{
+ struct tls_prot_info *prot = &ctx->prot_info;
+ int ret;
+
+ /* append tag
+ * device will fill in the tag, we just need to append a placeholder
+ * use socket memory to improve coalescing (re-using a single buffer
+ * increases frag count)
+ * if we can't allocate memory now, steal some back from data
+ */
+ if (likely(skb_page_frag_refill(prot->tag_size, pfrag,
+ sk->sk_allocation))) {
+ ret = 0;
+ tls_append_frag(record, pfrag, prot->tag_size);
+ } else {
+ ret = prot->tag_size;
+ if (record->len <= prot->overhead_size)
+ return -ENOMEM;
+ }
+
+ /* fill prepend */
+ tls_fill_prepend(ctx, skb_frag_address(&record->frags[0]),
+ record->len - prot->overhead_size,
+ record_type, prot->version);
+ return ret;
+}
+
static int tls_create_new_record(struct tls_offload_context_tx *offload_ctx,
struct page_frag *pfrag,
size_t prepend_size)
@@ -365,6 +372,31 @@ static int tls_do_allocation(struct sock *sk,
return 0;
}
+static int tls_device_copy_data(void *addr, size_t bytes, struct iov_iter *i)
+{
+ size_t pre_copy, nocache;
+
+ pre_copy = ~((unsigned long)addr - 1) & (SMP_CACHE_BYTES - 1);
+ if (pre_copy) {
+ pre_copy = min(pre_copy, bytes);
+ if (copy_from_iter(addr, pre_copy, i) != pre_copy)
+ return -EFAULT;
+ bytes -= pre_copy;
+ addr += pre_copy;
+ }
+
+ nocache = round_down(bytes, SMP_CACHE_BYTES);
+ if (copy_from_iter_nocache(addr, nocache, i) != nocache)
+ return -EFAULT;
+ bytes -= nocache;
+ addr += nocache;
+
+ if (bytes && copy_from_iter(addr, bytes, i) != bytes)
+ return -EFAULT;
+
+ return 0;
+}
+
static int tls_push_data(struct sock *sk,
struct iov_iter *msg_iter,
size_t size, int flags,
@@ -438,12 +470,10 @@ static int tls_push_data(struct sock *sk,
copy = min_t(size_t, size, (pfrag->size - pfrag->offset));
copy = min_t(size_t, copy, (max_open_record_len - record->len));
- if (copy_from_iter_nocache(page_address(pfrag->page) +
- pfrag->offset,
- copy, msg_iter) != copy) {
- rc = -EFAULT;
+ rc = tls_device_copy_data(page_address(pfrag->page) +
+ pfrag->offset, copy, msg_iter);
+ if (rc)
goto handle_error;
- }
tls_append_frag(record, pfrag, copy);
size -= copy;
@@ -461,13 +491,24 @@ static int tls_push_data(struct sock *sk,
if (done || record->len >= max_open_record_len ||
(record->num_frags >= MAX_SKB_FRAGS - 1)) {
+ rc = tls_device_record_close(sk, tls_ctx, record,
+ pfrag, record_type);
+ if (rc) {
+ if (rc > 0) {
+ size += rc;
+ } else {
+ size = orig_size;
+ destroy_record(record);
+ ctx->open_record = NULL;
+ break;
+ }
+ }
+
rc = tls_push_record(sk,
tls_ctx,
ctx,
record,
- pfrag,
- tls_push_record_flags,
- record_type);
+ tls_push_record_flags);
if (rc < 0)
break;
}
@@ -542,12 +583,16 @@ struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context,
/* if retransmit_hint is irrelevant start
* from the beggining of the list
*/
- info = list_first_entry(&context->records_list,
- struct tls_record_info, list);
+ info = list_first_entry_or_null(&context->records_list,
+ struct tls_record_info, list);
+ if (!info)
+ return NULL;
record_sn = context->unacked_record_sn;
}
- list_for_each_entry_from(info, &context->records_list, list) {
+ /* We just need the _rcu for the READ_ONCE() */
+ rcu_read_lock();
+ list_for_each_entry_from_rcu(info, &context->records_list, list) {
if (before(seq, info->end_seq)) {
if (!context->retransmit_hint ||
after(info->end_seq,
@@ -556,12 +601,15 @@ struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context,
context->retransmit_hint = info;
}
*p_record_sn = record_sn;
- return info;
+ goto exit_rcu_unlock;
}
record_sn++;
}
+ info = NULL;
- return NULL;
+exit_rcu_unlock:
+ rcu_read_unlock();
+ return info;
}
EXPORT_SYMBOL(tls_get_record);
@@ -838,22 +886,18 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
struct net_device *netdev;
char *iv, *rec_seq;
struct sk_buff *skb;
- int rc = -EINVAL;
__be64 rcd_sn;
+ int rc;
if (!ctx)
- goto out;
+ return -EINVAL;
- if (ctx->priv_ctx_tx) {
- rc = -EEXIST;
- goto out;
- }
+ if (ctx->priv_ctx_tx)
+ return -EEXIST;
start_marker_record = kmalloc(sizeof(*start_marker_record), GFP_KERNEL);
- if (!start_marker_record) {
- rc = -ENOMEM;
- goto out;
- }
+ if (!start_marker_record)
+ return -ENOMEM;
offload_ctx = kzalloc(TLS_OFFLOAD_CONTEXT_SIZE_TX, GFP_KERNEL);
if (!offload_ctx) {
@@ -939,17 +983,11 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
if (skb)
TCP_SKB_CB(skb)->eor = 1;
- /* We support starting offload on multiple sockets
- * concurrently, so we only need a read lock here.
- * This lock must precede get_netdev_for_sock to prevent races between
- * NETDEV_DOWN and setsockopt.
- */
- down_read(&device_offload_lock);
netdev = get_netdev_for_sock(sk);
if (!netdev) {
pr_err_ratelimited("%s: netdev not found\n", __func__);
rc = -EINVAL;
- goto release_lock;
+ goto disable_cad;
}
if (!(netdev->features & NETIF_F_HW_TLS_TX)) {
@@ -960,10 +998,15 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
/* Avoid offloading if the device is down
* We don't want to offload new flows after
* the NETDEV_DOWN event
+ *
+ * device_offload_lock is taken in tls_devices's NETDEV_DOWN
+ * handler thus protecting from the device going down before
+ * ctx was added to tls_device_list.
*/
+ down_read(&device_offload_lock);
if (!(netdev->flags & IFF_UP)) {
rc = -EINVAL;
- goto release_netdev;
+ goto release_lock;
}
ctx->priv_ctx_tx = offload_ctx;
@@ -971,9 +1014,10 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
&ctx->crypto_send.info,
tcp_sk(sk)->write_seq);
if (rc)
- goto release_netdev;
+ goto release_lock;
tls_device_attach(ctx, sk, netdev);
+ up_read(&device_offload_lock);
/* following this assignment tls_is_sk_tx_device_offloaded
* will return true and the context might be accessed
@@ -981,13 +1025,14 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
*/
smp_store_release(&sk->sk_validate_xmit_skb, tls_validate_xmit_skb);
dev_put(netdev);
- up_read(&device_offload_lock);
- goto out;
-release_netdev:
- dev_put(netdev);
+ return 0;
+
release_lock:
up_read(&device_offload_lock);
+release_netdev:
+ dev_put(netdev);
+disable_cad:
clean_acked_data_disable(inet_csk(sk));
crypto_free_aead(offload_ctx->aead_send);
free_rec_seq:
@@ -999,7 +1044,6 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx)
ctx->priv_ctx_tx = NULL;
free_marker_record:
kfree(start_marker_record);
-out:
return rc;
}
@@ -1012,17 +1056,10 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
if (ctx->crypto_recv.info.version != TLS_1_2_VERSION)
return -EOPNOTSUPP;
- /* We support starting offload on multiple sockets
- * concurrently, so we only need a read lock here.
- * This lock must precede get_netdev_for_sock to prevent races between
- * NETDEV_DOWN and setsockopt.
- */
- down_read(&device_offload_lock);
netdev = get_netdev_for_sock(sk);
if (!netdev) {
pr_err_ratelimited("%s: netdev not found\n", __func__);
- rc = -EINVAL;
- goto release_lock;
+ return -EINVAL;
}
if (!(netdev->features & NETIF_F_HW_TLS_RX)) {
@@ -1033,16 +1070,21 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
/* Avoid offloading if the device is down
* We don't want to offload new flows after
* the NETDEV_DOWN event
+ *
+ * device_offload_lock is taken in tls_devices's NETDEV_DOWN
+ * handler thus protecting from the device going down before
+ * ctx was added to tls_device_list.
*/
+ down_read(&device_offload_lock);
if (!(netdev->flags & IFF_UP)) {
rc = -EINVAL;
- goto release_netdev;
+ goto release_lock;
}
context = kzalloc(TLS_OFFLOAD_CONTEXT_SIZE_RX, GFP_KERNEL);
if (!context) {
rc = -ENOMEM;
- goto release_netdev;
+ goto release_lock;
}
context->resync_nh_reset = 1;
@@ -1058,7 +1100,11 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
goto free_sw_resources;
tls_device_attach(ctx, sk, netdev);
- goto release_netdev;
+ up_read(&device_offload_lock);
+
+ dev_put(netdev);
+
+ return 0;
free_sw_resources:
up_read(&device_offload_lock);
@@ -1066,10 +1112,10 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
down_read(&device_offload_lock);
release_ctx:
ctx->priv_ctx_rx = NULL;
-release_netdev:
- dev_put(netdev);
release_lock:
up_read(&device_offload_lock);
+release_netdev:
+ dev_put(netdev);
return rc;
}
diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index 43252a8..ac88877 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -39,6 +39,7 @@
#include <linux/netdevice.h>
#include <linux/sched/signal.h>
#include <linux/inetdevice.h>
+#include <linux/inet_diag.h>
#include <net/tls.h>
@@ -251,14 +252,26 @@ static void tls_write_space(struct sock *sk)
ctx->sk_write_space(sk);
}
-void tls_ctx_free(struct tls_context *ctx)
+/**
+ * tls_ctx_free() - free TLS ULP context
+ * @sk: socket to with @ctx is attached
+ * @ctx: TLS context structure
+ *
+ * Free TLS context. If @sk is %NULL caller guarantees that the socket
+ * to which @ctx was attached has no outstanding references.
+ */
+void tls_ctx_free(struct sock *sk, struct tls_context *ctx)
{
if (!ctx)
return;
memzero_explicit(&ctx->crypto_send, sizeof(ctx->crypto_send));
memzero_explicit(&ctx->crypto_recv, sizeof(ctx->crypto_recv));
- kfree(ctx);
+
+ if (sk)
+ kfree_rcu(ctx, rcu);
+ else
+ kfree(ctx);
}
static void tls_sk_proto_cleanup(struct sock *sk,
@@ -273,19 +286,14 @@ static void tls_sk_proto_cleanup(struct sock *sk,
kfree(ctx->tx.rec_seq);
kfree(ctx->tx.iv);
tls_sw_release_resources_tx(sk);
-#ifdef CONFIG_TLS_DEVICE
} else if (ctx->tx_conf == TLS_HW) {
tls_device_free_resources_tx(sk);
-#endif
}
if (ctx->rx_conf == TLS_SW)
tls_sw_release_resources_rx(sk);
-
-#ifdef CONFIG_TLS_DEVICE
- if (ctx->rx_conf == TLS_HW)
+ else if (ctx->rx_conf == TLS_HW)
tls_device_offload_cleanup_rx(sk);
-#endif
}
static void tls_sk_proto_close(struct sock *sk, long timeout)
@@ -306,7 +314,7 @@ static void tls_sk_proto_close(struct sock *sk, long timeout)
write_lock_bh(&sk->sk_callback_lock);
if (free_ctx)
- icsk->icsk_ulp_data = NULL;
+ rcu_assign_pointer(icsk->icsk_ulp_data, NULL);
sk->sk_prot = ctx->sk_proto;
if (sk->sk_write_space == tls_write_space)
sk->sk_write_space = ctx->sk_write_space;
@@ -318,10 +326,10 @@ static void tls_sk_proto_close(struct sock *sk, long timeout)
tls_sw_strparser_done(ctx);
if (ctx->rx_conf == TLS_SW)
tls_sw_free_ctx_rx(ctx);
- ctx->sk_proto_close(sk, timeout);
+ ctx->sk_proto->close(sk, timeout);
if (free_ctx)
- tls_ctx_free(ctx);
+ tls_ctx_free(sk, ctx);
}
static int do_tls_getsockopt_tx(struct sock *sk, char __user *optval,
@@ -438,7 +446,8 @@ static int tls_getsockopt(struct sock *sk, int level, int optname,
struct tls_context *ctx = tls_get_ctx(sk);
if (level != SOL_TLS)
- return ctx->getsockopt(sk, level, optname, optval, optlen);
+ return ctx->sk_proto->getsockopt(sk, level,
+ optname, optval, optlen);
return do_tls_getsockopt(sk, optname, optval, optlen);
}
@@ -523,26 +532,18 @@ static int do_tls_setsockopt_conf(struct sock *sk, char __user *optval,
}
if (tx) {
-#ifdef CONFIG_TLS_DEVICE
rc = tls_set_device_offload(sk, ctx);
conf = TLS_HW;
if (rc) {
-#else
- {
-#endif
rc = tls_set_sw_offload(sk, ctx, 1);
if (rc)
goto err_crypto_info;
conf = TLS_SW;
}
} else {
-#ifdef CONFIG_TLS_DEVICE
rc = tls_set_device_offload_rx(sk, ctx);
conf = TLS_HW;
if (rc) {
-#else
- {
-#endif
rc = tls_set_sw_offload(sk, ctx, 0);
if (rc)
goto err_crypto_info;
@@ -596,7 +597,8 @@ static int tls_setsockopt(struct sock *sk, int level, int optname,
struct tls_context *ctx = tls_get_ctx(sk);
if (level != SOL_TLS)
- return ctx->setsockopt(sk, level, optname, optval, optlen);
+ return ctx->sk_proto->setsockopt(sk, level, optname, optval,
+ optlen);
return do_tls_setsockopt(sk, optname, optval, optlen);
}
@@ -610,11 +612,8 @@ static struct tls_context *create_ctx(struct sock *sk)
if (!ctx)
return NULL;
- icsk->icsk_ulp_data = ctx;
- ctx->setsockopt = sk->sk_prot->setsockopt;
- ctx->getsockopt = sk->sk_prot->getsockopt;
- ctx->sk_proto_close = sk->sk_prot->close;
- ctx->unhash = sk->sk_prot->unhash;
+ rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
+ ctx->sk_proto = sk->sk_prot;
return ctx;
}
@@ -651,8 +650,8 @@ static void tls_hw_sk_destruct(struct sock *sk)
ctx->sk_destruct(sk);
/* Free ctx */
- tls_ctx_free(ctx);
- icsk->icsk_ulp_data = NULL;
+ rcu_assign_pointer(icsk->icsk_ulp_data, NULL);
+ tls_ctx_free(sk, ctx);
}
static int tls_hw_prot(struct sock *sk)
@@ -670,9 +669,6 @@ static int tls_hw_prot(struct sock *sk)
spin_unlock_bh(&device_spinlock);
tls_build_proto(sk);
- ctx->hash = sk->sk_prot->hash;
- ctx->unhash = sk->sk_prot->unhash;
- ctx->sk_proto_close = sk->sk_prot->close;
ctx->sk_destruct = sk->sk_destruct;
sk->sk_destruct = tls_hw_sk_destruct;
ctx->rx_conf = TLS_HW_RECORD;
@@ -704,7 +700,7 @@ static void tls_hw_unhash(struct sock *sk)
}
}
spin_unlock_bh(&device_spinlock);
- ctx->unhash(sk);
+ ctx->sk_proto->unhash(sk);
}
static int tls_hw_hash(struct sock *sk)
@@ -713,7 +709,7 @@ static int tls_hw_hash(struct sock *sk)
struct tls_device *dev;
int err;
- err = ctx->hash(sk);
+ err = ctx->sk_proto->hash(sk);
spin_lock_bh(&device_spinlock);
list_for_each_entry(dev, &device_list, dev_list) {
if (dev->hash) {
@@ -803,7 +799,6 @@ static int tls_init(struct sock *sk)
ctx->tx_conf = TLS_BASE;
ctx->rx_conf = TLS_BASE;
- ctx->sk_proto = sk->sk_prot;
update_sk_prot(sk, ctx);
out:
write_unlock_bh(&sk->sk_callback_lock);
@@ -815,12 +810,71 @@ static void tls_update(struct sock *sk, struct proto *p)
struct tls_context *ctx;
ctx = tls_get_ctx(sk);
- if (likely(ctx)) {
- ctx->sk_proto_close = p->close;
+ if (likely(ctx))
ctx->sk_proto = p;
- } else {
+ else
sk->sk_prot = p;
+}
+
+static int tls_get_info(const struct sock *sk, struct sk_buff *skb)
+{
+ u16 version, cipher_type;
+ struct tls_context *ctx;
+ struct nlattr *start;
+ int err;
+
+ start = nla_nest_start_noflag(skb, INET_ULP_INFO_TLS);
+ if (!start)
+ return -EMSGSIZE;
+
+ rcu_read_lock();
+ ctx = rcu_dereference(inet_csk(sk)->icsk_ulp_data);
+ if (!ctx) {
+ err = 0;
+ goto nla_failure;
}
+ version = ctx->prot_info.version;
+ if (version) {
+ err = nla_put_u16(skb, TLS_INFO_VERSION, version);
+ if (err)
+ goto nla_failure;
+ }
+ cipher_type = ctx->prot_info.cipher_type;
+ if (cipher_type) {
+ err = nla_put_u16(skb, TLS_INFO_CIPHER, cipher_type);
+ if (err)
+ goto nla_failure;
+ }
+ err = nla_put_u16(skb, TLS_INFO_TXCONF, tls_user_config(ctx, true));
+ if (err)
+ goto nla_failure;
+
+ err = nla_put_u16(skb, TLS_INFO_RXCONF, tls_user_config(ctx, false));
+ if (err)
+ goto nla_failure;
+
+ rcu_read_unlock();
+ nla_nest_end(skb, start);
+ return 0;
+
+nla_failure:
+ rcu_read_unlock();
+ nla_nest_cancel(skb, start);
+ return err;
+}
+
+static size_t tls_get_info_size(const struct sock *sk)
+{
+ size_t size = 0;
+
+ size += nla_total_size(0) + /* INET_ULP_INFO_TLS */
+ nla_total_size(sizeof(u16)) + /* TLS_INFO_VERSION */
+ nla_total_size(sizeof(u16)) + /* TLS_INFO_CIPHER */
+ nla_total_size(sizeof(u16)) + /* TLS_INFO_RXCONF */
+ nla_total_size(sizeof(u16)) + /* TLS_INFO_TXCONF */
+ 0;
+
+ return size;
}
void tls_register_device(struct tls_device *device)
@@ -844,6 +898,8 @@ static struct tcp_ulp_ops tcp_tls_ulp_ops __read_mostly = {
.owner = THIS_MODULE,
.init = tls_init,
.update = tls_update,
+ .get_info = tls_get_info,
+ .get_info_size = tls_get_info_size,
};
static int __init tls_register(void)
@@ -851,9 +907,7 @@ static int __init tls_register(void)
tls_sw_proto_ops = inet_stream_ops;
tls_sw_proto_ops.splice_read = tls_sw_splice_read;
-#ifdef CONFIG_TLS_DEVICE
tls_device_init();
-#endif
tcp_register_ulp(&tcp_tls_ulp_ops);
return 0;
@@ -862,9 +916,7 @@ static int __init tls_register(void)
static void __exit tls_unregister(void)
{
tcp_unregister_ulp(&tcp_tls_ulp_ops);
-#ifdef CONFIG_TLS_DEVICE
tls_device_cleanup();
-#endif
}
module_init(tls_register);
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index 91d21b0..c2b5e0d 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -1489,13 +1489,12 @@ static int decrypt_skb_update(struct sock *sk, struct sk_buff *skb,
int pad, err = 0;
if (!ctx->decrypted) {
-#ifdef CONFIG_TLS_DEVICE
if (tls_ctx->rx_conf == TLS_HW) {
err = tls_device_decrypted(sk, skb);
if (err < 0)
return err;
}
-#endif
+
/* Still not decrypted after tls_device */
if (!ctx->decrypted) {
err = decrypt_internal(sk, skb, dest, NULL, chunk, zc,
@@ -2014,10 +2013,9 @@ static int tls_read_size(struct strparser *strp, struct sk_buff *skb)
ret = -EINVAL;
goto read_failure;
}
-#ifdef CONFIG_TLS_DEVICE
+
tls_device_rx_resync_new_rec(strp->sk, data_len + TLS_HEADER_SIZE,
TCP_SKB_CB(skb)->seq + rxm->offset);
-#endif
return data_len + TLS_HEADER_SIZE;
read_failure:
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 94cc0fa..5bb70c6 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -307,8 +307,13 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
spin_unlock_bh(&vvs->rx_lock);
- /* We send a credit update only when the space available seen
- * by the transmitter is less than VIRTIO_VSOCK_MAX_PKT_BUF_SIZE
+ /* To reduce the number of credit update messages,
+ * don't update credits as long as lots of space is available.
+ * Note: the limit chosen here is arbitrary. Setting the limit
+ * too high causes extra messages. Too low causes transmitter
+ * stalls. As stalls are in theory more expensive than extra
+ * messages, we set the limit to a high value. TODO: experiment
+ * with different values.
*/
if (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE) {
virtio_transport_send_credit_update(vsk,
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index 646107a..5311d0a 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -2788,7 +2788,7 @@ static void reg_process_pending_hints(void)
/* When last_request->processed becomes true this will be rescheduled */
if (lr && !lr->processed) {
- reg_process_hint(lr);
+ pr_debug("Pending regulatory request, waiting for it to be processed...\n");
return;
}
diff --git a/net/wireless/util.c b/net/wireless/util.c
index 633f6a3..419eb12 100644
--- a/net/wireless/util.c
+++ b/net/wireless/util.c
@@ -242,25 +242,30 @@ int cfg80211_validate_key_settings(struct cfg80211_registered_device *rdev,
switch (params->cipher) {
case WLAN_CIPHER_SUITE_TKIP:
+ /* Extended Key ID can only be used with CCMP/GCMP ciphers */
+ if ((pairwise && key_idx) ||
+ params->mode != NL80211_KEY_RX_TX)
+ return -EINVAL;
+ break;
case WLAN_CIPHER_SUITE_CCMP:
case WLAN_CIPHER_SUITE_CCMP_256:
case WLAN_CIPHER_SUITE_GCMP:
case WLAN_CIPHER_SUITE_GCMP_256:
- /* IEEE802.11-2016 allows only 0 and - when using Extended Key
- * ID - 1 as index for pairwise keys.
+ /* IEEE802.11-2016 allows only 0 and - when supporting
+ * Extended Key ID - 1 as index for pairwise keys.
* @NL80211_KEY_NO_TX is only allowed for pairwise keys when
* the driver supports Extended Key ID.
* @NL80211_KEY_SET_TX can't be set when installing and
* validating a key.
*/
- if (params->mode == NL80211_KEY_NO_TX) {
- if (!wiphy_ext_feature_isset(&rdev->wiphy,
- NL80211_EXT_FEATURE_EXT_KEY_ID))
+ if ((params->mode == NL80211_KEY_NO_TX && !pairwise) ||
+ params->mode == NL80211_KEY_SET_TX)
+ return -EINVAL;
+ if (wiphy_ext_feature_isset(&rdev->wiphy,
+ NL80211_EXT_FEATURE_EXT_KEY_ID)) {
+ if (pairwise && (key_idx < 0 || key_idx > 1))
return -EINVAL;
- else if (!pairwise || key_idx < 0 || key_idx > 1)
- return -EINVAL;
- } else if ((pairwise && key_idx) ||
- params->mode == NL80211_KEY_SET_TX) {
+ } else if (pairwise && key_idx) {
return -EINVAL;
}
break;
diff --git a/net/xdp/xdp_umem.c b/net/xdp/xdp_umem.c
index a060796..947b8ff 100644
--- a/net/xdp/xdp_umem.c
+++ b/net/xdp/xdp_umem.c
@@ -14,7 +14,7 @@
#include <linux/netdevice.h>
#include <linux/rtnetlink.h>
#include <linux/idr.h>
-#include <linux/highmem.h>
+#include <linux/vmalloc.h>
#include "xdp_umem.h"
#include "xsk_queue.h"
@@ -106,14 +106,22 @@ int xdp_umem_assign_dev(struct xdp_umem *umem, struct net_device *dev,
umem->dev = dev;
umem->queue_id = queue_id;
+ if (flags & XDP_USE_NEED_WAKEUP) {
+ umem->flags |= XDP_UMEM_USES_NEED_WAKEUP;
+ /* Tx needs to be explicitly woken up the first time.
+ * Also for supporting drivers that do not implement this
+ * feature. They will always have to call sendto().
+ */
+ xsk_set_tx_need_wakeup(umem);
+ }
+
dev_hold(dev);
if (force_copy)
/* For copy-mode, we are done. */
return 0;
- if (!dev->netdev_ops->ndo_bpf ||
- !dev->netdev_ops->ndo_xsk_async_xmit) {
+ if (!dev->netdev_ops->ndo_bpf || !dev->netdev_ops->ndo_xsk_wakeup) {
err = -EOPNOTSUPP;
goto err_unreg_umem;
}
@@ -170,7 +178,30 @@ static void xdp_umem_unmap_pages(struct xdp_umem *umem)
unsigned int i;
for (i = 0; i < umem->npgs; i++)
- kunmap(umem->pgs[i]);
+ if (PageHighMem(umem->pgs[i]))
+ vunmap(umem->pages[i].addr);
+}
+
+static int xdp_umem_map_pages(struct xdp_umem *umem)
+{
+ unsigned int i;
+ void *addr;
+
+ for (i = 0; i < umem->npgs; i++) {
+ if (PageHighMem(umem->pgs[i]))
+ addr = vmap(&umem->pgs[i], 1, VM_MAP, PAGE_KERNEL);
+ else
+ addr = page_address(umem->pgs[i]);
+
+ if (!addr) {
+ xdp_umem_unmap_pages(umem);
+ return -ENOMEM;
+ }
+
+ umem->pages[i].addr = addr;
+ }
+
+ return 0;
}
static void xdp_umem_unpin_pages(struct xdp_umem *umem)
@@ -309,10 +340,11 @@ static int xdp_umem_account_pages(struct xdp_umem *umem)
static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr)
{
+ bool unaligned_chunks = mr->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG;
u32 chunk_size = mr->chunk_size, headroom = mr->headroom;
unsigned int chunks, chunks_per_page;
u64 addr = mr->addr, size = mr->len;
- int size_chk, err, i;
+ int size_chk, err;
if (chunk_size < XDP_UMEM_MIN_CHUNK_SIZE || chunk_size > PAGE_SIZE) {
/* Strictly speaking we could support this, if:
@@ -324,7 +356,11 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr)
return -EINVAL;
}
- if (!is_power_of_2(chunk_size))
+ if (mr->flags & ~(XDP_UMEM_UNALIGNED_CHUNK_FLAG |
+ XDP_UMEM_USES_NEED_WAKEUP))
+ return -EINVAL;
+
+ if (!unaligned_chunks && !is_power_of_2(chunk_size))
return -EINVAL;
if (!PAGE_ALIGNED(addr)) {
@@ -341,9 +377,11 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr)
if (chunks == 0)
return -EINVAL;
- chunks_per_page = PAGE_SIZE / chunk_size;
- if (chunks < chunks_per_page || chunks % chunks_per_page)
- return -EINVAL;
+ if (!unaligned_chunks) {
+ chunks_per_page = PAGE_SIZE / chunk_size;
+ if (chunks < chunks_per_page || chunks % chunks_per_page)
+ return -EINVAL;
+ }
headroom = ALIGN(headroom, 64);
@@ -352,13 +390,15 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr)
return -EINVAL;
umem->address = (unsigned long)addr;
- umem->chunk_mask = ~((u64)chunk_size - 1);
+ umem->chunk_mask = unaligned_chunks ? XSK_UNALIGNED_BUF_ADDR_MASK
+ : ~((u64)chunk_size - 1);
umem->size = size;
umem->headroom = headroom;
umem->chunk_size_nohr = chunk_size - headroom;
umem->npgs = size / PAGE_SIZE;
umem->pgs = NULL;
umem->user = NULL;
+ umem->flags = mr->flags;
INIT_LIST_HEAD(&umem->xsk_list);
spin_lock_init(&umem->xsk_list_lock);
@@ -375,14 +415,17 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr)
umem->pages = kcalloc(umem->npgs, sizeof(*umem->pages), GFP_KERNEL);
if (!umem->pages) {
err = -ENOMEM;
- goto out_account;
+ goto out_pin;
}
- for (i = 0; i < umem->npgs; i++)
- umem->pages[i].addr = kmap(umem->pgs[i]);
+ err = xdp_umem_map_pages(umem);
+ if (!err)
+ return 0;
- return 0;
+ kfree(umem->pages);
+out_pin:
+ xdp_umem_unpin_pages(umem);
out_account:
xdp_umem_unaccount_pages(umem);
return err;
diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c
index 59b57d7..c2f1af3 100644
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -45,7 +45,7 @@ EXPORT_SYMBOL(xsk_umem_has_addrs);
u64 *xsk_umem_peek_addr(struct xdp_umem *umem, u64 *addr)
{
- return xskq_peek_addr(umem->fq, addr);
+ return xskq_peek_addr(umem->fq, addr, umem);
}
EXPORT_SYMBOL(xsk_umem_peek_addr);
@@ -55,21 +55,103 @@ void xsk_umem_discard_addr(struct xdp_umem *umem)
}
EXPORT_SYMBOL(xsk_umem_discard_addr);
+void xsk_set_rx_need_wakeup(struct xdp_umem *umem)
+{
+ if (umem->need_wakeup & XDP_WAKEUP_RX)
+ return;
+
+ umem->fq->ring->flags |= XDP_RING_NEED_WAKEUP;
+ umem->need_wakeup |= XDP_WAKEUP_RX;
+}
+EXPORT_SYMBOL(xsk_set_rx_need_wakeup);
+
+void xsk_set_tx_need_wakeup(struct xdp_umem *umem)
+{
+ struct xdp_sock *xs;
+
+ if (umem->need_wakeup & XDP_WAKEUP_TX)
+ return;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(xs, &umem->xsk_list, list) {
+ xs->tx->ring->flags |= XDP_RING_NEED_WAKEUP;
+ }
+ rcu_read_unlock();
+
+ umem->need_wakeup |= XDP_WAKEUP_TX;
+}
+EXPORT_SYMBOL(xsk_set_tx_need_wakeup);
+
+void xsk_clear_rx_need_wakeup(struct xdp_umem *umem)
+{
+ if (!(umem->need_wakeup & XDP_WAKEUP_RX))
+ return;
+
+ umem->fq->ring->flags &= ~XDP_RING_NEED_WAKEUP;
+ umem->need_wakeup &= ~XDP_WAKEUP_RX;
+}
+EXPORT_SYMBOL(xsk_clear_rx_need_wakeup);
+
+void xsk_clear_tx_need_wakeup(struct xdp_umem *umem)
+{
+ struct xdp_sock *xs;
+
+ if (!(umem->need_wakeup & XDP_WAKEUP_TX))
+ return;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(xs, &umem->xsk_list, list) {
+ xs->tx->ring->flags &= ~XDP_RING_NEED_WAKEUP;
+ }
+ rcu_read_unlock();
+
+ umem->need_wakeup &= ~XDP_WAKEUP_TX;
+}
+EXPORT_SYMBOL(xsk_clear_tx_need_wakeup);
+
+bool xsk_umem_uses_need_wakeup(struct xdp_umem *umem)
+{
+ return umem->flags & XDP_UMEM_USES_NEED_WAKEUP;
+}
+EXPORT_SYMBOL(xsk_umem_uses_need_wakeup);
+
+/* If a buffer crosses a page boundary, we need to do 2 memcpy's, one for
+ * each page. This is only required in copy mode.
+ */
+static void __xsk_rcv_memcpy(struct xdp_umem *umem, u64 addr, void *from_buf,
+ u32 len, u32 metalen)
+{
+ void *to_buf = xdp_umem_get_data(umem, addr);
+
+ addr = xsk_umem_add_offset_to_addr(addr);
+ if (xskq_crosses_non_contig_pg(umem, addr, len + metalen)) {
+ void *next_pg_addr = umem->pages[(addr >> PAGE_SHIFT) + 1].addr;
+ u64 page_start = addr & ~(PAGE_SIZE - 1);
+ u64 first_len = PAGE_SIZE - (addr - page_start);
+
+ memcpy(to_buf, from_buf, first_len + metalen);
+ memcpy(next_pg_addr, from_buf + first_len, len - first_len);
+
+ return;
+ }
+
+ memcpy(to_buf, from_buf, len + metalen);
+}
+
static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
{
- void *to_buf, *from_buf;
+ u64 offset = xs->umem->headroom;
+ u64 addr, memcpy_addr;
+ void *from_buf;
u32 metalen;
- u64 addr;
int err;
- if (!xskq_peek_addr(xs->umem->fq, &addr) ||
+ if (!xskq_peek_addr(xs->umem->fq, &addr, xs->umem) ||
len > xs->umem->chunk_size_nohr - XDP_PACKET_HEADROOM) {
xs->rx_dropped++;
return -ENOSPC;
}
- addr += xs->umem->headroom;
-
if (unlikely(xdp_data_meta_unsupported(xdp))) {
from_buf = xdp->data;
metalen = 0;
@@ -78,9 +160,11 @@ static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
metalen = xdp->data - xdp->data_meta;
}
- to_buf = xdp_umem_get_data(xs->umem, addr);
- memcpy(to_buf, from_buf, len + metalen);
- addr += metalen;
+ memcpy_addr = xsk_umem_adjust_offset(xs->umem, addr, offset);
+ __xsk_rcv_memcpy(xs->umem, memcpy_addr, from_buf, len, metalen);
+
+ offset += metalen;
+ addr = xsk_umem_adjust_offset(xs->umem, addr, offset);
err = xskq_produce_batch_desc(xs->rx, addr, len);
if (!err) {
xskq_discard_addr(xs->umem->fq);
@@ -102,10 +186,23 @@ static int __xsk_rcv_zc(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
return err;
}
+static bool xsk_is_bound(struct xdp_sock *xs)
+{
+ if (READ_ONCE(xs->state) == XSK_BOUND) {
+ /* Matches smp_wmb() in bind(). */
+ smp_rmb();
+ return true;
+ }
+ return false;
+}
+
int xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp)
{
u32 len;
+ if (!xsk_is_bound(xs))
+ return -EINVAL;
+
if (xs->dev != xdp->rxq->dev || xs->queue_id != xdp->rxq->queue_index)
return -EINVAL;
@@ -125,6 +222,7 @@ int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp)
{
u32 metalen = xdp->data - xdp->data_meta;
u32 len = xdp->data_end - xdp->data;
+ u64 offset = xs->umem->headroom;
void *buffer;
u64 addr;
int err;
@@ -136,17 +234,17 @@ int xsk_generic_rcv(struct xdp_sock *xs, struct xdp_buff *xdp)
goto out_unlock;
}
- if (!xskq_peek_addr(xs->umem->fq, &addr) ||
+ if (!xskq_peek_addr(xs->umem->fq, &addr, xs->umem) ||
len > xs->umem->chunk_size_nohr - XDP_PACKET_HEADROOM) {
err = -ENOSPC;
goto out_drop;
}
- addr += xs->umem->headroom;
-
+ addr = xsk_umem_adjust_offset(xs->umem, addr, offset);
buffer = xdp_umem_get_data(xs->umem, addr);
memcpy(buffer, xdp->data_meta, len + metalen);
- addr += metalen;
+
+ addr = xsk_umem_adjust_offset(xs->umem, addr, metalen);
err = xskq_produce_batch_desc(xs->rx, addr, len);
if (err)
goto out_drop;
@@ -190,7 +288,7 @@ bool xsk_umem_consume_tx(struct xdp_umem *umem, struct xdp_desc *desc)
rcu_read_lock();
list_for_each_entry_rcu(xs, &umem->xsk_list, list) {
- if (!xskq_peek_desc(xs->tx, desc))
+ if (!xskq_peek_desc(xs->tx, desc, umem))
continue;
if (xskq_produce_addr_lazy(umem->cq, desc->addr))
@@ -212,7 +310,8 @@ static int xsk_zc_xmit(struct sock *sk)
struct xdp_sock *xs = xdp_sk(sk);
struct net_device *dev = xs->dev;
- return dev->netdev_ops->ndo_xsk_async_xmit(dev, xs->queue_id);
+ return dev->netdev_ops->ndo_xsk_wakeup(dev, xs->queue_id,
+ XDP_WAKEUP_TX);
}
static void xsk_destruct_skb(struct sk_buff *skb)
@@ -243,7 +342,7 @@ static int xsk_generic_xmit(struct sock *sk, struct msghdr *m,
if (xs->queue_id >= xs->dev->real_num_tx_queues)
goto out;
- while (xskq_peek_desc(xs->tx, &desc)) {
+ while (xskq_peek_desc(xs->tx, &desc, xs->umem)) {
char *buffer;
u64 addr;
u32 len;
@@ -272,7 +371,7 @@ static int xsk_generic_xmit(struct sock *sk, struct msghdr *m,
skb->dev = xs->dev;
skb->priority = sk->sk_priority;
skb->mark = sk->sk_mark;
- skb_shinfo(skb)->destructor_arg = (void *)(long)addr;
+ skb_shinfo(skb)->destructor_arg = (void *)(long)desc.addr;
skb->destructor = xsk_destruct_skb;
err = dev_direct_xmit(skb, xs->queue_id);
@@ -301,7 +400,7 @@ static int xsk_sendmsg(struct socket *sock, struct msghdr *m, size_t total_len)
struct sock *sk = sock->sk;
struct xdp_sock *xs = xdp_sk(sk);
- if (unlikely(!xs->dev))
+ if (unlikely(!xsk_is_bound(xs)))
return -ENXIO;
if (unlikely(!(xs->dev->flags & IFF_UP)))
return -ENETDOWN;
@@ -317,8 +416,19 @@ static unsigned int xsk_poll(struct file *file, struct socket *sock,
struct poll_table_struct *wait)
{
unsigned int mask = datagram_poll(file, sock, wait);
- struct sock *sk = sock->sk;
- struct xdp_sock *xs = xdp_sk(sk);
+ struct xdp_sock *xs = xdp_sk(sock->sk);
+ struct net_device *dev;
+ struct xdp_umem *umem;
+
+ if (unlikely(!xsk_is_bound(xs)))
+ return mask;
+
+ dev = xs->dev;
+ umem = xs->umem;
+
+ if (umem->need_wakeup)
+ dev->netdev_ops->ndo_xsk_wakeup(dev, xs->queue_id,
+ umem->need_wakeup);
if (xs->rx && !xskq_empty_desc(xs->rx))
mask |= POLLIN | POLLRDNORM;
@@ -342,7 +452,7 @@ static int xsk_init_queue(u32 entries, struct xsk_queue **queue,
/* Make sure queue is ready before it can be seen by others */
smp_wmb();
- *queue = q;
+ WRITE_ONCE(*queue, q);
return 0;
}
@@ -350,10 +460,9 @@ static void xsk_unbind_dev(struct xdp_sock *xs)
{
struct net_device *dev = xs->dev;
- if (!dev || xs->state != XSK_BOUND)
+ if (xs->state != XSK_BOUND)
return;
-
- xs->state = XSK_UNBOUND;
+ WRITE_ONCE(xs->state, XSK_UNBOUND);
/* Wait for driver to stop using the xdp socket. */
xdp_del_sk_umem(xs->umem, xs);
@@ -362,6 +471,52 @@ static void xsk_unbind_dev(struct xdp_sock *xs)
dev_put(dev);
}
+static struct xsk_map *xsk_get_map_list_entry(struct xdp_sock *xs,
+ struct xdp_sock ***map_entry)
+{
+ struct xsk_map *map = NULL;
+ struct xsk_map_node *node;
+
+ *map_entry = NULL;
+
+ spin_lock_bh(&xs->map_list_lock);
+ node = list_first_entry_or_null(&xs->map_list, struct xsk_map_node,
+ node);
+ if (node) {
+ WARN_ON(xsk_map_inc(node->map));
+ map = node->map;
+ *map_entry = node->map_entry;
+ }
+ spin_unlock_bh(&xs->map_list_lock);
+ return map;
+}
+
+static void xsk_delete_from_maps(struct xdp_sock *xs)
+{
+ /* This function removes the current XDP socket from all the
+ * maps it resides in. We need to take extra care here, due to
+ * the two locks involved. Each map has a lock synchronizing
+ * updates to the entries, and each socket has a lock that
+ * synchronizes access to the list of maps (map_list). For
+ * deadlock avoidance the locks need to be taken in the order
+ * "map lock"->"socket map list lock". We start off by
+ * accessing the socket map list, and take a reference to the
+ * map to guarantee existence between the
+ * xsk_get_map_list_entry() and xsk_map_try_sock_delete()
+ * calls. Then we ask the map to remove the socket, which
+ * tries to remove the socket from the map. Note that there
+ * might be updates to the map between
+ * xsk_get_map_list_entry() and xsk_map_try_sock_delete().
+ */
+ struct xdp_sock **map_entry = NULL;
+ struct xsk_map *map;
+
+ while ((map = xsk_get_map_list_entry(xs, &map_entry))) {
+ xsk_map_try_sock_delete(map, xs, map_entry);
+ xsk_map_put(map);
+ }
+}
+
static int xsk_release(struct socket *sock)
{
struct sock *sk = sock->sk;
@@ -381,7 +536,10 @@ static int xsk_release(struct socket *sock)
sock_prot_inuse_add(net, sk->sk_prot, -1);
local_bh_enable();
+ xsk_delete_from_maps(xs);
+ mutex_lock(&xs->mutex);
xsk_unbind_dev(xs);
+ mutex_unlock(&xs->mutex);
xskq_destroy(xs->rx);
xskq_destroy(xs->tx);
@@ -412,6 +570,24 @@ static struct socket *xsk_lookup_xsk_from_fd(int fd)
return sock;
}
+/* Check if umem pages are contiguous.
+ * If zero-copy mode, use the DMA address to do the page contiguity check
+ * For all other modes we use addr (kernel virtual address)
+ * Store the result in the low bits of addr.
+ */
+static void xsk_check_page_contiguity(struct xdp_umem *umem, u32 flags)
+{
+ struct xdp_umem_page *pgs = umem->pages;
+ int i, is_contig;
+
+ for (i = 0; i < umem->npgs - 1; i++) {
+ is_contig = (flags & XDP_ZEROCOPY) ?
+ (pgs[i].dma + PAGE_SIZE == pgs[i + 1].dma) :
+ (pgs[i].addr + PAGE_SIZE == pgs[i + 1].addr);
+ pgs[i].addr += is_contig << XSK_NEXT_PG_CONTIG_SHIFT;
+ }
+}
+
static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len)
{
struct sockaddr_xdp *sxdp = (struct sockaddr_xdp *)addr;
@@ -427,7 +603,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len)
return -EINVAL;
flags = sxdp->sxdp_flags;
- if (flags & ~(XDP_SHARED_UMEM | XDP_COPY | XDP_ZEROCOPY))
+ if (flags & ~(XDP_SHARED_UMEM | XDP_COPY | XDP_ZEROCOPY |
+ XDP_USE_NEED_WAKEUP))
return -EINVAL;
rtnl_lock();
@@ -454,7 +631,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len)
struct xdp_sock *umem_xs;
struct socket *sock;
- if ((flags & XDP_COPY) || (flags & XDP_ZEROCOPY)) {
+ if ((flags & XDP_COPY) || (flags & XDP_ZEROCOPY) ||
+ (flags & XDP_USE_NEED_WAKEUP)) {
/* Cannot specify flags for shared sockets. */
err = -EINVAL;
goto out_unlock;
@@ -473,19 +651,19 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len)
}
umem_xs = xdp_sk(sock->sk);
- if (!umem_xs->umem) {
- /* No umem to inherit. */
+ if (!xsk_is_bound(umem_xs)) {
err = -EBADF;
sockfd_put(sock);
goto out_unlock;
- } else if (umem_xs->dev != dev || umem_xs->queue_id != qid) {
+ }
+ if (umem_xs->dev != dev || umem_xs->queue_id != qid) {
err = -EINVAL;
sockfd_put(sock);
goto out_unlock;
}
xdp_get_umem(umem_xs->umem);
- xs->umem = umem_xs->umem;
+ WRITE_ONCE(xs->umem, umem_xs->umem);
sockfd_put(sock);
} else if (!xs->umem || !xdp_umem_validate_queues(xs->umem)) {
err = -EINVAL;
@@ -500,6 +678,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len)
err = xdp_umem_assign_dev(xs->umem, dev, qid, flags);
if (err)
goto out_unlock;
+
+ xsk_check_page_contiguity(xs->umem, flags);
}
xs->dev = dev;
@@ -510,16 +690,28 @@ static int xsk_bind(struct socket *sock, struct sockaddr *addr, int addr_len)
xdp_add_sk_umem(xs->umem, xs);
out_unlock:
- if (err)
+ if (err) {
dev_put(dev);
- else
- xs->state = XSK_BOUND;
+ } else {
+ /* Matches smp_rmb() in bind() for shared umem
+ * sockets, and xsk_is_bound().
+ */
+ smp_wmb();
+ WRITE_ONCE(xs->state, XSK_BOUND);
+ }
out_release:
mutex_unlock(&xs->mutex);
rtnl_unlock();
return err;
}
+struct xdp_umem_reg_v1 {
+ __u64 addr; /* Start of packet data area */
+ __u64 len; /* Length of packet data area */
+ __u32 chunk_size;
+ __u32 headroom;
+};
+
static int xsk_setsockopt(struct socket *sock, int level, int optname,
char __user *optval, unsigned int optlen)
{
@@ -549,15 +741,24 @@ static int xsk_setsockopt(struct socket *sock, int level, int optname,
}
q = (optname == XDP_TX_RING) ? &xs->tx : &xs->rx;
err = xsk_init_queue(entries, q, false);
+ if (!err && optname == XDP_TX_RING)
+ /* Tx needs to be explicitly woken up the first time */
+ xs->tx->ring->flags |= XDP_RING_NEED_WAKEUP;
mutex_unlock(&xs->mutex);
return err;
}
case XDP_UMEM_REG:
{
- struct xdp_umem_reg mr;
+ size_t mr_size = sizeof(struct xdp_umem_reg);
+ struct xdp_umem_reg mr = {};
struct xdp_umem *umem;
- if (copy_from_user(&mr, optval, sizeof(mr)))
+ if (optlen < sizeof(struct xdp_umem_reg_v1))
+ return -EINVAL;
+ else if (optlen < sizeof(mr))
+ mr_size = sizeof(struct xdp_umem_reg_v1);
+
+ if (copy_from_user(&mr, optval, mr_size))
return -EFAULT;
mutex_lock(&xs->mutex);
@@ -574,7 +775,7 @@ static int xsk_setsockopt(struct socket *sock, int level, int optname,
/* Make sure umem is ready before it can be seen by others */
smp_wmb();
- xs->umem = umem;
+ WRITE_ONCE(xs->umem, umem);
mutex_unlock(&xs->mutex);
return 0;
}
@@ -610,6 +811,20 @@ static int xsk_setsockopt(struct socket *sock, int level, int optname,
return -ENOPROTOOPT;
}
+static void xsk_enter_rxtx_offsets(struct xdp_ring_offset_v1 *ring)
+{
+ ring->producer = offsetof(struct xdp_rxtx_ring, ptrs.producer);
+ ring->consumer = offsetof(struct xdp_rxtx_ring, ptrs.consumer);
+ ring->desc = offsetof(struct xdp_rxtx_ring, desc);
+}
+
+static void xsk_enter_umem_offsets(struct xdp_ring_offset_v1 *ring)
+{
+ ring->producer = offsetof(struct xdp_umem_ring, ptrs.producer);
+ ring->consumer = offsetof(struct xdp_umem_ring, ptrs.consumer);
+ ring->desc = offsetof(struct xdp_umem_ring, desc);
+}
+
static int xsk_getsockopt(struct socket *sock, int level, int optname,
char __user *optval, int __user *optlen)
{
@@ -649,26 +864,49 @@ static int xsk_getsockopt(struct socket *sock, int level, int optname,
case XDP_MMAP_OFFSETS:
{
struct xdp_mmap_offsets off;
+ struct xdp_mmap_offsets_v1 off_v1;
+ bool flags_supported = true;
+ void *to_copy;
- if (len < sizeof(off))
+ if (len < sizeof(off_v1))
return -EINVAL;
+ else if (len < sizeof(off))
+ flags_supported = false;
- off.rx.producer = offsetof(struct xdp_rxtx_ring, ptrs.producer);
- off.rx.consumer = offsetof(struct xdp_rxtx_ring, ptrs.consumer);
- off.rx.desc = offsetof(struct xdp_rxtx_ring, desc);
- off.tx.producer = offsetof(struct xdp_rxtx_ring, ptrs.producer);
- off.tx.consumer = offsetof(struct xdp_rxtx_ring, ptrs.consumer);
- off.tx.desc = offsetof(struct xdp_rxtx_ring, desc);
+ if (flags_supported) {
+ /* xdp_ring_offset is identical to xdp_ring_offset_v1
+ * except for the flags field added to the end.
+ */
+ xsk_enter_rxtx_offsets((struct xdp_ring_offset_v1 *)
+ &off.rx);
+ xsk_enter_rxtx_offsets((struct xdp_ring_offset_v1 *)
+ &off.tx);
+ xsk_enter_umem_offsets((struct xdp_ring_offset_v1 *)
+ &off.fr);
+ xsk_enter_umem_offsets((struct xdp_ring_offset_v1 *)
+ &off.cr);
+ off.rx.flags = offsetof(struct xdp_rxtx_ring,
+ ptrs.flags);
+ off.tx.flags = offsetof(struct xdp_rxtx_ring,
+ ptrs.flags);
+ off.fr.flags = offsetof(struct xdp_umem_ring,
+ ptrs.flags);
+ off.cr.flags = offsetof(struct xdp_umem_ring,
+ ptrs.flags);
- off.fr.producer = offsetof(struct xdp_umem_ring, ptrs.producer);
- off.fr.consumer = offsetof(struct xdp_umem_ring, ptrs.consumer);
- off.fr.desc = offsetof(struct xdp_umem_ring, desc);
- off.cr.producer = offsetof(struct xdp_umem_ring, ptrs.producer);
- off.cr.consumer = offsetof(struct xdp_umem_ring, ptrs.consumer);
- off.cr.desc = offsetof(struct xdp_umem_ring, desc);
+ len = sizeof(off);
+ to_copy = &off;
+ } else {
+ xsk_enter_rxtx_offsets(&off_v1.rx);
+ xsk_enter_rxtx_offsets(&off_v1.tx);
+ xsk_enter_umem_offsets(&off_v1.fr);
+ xsk_enter_umem_offsets(&off_v1.cr);
- len = sizeof(off);
- if (copy_to_user(optval, &off, len))
+ len = sizeof(off_v1);
+ to_copy = &off_v1;
+ }
+
+ if (copy_to_user(optval, to_copy, len))
return -EFAULT;
if (put_user(len, optlen))
return -EFAULT;
@@ -713,7 +951,7 @@ static int xsk_mmap(struct file *file, struct socket *sock,
unsigned long pfn;
struct page *qpg;
- if (xs->state != XSK_READY)
+ if (READ_ONCE(xs->state) != XSK_READY)
return -EBUSY;
if (offset == XDP_PGOFF_RX_RING) {
@@ -855,6 +1093,9 @@ static int xsk_create(struct net *net, struct socket *sock, int protocol,
spin_lock_init(&xs->rx_lock);
spin_lock_init(&xs->tx_completion_lock);
+ INIT_LIST_HEAD(&xs->map_list);
+ spin_lock_init(&xs->map_list_lock);
+
mutex_lock(&net->xdp.lock);
sk_add_node_rcu(sk, &net->xdp.list);
mutex_unlock(&net->xdp.lock);
diff --git a/net/xdp/xsk.h b/net/xdp/xsk.h
index ba81206..4cfd106 100644
--- a/net/xdp/xsk.h
+++ b/net/xdp/xsk.h
@@ -4,6 +4,19 @@
#ifndef XSK_H_
#define XSK_H_
+struct xdp_ring_offset_v1 {
+ __u64 producer;
+ __u64 consumer;
+ __u64 desc;
+};
+
+struct xdp_mmap_offsets_v1 {
+ struct xdp_ring_offset_v1 rx;
+ struct xdp_ring_offset_v1 tx;
+ struct xdp_ring_offset_v1 fr;
+ struct xdp_ring_offset_v1 cr;
+};
+
static inline struct xdp_sock *xdp_sk(struct sock *sk)
{
return (struct xdp_sock *)sk;
diff --git a/net/xdp/xsk_diag.c b/net/xdp/xsk_diag.c
index d5e06c8..f59791b 100644
--- a/net/xdp/xsk_diag.c
+++ b/net/xdp/xsk_diag.c
@@ -56,7 +56,7 @@ static int xsk_diag_put_umem(const struct xdp_sock *xs, struct sk_buff *nlskb)
du.id = umem->id;
du.size = umem->size;
du.num_pages = umem->npgs;
- du.chunk_size = (__u32)(~umem->chunk_mask + 1);
+ du.chunk_size = umem->chunk_size_nohr + umem->headroom;
du.headroom = umem->headroom;
du.ifindex = umem->dev ? umem->dev->ifindex : 0;
du.queue_id = umem->queue_id;
@@ -97,6 +97,7 @@ static int xsk_diag_fill(struct sock *sk, struct sk_buff *nlskb,
msg->xdiag_ino = sk_ino;
sock_diag_save_cookie(sk, msg->xdiag_cookie);
+ mutex_lock(&xs->mutex);
if ((req->xdiag_show & XDP_SHOW_INFO) && xsk_diag_put_info(xs, nlskb))
goto out_nlmsg_trim;
@@ -117,10 +118,12 @@ static int xsk_diag_fill(struct sock *sk, struct sk_buff *nlskb,
sock_diag_put_meminfo(sk, nlskb, XDP_DIAG_MEMINFO))
goto out_nlmsg_trim;
+ mutex_unlock(&xs->mutex);
nlmsg_end(nlskb, nlh);
return 0;
out_nlmsg_trim:
+ mutex_unlock(&xs->mutex);
nlmsg_cancel(nlskb, nlh);
return -EMSGSIZE;
}
diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h
index 909c516..eddae46 100644
--- a/net/xdp/xsk_queue.h
+++ b/net/xdp/xsk_queue.h
@@ -16,6 +16,7 @@
struct xdp_ring {
u32 producer ____cacheline_aligned_in_smp;
u32 consumer ____cacheline_aligned_in_smp;
+ u32 flags;
};
/* Used for the RX and TX queues for packets */
@@ -133,6 +134,17 @@ static inline bool xskq_has_addrs(struct xsk_queue *q, u32 cnt)
/* UMEM queue */
+static inline bool xskq_crosses_non_contig_pg(struct xdp_umem *umem, u64 addr,
+ u64 length)
+{
+ bool cross_pg = (addr & (PAGE_SIZE - 1)) + length > PAGE_SIZE;
+ bool next_pg_contig =
+ (unsigned long)umem->pages[(addr >> PAGE_SHIFT)].addr &
+ XSK_NEXT_PG_CONTIG_MASK;
+
+ return cross_pg && !next_pg_contig;
+}
+
static inline bool xskq_is_valid_addr(struct xsk_queue *q, u64 addr)
{
if (addr >= q->size) {
@@ -143,23 +155,51 @@ static inline bool xskq_is_valid_addr(struct xsk_queue *q, u64 addr)
return true;
}
-static inline u64 *xskq_validate_addr(struct xsk_queue *q, u64 *addr)
+static inline bool xskq_is_valid_addr_unaligned(struct xsk_queue *q, u64 addr,
+ u64 length,
+ struct xdp_umem *umem)
+{
+ u64 base_addr = xsk_umem_extract_addr(addr);
+
+ addr = xsk_umem_add_offset_to_addr(addr);
+ if (base_addr >= q->size || addr >= q->size ||
+ xskq_crosses_non_contig_pg(umem, addr, length)) {
+ q->invalid_descs++;
+ return false;
+ }
+
+ return true;
+}
+
+static inline u64 *xskq_validate_addr(struct xsk_queue *q, u64 *addr,
+ struct xdp_umem *umem)
{
while (q->cons_tail != q->cons_head) {
struct xdp_umem_ring *ring = (struct xdp_umem_ring *)q->ring;
unsigned int idx = q->cons_tail & q->ring_mask;
*addr = READ_ONCE(ring->desc[idx]) & q->chunk_mask;
+
+ if (umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG) {
+ if (xskq_is_valid_addr_unaligned(q, *addr,
+ umem->chunk_size_nohr,
+ umem))
+ return addr;
+ goto out;
+ }
+
if (xskq_is_valid_addr(q, *addr))
return addr;
+out:
q->cons_tail++;
}
return NULL;
}
-static inline u64 *xskq_peek_addr(struct xsk_queue *q, u64 *addr)
+static inline u64 *xskq_peek_addr(struct xsk_queue *q, u64 *addr,
+ struct xdp_umem *umem)
{
if (q->cons_tail == q->cons_head) {
smp_mb(); /* D, matches A */
@@ -170,7 +210,7 @@ static inline u64 *xskq_peek_addr(struct xsk_queue *q, u64 *addr)
smp_rmb();
}
- return xskq_validate_addr(q, addr);
+ return xskq_validate_addr(q, addr, umem);
}
static inline void xskq_discard_addr(struct xsk_queue *q)
@@ -229,8 +269,21 @@ static inline int xskq_reserve_addr(struct xsk_queue *q)
/* Rx/Tx queue */
-static inline bool xskq_is_valid_desc(struct xsk_queue *q, struct xdp_desc *d)
+static inline bool xskq_is_valid_desc(struct xsk_queue *q, struct xdp_desc *d,
+ struct xdp_umem *umem)
{
+ if (umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG) {
+ if (!xskq_is_valid_addr_unaligned(q, d->addr, d->len, umem))
+ return false;
+
+ if (d->len > umem->chunk_size_nohr || d->options) {
+ q->invalid_descs++;
+ return false;
+ }
+
+ return true;
+ }
+
if (!xskq_is_valid_addr(q, d->addr))
return false;
@@ -244,14 +297,15 @@ static inline bool xskq_is_valid_desc(struct xsk_queue *q, struct xdp_desc *d)
}
static inline struct xdp_desc *xskq_validate_desc(struct xsk_queue *q,
- struct xdp_desc *desc)
+ struct xdp_desc *desc,
+ struct xdp_umem *umem)
{
while (q->cons_tail != q->cons_head) {
struct xdp_rxtx_ring *ring = (struct xdp_rxtx_ring *)q->ring;
unsigned int idx = q->cons_tail & q->ring_mask;
*desc = READ_ONCE(ring->desc[idx]);
- if (xskq_is_valid_desc(q, desc))
+ if (xskq_is_valid_desc(q, desc, umem))
return desc;
q->cons_tail++;
@@ -261,7 +315,8 @@ static inline struct xdp_desc *xskq_validate_desc(struct xsk_queue *q,
}
static inline struct xdp_desc *xskq_peek_desc(struct xsk_queue *q,
- struct xdp_desc *desc)
+ struct xdp_desc *desc,
+ struct xdp_umem *umem)
{
if (q->cons_tail == q->cons_head) {
smp_mb(); /* D, matches A */
@@ -272,7 +327,7 @@ static inline struct xdp_desc *xskq_peek_desc(struct xsk_queue *q,
smp_rmb(); /* C, matches B */
}
- return xskq_validate_desc(q, desc);
+ return xskq_validate_desc(q, desc, umem);
}
static inline void xskq_discard_desc(struct xsk_queue *q)
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 8ca637a..ec94f57 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -3269,7 +3269,7 @@ decode_session4(struct sk_buff *skb, struct flowi *fl, bool reverse)
struct flowi4 *fl4 = &fl->u.ip4;
int oif = 0;
- if (skb_dst(skb))
+ if (skb_dst(skb) && skb_dst(skb)->dev)
oif = skb_dst(skb)->dev->ifindex;
memset(fl4, 0, sizeof(struct flowi4));
@@ -3387,7 +3387,7 @@ decode_session6(struct sk_buff *skb, struct flowi *fl, bool reverse)
nexthdr = nh[nhoff];
- if (skb_dst(skb))
+ if (skb_dst(skb) && skb_dst(skb)->dev)
oif = skb_dst(skb)->dev->ifindex;
memset(fl6, 0, sizeof(struct flowi6));
diff --git a/samples/bpf/syscall_nrs.c b/samples/bpf/syscall_nrs.c
index 516e255..88f9400 100644
--- a/samples/bpf/syscall_nrs.c
+++ b/samples/bpf/syscall_nrs.c
@@ -9,5 +9,11 @@ void syscall_defines(void)
COMMENT("Linux system call numbers.");
SYSNR(__NR_write);
SYSNR(__NR_read);
+#ifdef __NR_mmap2
+ SYSNR(__NR_mmap2);
+#endif
+#ifdef __NR_mmap
SYSNR(__NR_mmap);
+#endif
+
}
diff --git a/samples/bpf/tracex5_kern.c b/samples/bpf/tracex5_kern.c
index f57f4e1..35cb0ee 100644
--- a/samples/bpf/tracex5_kern.c
+++ b/samples/bpf/tracex5_kern.c
@@ -68,12 +68,25 @@ PROG(SYS__NR_read)(struct pt_regs *ctx)
return 0;
}
-PROG(SYS__NR_mmap)(struct pt_regs *ctx)
+#ifdef __NR_mmap2
+PROG(SYS__NR_mmap2)(struct pt_regs *ctx)
{
- char fmt[] = "mmap\n";
+ char fmt[] = "mmap2\n";
+
bpf_trace_printk(fmt, sizeof(fmt));
return 0;
}
+#endif
+
+#ifdef __NR_mmap
+PROG(SYS__NR_mmap)(struct pt_regs *ctx)
+{
+ char fmt[] = "mmap\n";
+
+ bpf_trace_printk(fmt, sizeof(fmt));
+ return 0;
+}
+#endif
char _license[] SEC("license") = "GPL";
u32 _version SEC("version") = LINUX_VERSION_CODE;
diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index 93eaaf7..102eace 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -67,8 +67,14 @@ static int opt_ifindex;
static int opt_queue;
static int opt_poll;
static int opt_interval = 1;
+static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP;
+static u32 opt_umem_flags;
+static int opt_unaligned_chunks;
+static int opt_mmap_flags;
static u32 opt_xdp_bind_flags;
static int opt_xsk_frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
+static int opt_timeout = 1000;
+static bool opt_need_wakeup = true;
static __u32 prog_id;
struct xsk_umem_info {
@@ -282,7 +288,9 @@ static struct xsk_umem_info *xsk_configure_umem(void *buffer, u64 size)
.comp_size = XSK_RING_CONS__DEFAULT_NUM_DESCS,
.frame_size = opt_xsk_frame_size,
.frame_headroom = XSK_UMEM__DEFAULT_FRAME_HEADROOM,
+ .flags = opt_umem_flags
};
+
int ret;
umem = calloc(1, sizeof(*umem));
@@ -291,6 +299,7 @@ static struct xsk_umem_info *xsk_configure_umem(void *buffer, u64 size)
ret = xsk_umem__create(&umem->umem, buffer, size, &umem->fq, &umem->cq,
&cfg);
+
if (ret)
exit_with_error(-ret);
@@ -352,6 +361,8 @@ static struct option long_options[] = {
{"zero-copy", no_argument, 0, 'z'},
{"copy", no_argument, 0, 'c'},
{"frame-size", required_argument, 0, 'f'},
+ {"no-need-wakeup", no_argument, 0, 'm'},
+ {"unaligned", no_argument, 0, 'u'},
{0, 0, 0, 0}
};
@@ -372,6 +383,9 @@ static void usage(const char *prog)
" -z, --zero-copy Force zero-copy mode.\n"
" -c, --copy Force copy mode.\n"
" -f, --frame-size=n Set the frame size (must be a power of two, default is %d).\n"
+ " -m, --no-need-wakeup Turn off use of driver need wakeup flag.\n"
+ " -f, --frame-size=n Set the frame size (must be a power of two in aligned mode, default is %d).\n"
+ " -u, --unaligned Enable unaligned chunk placement\n"
"\n";
fprintf(stderr, str, prog, XSK_UMEM__DEFAULT_FRAME_SIZE);
exit(EXIT_FAILURE);
@@ -384,8 +398,8 @@ static void parse_command_line(int argc, char **argv)
opterr = 0;
for (;;) {
- c = getopt_long(argc, argv, "Frtli:q:psSNn:czf:", long_options,
- &option_index);
+ c = getopt_long(argc, argv, "Frtli:q:psSNn:czf:mu",
+ long_options, &option_index);
if (c == -1)
break;
@@ -424,12 +438,21 @@ static void parse_command_line(int argc, char **argv)
case 'c':
opt_xdp_bind_flags |= XDP_COPY;
break;
+ case 'u':
+ opt_umem_flags |= XDP_UMEM_UNALIGNED_CHUNK_FLAG;
+ opt_unaligned_chunks = 1;
+ opt_mmap_flags = MAP_HUGETLB;
+ break;
case 'F':
opt_xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST;
break;
case 'f':
opt_xsk_frame_size = atoi(optarg);
+ case 'm':
+ opt_need_wakeup = false;
+ opt_xdp_bind_flags &= ~XDP_USE_NEED_WAKEUP;
break;
+
default:
usage(basename(argv[0]));
}
@@ -442,7 +465,8 @@ static void parse_command_line(int argc, char **argv)
usage(basename(argv[0]));
}
- if (opt_xsk_frame_size & (opt_xsk_frame_size - 1)) {
+ if ((opt_xsk_frame_size & (opt_xsk_frame_size - 1)) &&
+ !opt_unaligned_chunks) {
fprintf(stderr, "--frame-size=%d is not a power of two\n",
opt_xsk_frame_size);
usage(basename(argv[0]));
@@ -459,8 +483,10 @@ static void kick_tx(struct xsk_socket_info *xsk)
exit_with_error(errno);
}
-static inline void complete_tx_l2fwd(struct xsk_socket_info *xsk)
+static inline void complete_tx_l2fwd(struct xsk_socket_info *xsk,
+ struct pollfd *fds)
{
+ struct xsk_umem_info *umem = xsk->umem;
u32 idx_cq = 0, idx_fq = 0;
unsigned int rcvd;
size_t ndescs;
@@ -468,27 +494,30 @@ static inline void complete_tx_l2fwd(struct xsk_socket_info *xsk)
if (!xsk->outstanding_tx)
return;
- kick_tx(xsk);
+ if (!opt_need_wakeup || xsk_ring_prod__needs_wakeup(&xsk->tx))
+ kick_tx(xsk);
+
ndescs = (xsk->outstanding_tx > BATCH_SIZE) ? BATCH_SIZE :
xsk->outstanding_tx;
/* re-add completed Tx buffers */
- rcvd = xsk_ring_cons__peek(&xsk->umem->cq, ndescs, &idx_cq);
+ rcvd = xsk_ring_cons__peek(&umem->cq, ndescs, &idx_cq);
if (rcvd > 0) {
unsigned int i;
int ret;
- ret = xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
+ ret = xsk_ring_prod__reserve(&umem->fq, rcvd, &idx_fq);
while (ret != rcvd) {
if (ret < 0)
exit_with_error(-ret);
- ret = xsk_ring_prod__reserve(&xsk->umem->fq, rcvd,
- &idx_fq);
+ if (xsk_ring_prod__needs_wakeup(&umem->fq))
+ ret = poll(fds, num_socks, opt_timeout);
+ ret = xsk_ring_prod__reserve(&umem->fq, rcvd, &idx_fq);
}
+
for (i = 0; i < rcvd; i++)
- *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) =
- *xsk_ring_cons__comp_addr(&xsk->umem->cq,
- idx_cq++);
+ *xsk_ring_prod__fill_addr(&umem->fq, idx_fq++) =
+ *xsk_ring_cons__comp_addr(&umem->cq, idx_cq++);
xsk_ring_prod__submit(&xsk->umem->fq, rcvd);
xsk_ring_cons__release(&xsk->umem->cq, rcvd);
@@ -505,7 +534,8 @@ static inline void complete_tx_only(struct xsk_socket_info *xsk)
if (!xsk->outstanding_tx)
return;
- kick_tx(xsk);
+ if (!opt_need_wakeup || xsk_ring_prod__needs_wakeup(&xsk->tx))
+ kick_tx(xsk);
rcvd = xsk_ring_cons__peek(&xsk->umem->cq, BATCH_SIZE, &idx);
if (rcvd > 0) {
@@ -515,30 +545,38 @@ static inline void complete_tx_only(struct xsk_socket_info *xsk)
}
}
-static void rx_drop(struct xsk_socket_info *xsk)
+static void rx_drop(struct xsk_socket_info *xsk, struct pollfd *fds)
{
unsigned int rcvd, i;
u32 idx_rx = 0, idx_fq = 0;
int ret;
rcvd = xsk_ring_cons__peek(&xsk->rx, BATCH_SIZE, &idx_rx);
- if (!rcvd)
+ if (!rcvd) {
+ if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq))
+ ret = poll(fds, num_socks, opt_timeout);
return;
+ }
ret = xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
while (ret != rcvd) {
if (ret < 0)
exit_with_error(-ret);
+ if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq))
+ ret = poll(fds, num_socks, opt_timeout);
ret = xsk_ring_prod__reserve(&xsk->umem->fq, rcvd, &idx_fq);
}
for (i = 0; i < rcvd; i++) {
u64 addr = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx)->addr;
u32 len = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx++)->len;
+ u64 orig = xsk_umem__extract_addr(addr);
+
+ addr = xsk_umem__add_offset_to_addr(addr);
char *pkt = xsk_umem__get_data(xsk->umem->buffer, addr);
hex_dump(pkt, len, addr);
- *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = addr;
+ *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx_fq++) = orig;
}
xsk_ring_prod__submit(&xsk->umem->fq, rcvd);
@@ -549,42 +587,65 @@ static void rx_drop(struct xsk_socket_info *xsk)
static void rx_drop_all(void)
{
struct pollfd fds[MAX_SOCKS + 1];
- int i, ret, timeout, nfds = 1;
+ int i, ret;
memset(fds, 0, sizeof(fds));
for (i = 0; i < num_socks; i++) {
fds[i].fd = xsk_socket__fd(xsks[i]->xsk);
fds[i].events = POLLIN;
- timeout = 1000; /* 1sn */
}
for (;;) {
if (opt_poll) {
- ret = poll(fds, nfds, timeout);
+ ret = poll(fds, num_socks, opt_timeout);
if (ret <= 0)
continue;
}
for (i = 0; i < num_socks; i++)
- rx_drop(xsks[i]);
+ rx_drop(xsks[i], fds);
}
}
-static void tx_only(struct xsk_socket_info *xsk)
+static void tx_only(struct xsk_socket_info *xsk, u32 frame_nb)
{
- int timeout, ret, nfds = 1;
- struct pollfd fds[nfds + 1];
- u32 idx, frame_nb = 0;
+ u32 idx;
+
+ if (xsk_ring_prod__reserve(&xsk->tx, BATCH_SIZE, &idx) == BATCH_SIZE) {
+ unsigned int i;
+
+ for (i = 0; i < BATCH_SIZE; i++) {
+ xsk_ring_prod__tx_desc(&xsk->tx, idx + i)->addr =
+ (frame_nb + i) << XSK_UMEM__DEFAULT_FRAME_SHIFT;
+ xsk_ring_prod__tx_desc(&xsk->tx, idx + i)->len =
+ sizeof(pkt_data) - 1;
+ }
+
+ xsk_ring_prod__submit(&xsk->tx, BATCH_SIZE);
+ xsk->outstanding_tx += BATCH_SIZE;
+ frame_nb += BATCH_SIZE;
+ frame_nb %= NUM_FRAMES;
+ }
+
+ complete_tx_only(xsk);
+}
+
+static void tx_only_all(void)
+{
+ struct pollfd fds[MAX_SOCKS];
+ u32 frame_nb[MAX_SOCKS] = {};
+ int i, ret;
memset(fds, 0, sizeof(fds));
- fds[0].fd = xsk_socket__fd(xsk->xsk);
- fds[0].events = POLLOUT;
- timeout = 1000; /* 1sn */
+ for (i = 0; i < num_socks; i++) {
+ fds[0].fd = xsk_socket__fd(xsks[i]->xsk);
+ fds[0].events = POLLOUT;
+ }
for (;;) {
if (opt_poll) {
- ret = poll(fds, nfds, timeout);
+ ret = poll(fds, num_socks, opt_timeout);
if (ret <= 0)
continue;
@@ -592,69 +653,78 @@ static void tx_only(struct xsk_socket_info *xsk)
continue;
}
- if (xsk_ring_prod__reserve(&xsk->tx, BATCH_SIZE, &idx) ==
- BATCH_SIZE) {
- unsigned int i;
-
- for (i = 0; i < BATCH_SIZE; i++) {
- xsk_ring_prod__tx_desc(&xsk->tx, idx + i)->addr
- = (frame_nb + i) * opt_xsk_frame_size;
- xsk_ring_prod__tx_desc(&xsk->tx, idx + i)->len =
- sizeof(pkt_data) - 1;
- }
-
- xsk_ring_prod__submit(&xsk->tx, BATCH_SIZE);
- xsk->outstanding_tx += BATCH_SIZE;
- frame_nb += BATCH_SIZE;
- frame_nb %= NUM_FRAMES;
- }
-
- complete_tx_only(xsk);
+ for (i = 0; i < num_socks; i++)
+ tx_only(xsks[i], frame_nb[i]);
}
}
-static void l2fwd(struct xsk_socket_info *xsk)
+static void l2fwd(struct xsk_socket_info *xsk, struct pollfd *fds)
{
- for (;;) {
- unsigned int rcvd, i;
- u32 idx_rx = 0, idx_tx = 0;
- int ret;
+ unsigned int rcvd, i;
+ u32 idx_rx = 0, idx_tx = 0;
+ int ret;
- for (;;) {
- complete_tx_l2fwd(xsk);
+ complete_tx_l2fwd(xsk, fds);
- rcvd = xsk_ring_cons__peek(&xsk->rx, BATCH_SIZE,
- &idx_rx);
- if (rcvd > 0)
- break;
- }
+ rcvd = xsk_ring_cons__peek(&xsk->rx, BATCH_SIZE, &idx_rx);
+ if (!rcvd) {
+ if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq))
+ ret = poll(fds, num_socks, opt_timeout);
+ return;
+ }
+ ret = xsk_ring_prod__reserve(&xsk->tx, rcvd, &idx_tx);
+ while (ret != rcvd) {
+ if (ret < 0)
+ exit_with_error(-ret);
+ if (xsk_ring_prod__needs_wakeup(&xsk->tx))
+ kick_tx(xsk);
ret = xsk_ring_prod__reserve(&xsk->tx, rcvd, &idx_tx);
- while (ret != rcvd) {
- if (ret < 0)
- exit_with_error(-ret);
- ret = xsk_ring_prod__reserve(&xsk->tx, rcvd, &idx_tx);
+ }
+
+ for (i = 0; i < rcvd; i++) {
+ u64 addr = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx)->addr;
+ u32 len = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx++)->len;
+ u64 orig = xsk_umem__extract_addr(addr);
+
+ addr = xsk_umem__add_offset_to_addr(addr);
+ char *pkt = xsk_umem__get_data(xsk->umem->buffer, addr);
+
+ swap_mac_addresses(pkt);
+
+ hex_dump(pkt, len, addr);
+ xsk_ring_prod__tx_desc(&xsk->tx, idx_tx)->addr = orig;
+ xsk_ring_prod__tx_desc(&xsk->tx, idx_tx++)->len = len;
+ }
+
+ xsk_ring_prod__submit(&xsk->tx, rcvd);
+ xsk_ring_cons__release(&xsk->rx, rcvd);
+
+ xsk->rx_npkts += rcvd;
+ xsk->outstanding_tx += rcvd;
+}
+
+static void l2fwd_all(void)
+{
+ struct pollfd fds[MAX_SOCKS];
+ int i, ret;
+
+ memset(fds, 0, sizeof(fds));
+
+ for (i = 0; i < num_socks; i++) {
+ fds[i].fd = xsk_socket__fd(xsks[i]->xsk);
+ fds[i].events = POLLOUT | POLLIN;
+ }
+
+ for (;;) {
+ if (opt_poll) {
+ ret = poll(fds, num_socks, opt_timeout);
+ if (ret <= 0)
+ continue;
}
- for (i = 0; i < rcvd; i++) {
- u64 addr = xsk_ring_cons__rx_desc(&xsk->rx,
- idx_rx)->addr;
- u32 len = xsk_ring_cons__rx_desc(&xsk->rx,
- idx_rx++)->len;
- char *pkt = xsk_umem__get_data(xsk->umem->buffer, addr);
-
- swap_mac_addresses(pkt);
-
- hex_dump(pkt, len, addr);
- xsk_ring_prod__tx_desc(&xsk->tx, idx_tx)->addr = addr;
- xsk_ring_prod__tx_desc(&xsk->tx, idx_tx++)->len = len;
- }
-
- xsk_ring_prod__submit(&xsk->tx, rcvd);
- xsk_ring_cons__release(&xsk->rx, rcvd);
-
- xsk->rx_npkts += rcvd;
- xsk->outstanding_tx += rcvd;
+ for (i = 0; i < num_socks; i++)
+ l2fwd(xsks[i], fds);
}
}
@@ -674,11 +744,14 @@ int main(int argc, char **argv)
exit(EXIT_FAILURE);
}
- ret = posix_memalign(&bufs, getpagesize(), /* PAGE_SIZE aligned */
- NUM_FRAMES * opt_xsk_frame_size);
- if (ret)
- exit_with_error(ret);
-
+ /* Reserve memory for the umem. Use hugepages if unaligned chunk mode */
+ bufs = mmap(NULL, NUM_FRAMES * opt_xsk_frame_size,
+ PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS | opt_mmap_flags, -1, 0);
+ if (bufs == MAP_FAILED) {
+ printf("ERROR: mmap failed\n");
+ exit(EXIT_FAILURE);
+ }
/* Create sockets... */
umem = xsk_configure_umem(bufs, NUM_FRAMES * opt_xsk_frame_size);
xsks[num_socks++] = xsk_configure_socket(umem);
@@ -705,9 +778,9 @@ int main(int argc, char **argv)
if (opt_bench == BENCH_RXDROP)
rx_drop_all();
else if (opt_bench == BENCH_TXONLY)
- tx_only(xsks[0]);
+ tx_only_all();
else
- l2fwd(xsks[0]);
+ l2fwd_all();
return 0;
}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index c311933..0d8f41d 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -115,10 +115,12 @@
LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
# dump .BTF section into raw binary file to link with final vmlinux
- bin_arch=$(${OBJDUMP} -f ${1} | grep architecture | \
+ bin_arch=$(LANG=C ${OBJDUMP} -f ${1} | grep architecture | \
cut -d, -f1 | cut -d' ' -f2)
+ bin_format=$(LANG=C ${OBJDUMP} -f ${1} | grep 'file format' | \
+ awk '{print $4}')
${OBJCOPY} --dump-section .BTF=.btf.vmlinux.bin ${1} 2>/dev/null
- ${OBJCOPY} -I binary -O ${CONFIG_OUTPUT_FORMAT} -B ${bin_arch} \
+ ${OBJCOPY} -I binary -O ${bin_format} -B ${bin_arch} \
--rename-section .data=.BTF .btf.vmlinux.bin ${2}
}
diff --git a/security/keys/request_key.c b/security/keys/request_key.c
index 7325f38..957b9e3 100644
--- a/security/keys/request_key.c
+++ b/security/keys/request_key.c
@@ -595,7 +595,7 @@ struct key *request_key_and_link(struct key_type *type,
key = check_cached_key(&ctx);
if (key)
- return key;
+ goto error_free;
/* search all the process keyrings for a key */
rcu_read_lock();
diff --git a/sound/core/seq/seq_clientmgr.c b/sound/core/seq/seq_clientmgr.c
index 7737b26..6d9592f 100644
--- a/sound/core/seq/seq_clientmgr.c
+++ b/sound/core/seq/seq_clientmgr.c
@@ -1835,8 +1835,7 @@ static int snd_seq_ioctl_get_client_pool(struct snd_seq_client *client,
if (cptr->type == USER_CLIENT) {
info->input_pool = cptr->data.user.fifo_pool_size;
info->input_free = info->input_pool;
- if (cptr->data.user.fifo)
- info->input_free = snd_seq_unused_cells(cptr->data.user.fifo->pool);
+ info->input_free = snd_seq_fifo_unused_cells(cptr->data.user.fifo);
} else {
info->input_pool = 0;
info->input_free = 0;
diff --git a/sound/core/seq/seq_fifo.c b/sound/core/seq/seq_fifo.c
index ea69261..eaaa8b5 100644
--- a/sound/core/seq/seq_fifo.c
+++ b/sound/core/seq/seq_fifo.c
@@ -263,3 +263,20 @@ int snd_seq_fifo_resize(struct snd_seq_fifo *f, int poolsize)
return 0;
}
+
+/* get the number of unused cells safely */
+int snd_seq_fifo_unused_cells(struct snd_seq_fifo *f)
+{
+ unsigned long flags;
+ int cells;
+
+ if (!f)
+ return 0;
+
+ snd_use_lock_use(&f->use_lock);
+ spin_lock_irqsave(&f->lock, flags);
+ cells = snd_seq_unused_cells(f->pool);
+ spin_unlock_irqrestore(&f->lock, flags);
+ snd_use_lock_free(&f->use_lock);
+ return cells;
+}
diff --git a/sound/core/seq/seq_fifo.h b/sound/core/seq/seq_fifo.h
index edc6874..b56a7b8 100644
--- a/sound/core/seq/seq_fifo.h
+++ b/sound/core/seq/seq_fifo.h
@@ -53,5 +53,7 @@ int snd_seq_fifo_poll_wait(struct snd_seq_fifo *f, struct file *file, poll_table
/* resize pool in fifo */
int snd_seq_fifo_resize(struct snd_seq_fifo *f, int poolsize);
+/* get the number of unused cells safely */
+int snd_seq_fifo_unused_cells(struct snd_seq_fifo *f);
#endif
diff --git a/sound/firewire/oxfw/oxfw-pcm.c b/sound/firewire/oxfw/oxfw-pcm.c
index 9ea3934..7c6d1c2 100644
--- a/sound/firewire/oxfw/oxfw-pcm.c
+++ b/sound/firewire/oxfw/oxfw-pcm.c
@@ -248,7 +248,7 @@ static int pcm_playback_hw_params(struct snd_pcm_substream *substream,
unsigned int channels = params_channels(hw_params);
mutex_lock(&oxfw->mutex);
- err = snd_oxfw_stream_reserve_duplex(oxfw, &oxfw->tx_stream,
+ err = snd_oxfw_stream_reserve_duplex(oxfw, &oxfw->rx_stream,
rate, channels);
if (err >= 0)
++oxfw->substreams_count;
diff --git a/sound/pci/hda/patch_ca0132.c b/sound/pci/hda/patch_ca0132.c
index 0d51823..6d1fb7c 100644
--- a/sound/pci/hda/patch_ca0132.c
+++ b/sound/pci/hda/patch_ca0132.c
@@ -1175,6 +1175,7 @@ static const struct snd_pci_quirk ca0132_quirks[] = {
SND_PCI_QUIRK(0x1028, 0x0708, "Alienware 15 R2 2016", QUIRK_ALIENWARE),
SND_PCI_QUIRK(0x1102, 0x0010, "Sound Blaster Z", QUIRK_SBZ),
SND_PCI_QUIRK(0x1102, 0x0023, "Sound Blaster Z", QUIRK_SBZ),
+ SND_PCI_QUIRK(0x1102, 0x0027, "Sound Blaster Z", QUIRK_SBZ),
SND_PCI_QUIRK(0x1102, 0x0033, "Sound Blaster ZxR", QUIRK_SBZ),
SND_PCI_QUIRK(0x1458, 0xA016, "Recon3Di", QUIRK_R3DI),
SND_PCI_QUIRK(0x1458, 0xA026, "Gigabyte G1.Sniper Z97", QUIRK_R3DI),
diff --git a/sound/pci/hda/patch_conexant.c b/sound/pci/hda/patch_conexant.c
index 14298ef..968d3ca 100644
--- a/sound/pci/hda/patch_conexant.c
+++ b/sound/pci/hda/patch_conexant.c
@@ -611,18 +611,20 @@ static void cxt_fixup_hp_gate_mic_jack(struct hda_codec *codec,
/* update LED status via GPIO */
static void cxt_update_gpio_led(struct hda_codec *codec, unsigned int mask,
- bool enabled)
+ bool led_on)
{
struct conexant_spec *spec = codec->spec;
unsigned int oldval = spec->gpio_led;
if (spec->mute_led_polarity)
- enabled = !enabled;
+ led_on = !led_on;
- if (enabled)
- spec->gpio_led &= ~mask;
- else
+ if (led_on)
spec->gpio_led |= mask;
+ else
+ spec->gpio_led &= ~mask;
+ codec_dbg(codec, "mask:%d enabled:%d gpio_led:%d\n",
+ mask, led_on, spec->gpio_led);
if (spec->gpio_led != oldval)
snd_hda_codec_write(codec, 0x01, 0, AC_VERB_SET_GPIO_DATA,
spec->gpio_led);
@@ -633,8 +635,8 @@ static void cxt_fixup_gpio_mute_hook(void *private_data, int enabled)
{
struct hda_codec *codec = private_data;
struct conexant_spec *spec = codec->spec;
-
- cxt_update_gpio_led(codec, spec->gpio_mute_led_mask, enabled);
+ /* muted -> LED on */
+ cxt_update_gpio_led(codec, spec->gpio_mute_led_mask, !enabled);
}
/* turn on/off mic-mute LED via GPIO per capture hook */
@@ -656,7 +658,6 @@ static void cxt_fixup_mute_led_gpio(struct hda_codec *codec,
{ 0x01, AC_VERB_SET_GPIO_DIRECTION, 0x03 },
{}
};
- codec_info(codec, "action: %d gpio_led: %d\n", action, spec->gpio_led);
if (action == HDA_FIXUP_ACT_PRE_PROBE) {
spec->gen.vmaster_mute.hook = cxt_fixup_gpio_mute_hook;
diff --git a/sound/usb/line6/pcm.c b/sound/usb/line6/pcm.c
index 2c03e0f..f70211e 100644
--- a/sound/usb/line6/pcm.c
+++ b/sound/usb/line6/pcm.c
@@ -550,6 +550,15 @@ int line6_init_pcm(struct usb_line6 *line6,
line6pcm->volume_monitor = 255;
line6pcm->line6 = line6;
+ spin_lock_init(&line6pcm->out.lock);
+ spin_lock_init(&line6pcm->in.lock);
+ line6pcm->impulse_period = LINE6_IMPULSE_DEFAULT_PERIOD;
+
+ line6->line6pcm = line6pcm;
+
+ pcm->private_data = line6pcm;
+ pcm->private_free = line6_cleanup_pcm;
+
line6pcm->max_packet_size_in =
usb_maxpacket(line6->usbdev,
usb_rcvisocpipe(line6->usbdev, ep_read), 0);
@@ -562,15 +571,6 @@ int line6_init_pcm(struct usb_line6 *line6,
return -EINVAL;
}
- spin_lock_init(&line6pcm->out.lock);
- spin_lock_init(&line6pcm->in.lock);
- line6pcm->impulse_period = LINE6_IMPULSE_DEFAULT_PERIOD;
-
- line6->line6pcm = line6pcm;
-
- pcm->private_data = line6pcm;
- pcm->private_free = line6_cleanup_pcm;
-
err = line6_create_audio_out_urbs(line6pcm);
if (err < 0)
return err;
diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
index b5927c3..eceab19 100644
--- a/sound/usb/mixer.c
+++ b/sound/usb/mixer.c
@@ -739,7 +739,6 @@ static int uac_mixer_unit_get_channels(struct mixer_build *state,
struct uac_mixer_unit_descriptor *desc)
{
int mu_channels;
- void *c;
if (desc->bLength < sizeof(*desc))
return -EINVAL;
@@ -762,13 +761,6 @@ static int uac_mixer_unit_get_channels(struct mixer_build *state,
break;
}
- if (!mu_channels)
- return 0;
-
- c = uac_mixer_unit_bmControls(desc, state->mixer->protocol);
- if (c - (void *)desc + (mu_channels - 1) / 8 >= desc->bLength)
- return 0; /* no bmControls -> skip */
-
return mu_channels;
}
@@ -2009,6 +2001,31 @@ static int parse_audio_feature_unit(struct mixer_build *state, int unitid,
* Mixer Unit
*/
+/* check whether the given in/out overflows bmMixerControls matrix */
+static bool mixer_bitmap_overflow(struct uac_mixer_unit_descriptor *desc,
+ int protocol, int num_ins, int num_outs)
+{
+ u8 *hdr = (u8 *)desc;
+ u8 *c = uac_mixer_unit_bmControls(desc, protocol);
+ size_t rest; /* remaining bytes after bmMixerControls */
+
+ switch (protocol) {
+ case UAC_VERSION_1:
+ default:
+ rest = 1; /* iMixer */
+ break;
+ case UAC_VERSION_2:
+ rest = 2; /* bmControls + iMixer */
+ break;
+ case UAC_VERSION_3:
+ rest = 6; /* bmControls + wMixerDescrStr */
+ break;
+ }
+
+ /* overflow? */
+ return c + (num_ins * num_outs + 7) / 8 + rest > hdr + hdr[0];
+}
+
/*
* build a mixer unit control
*
@@ -2137,6 +2154,9 @@ static int parse_audio_mixer_unit(struct mixer_build *state, int unitid,
if (err < 0)
return err;
num_ins += iterm.channels;
+ if (mixer_bitmap_overflow(desc, state->mixer->protocol,
+ num_ins, num_outs))
+ break;
for (; ich < num_ins; ich++) {
int och, ich_has_controls = 0;
diff --git a/sound/usb/mixer_quirks.c b/sound/usb/mixer_quirks.c
index 199fa15..27dcb37 100644
--- a/sound/usb/mixer_quirks.c
+++ b/sound/usb/mixer_quirks.c
@@ -1155,17 +1155,17 @@ void snd_emuusb_set_samplerate(struct snd_usb_audio *chip,
{
struct usb_mixer_interface *mixer;
struct usb_mixer_elem_info *cval;
- int unitid = 12; /* SamleRate ExtensionUnit ID */
+ int unitid = 12; /* SampleRate ExtensionUnit ID */
list_for_each_entry(mixer, &chip->mixer_list, list) {
- cval = mixer_elem_list_to_info(mixer->id_elems[unitid]);
- if (cval) {
+ if (mixer->id_elems[unitid]) {
+ cval = mixer_elem_list_to_info(mixer->id_elems[unitid]);
snd_usb_mixer_set_ctl_value(cval, UAC_SET_CUR,
cval->control << 8,
samplerate_id);
snd_usb_mixer_notify_id(mixer, unitid);
+ break;
}
- break;
}
}
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 75b9692..e4bbf79 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -339,6 +339,7 @@ static int set_sync_ep_implicit_fb_quirk(struct snd_usb_substream *subs,
ep = 0x81;
ifnum = 2;
goto add_sync_ep_from_ifnum;
+ case USB_ID(0x1397, 0x0001): /* Behringer UFX1604 */
case USB_ID(0x1397, 0x0002): /* Behringer UFX1204 */
ep = 0x81;
ifnum = 1;
diff --git a/tools/bpf/.gitignore b/tools/bpf/.gitignore
index dfe2bd5..5902419 100644
--- a/tools/bpf/.gitignore
+++ b/tools/bpf/.gitignore
@@ -1,4 +1,5 @@
FEATURE-DUMP.bpf
+feature
bpf_asm
bpf_dbg
bpf_exp.yacc.*
diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
index 53b60ad..fbf5e4a 100644
--- a/tools/bpf/Makefile
+++ b/tools/bpf/Makefile
@@ -81,10 +81,11 @@
clean: bpftool_clean
$(call QUIET_CLEAN, bpf-progs)
- $(Q)rm -rf $(OUTPUT)*.o $(OUTPUT)bpf_jit_disasm $(OUTPUT)bpf_dbg \
+ $(Q)$(RM) -r -- $(OUTPUT)*.o $(OUTPUT)bpf_jit_disasm $(OUTPUT)bpf_dbg \
$(OUTPUT)bpf_asm $(OUTPUT)bpf_exp.yacc.* $(OUTPUT)bpf_exp.lex.*
$(call QUIET_CLEAN, core-gen)
- $(Q)rm -f $(OUTPUT)FEATURE-DUMP.bpf
+ $(Q)$(RM) -- $(OUTPUT)FEATURE-DUMP.bpf
+ $(Q)$(RM) -r -- $(OUTPUT)feature
install: $(PROGS) bpftool_install
$(call QUIET_INSTALL, bpf_jit_disasm)
diff --git a/tools/bpf/bpftool/.gitignore b/tools/bpf/bpftool/.gitignore
index 8248b8d..b139264 100644
--- a/tools/bpf/bpftool/.gitignore
+++ b/tools/bpf/bpftool/.gitignore
@@ -3,3 +3,5 @@
bpftool*.8
bpf-helpers.*
FEATURE-DUMP.bpftool
+feature
+libbpf
diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
index 6694a0fc..39615f8 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
@@ -19,6 +19,7 @@
BTF COMMANDS
=============
+| **bpftool** **btf** { **show** | **list** } [**id** *BTF_ID*]
| **bpftool** **btf dump** *BTF_SRC* [**format** *FORMAT*]
| **bpftool** **btf help**
|
@@ -29,6 +30,12 @@
DESCRIPTION
===========
+ **bpftool btf { show | list }** [**id** *BTF_ID*]
+ Show information about loaded BTF objects. If a BTF ID is
+ specified, show information only about given BTF object,
+ otherwise list all BTF objects currently loaded on the
+ system.
+
**bpftool btf dump** *BTF_SRC*
Dump BTF entries from a given *BTF_SRC*.
diff --git a/tools/bpf/bpftool/Documentation/bpftool-map.rst b/tools/bpf/bpftool/Documentation/bpftool-map.rst
index 61d1d27..1c0f714 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-map.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-map.rst
@@ -36,6 +36,7 @@
| **bpftool** **map pop** *MAP*
| **bpftool** **map enqueue** *MAP* **value** *VALUE*
| **bpftool** **map dequeue** *MAP*
+| **bpftool** **map freeze** *MAP*
| **bpftool** **map help**
|
| *MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
@@ -127,6 +128,14 @@
**bpftool map dequeue** *MAP*
Dequeue and print **value** from the queue.
+ **bpftool map freeze** *MAP*
+ Freeze the map as read-only from user space. Entries from a
+ frozen map can not longer be updated or deleted with the
+ **bpf\ ()** system call. This operation is not reversible,
+ and the map remains immutable from user space until its
+ destruction. However, read and write permissions for BPF
+ programs to the map remain unchanged.
+
**bpftool map help**
Print short help message.
diff --git a/tools/bpf/bpftool/Documentation/bpftool-net.rst b/tools/bpf/bpftool/Documentation/bpftool-net.rst
index d8e5237..8651b00 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-net.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-net.rst
@@ -15,17 +15,22 @@
*OPTIONS* := { [{ **-j** | **--json** }] [{ **-p** | **--pretty** }] }
*COMMANDS* :=
- { **show** | **list** } [ **dev** name ] | **help**
+ { **show** | **list** | **attach** | **detach** | **help** }
NET COMMANDS
============
-| **bpftool** **net { show | list } [ dev name ]**
+| **bpftool** **net { show | list }** [ **dev** *NAME* ]
+| **bpftool** **net attach** *ATTACH_TYPE* *PROG* **dev** *NAME* [ **overwrite** ]
+| **bpftool** **net detach** *ATTACH_TYPE* **dev** *NAME*
| **bpftool** **net help**
+|
+| *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* }
+| *ATTACH_TYPE* := { **xdp** | **xdpgeneric** | **xdpdrv** | **xdpoffload** }
DESCRIPTION
===========
- **bpftool net { show | list } [ dev name ]**
+ **bpftool net { show | list }** [ **dev** *NAME* ]
List bpf program attachments in the kernel networking subsystem.
Currently, only device driver xdp attachments and tc filter
@@ -47,6 +52,24 @@
all bpf programs attached to non clsact qdiscs, and finally all
bpf programs attached to root and clsact qdisc.
+ **bpftool** **net attach** *ATTACH_TYPE* *PROG* **dev** *NAME* [ **overwrite** ]
+ Attach bpf program *PROG* to network interface *NAME* with
+ type specified by *ATTACH_TYPE*. Previously attached bpf program
+ can be replaced by the command used with **overwrite** option.
+ Currently, only XDP-related modes are supported for *ATTACH_TYPE*.
+
+ *ATTACH_TYPE* can be of:
+ **xdp** - try native XDP and fallback to generic XDP if NIC driver does not support it;
+ **xdpgeneric** - Generic XDP. runs at generic XDP hook when packet already enters receive path as skb;
+ **xdpdrv** - Native XDP. runs earliest point in driver's receive path;
+ **xdpoffload** - Offload XDP. runs directly on NIC on each packet reception;
+
+ **bpftool** **net detach** *ATTACH_TYPE* **dev** *NAME*
+ Detach bpf program attached to network interface *NAME* with
+ type specified by *ATTACH_TYPE*. To detach bpf program, same
+ *ATTACH_TYPE* previously used for attach must be specified.
+ Currently, only XDP-related modes are supported for *ATTACH_TYPE*.
+
**bpftool net help**
Print short help message.
@@ -137,6 +160,34 @@
}
]
+|
+| **# bpftool net attach xdpdrv id 16 dev enp6s0np0**
+| **# bpftool net**
+
+::
+
+ xdp:
+ enp6s0np0(4) driver id 16
+
+|
+| **# bpftool net attach xdpdrv id 16 dev enp6s0np0**
+| **# bpftool net attach xdpdrv id 20 dev enp6s0np0 overwrite**
+| **# bpftool net**
+
+::
+
+ xdp:
+ enp6s0np0(4) driver id 20
+
+|
+| **# bpftool net attach xdpdrv id 16 dev enp6s0np0**
+| **# bpftool net detach xdpdrv dev enp6s0np0**
+| **# bpftool net**
+
+::
+
+ xdp:
+
SEE ALSO
========
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 4c9d1ff..39bc6f0 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -17,27 +17,30 @@
BPF_DIR = $(srctree)/tools/lib/bpf/
ifneq ($(OUTPUT),)
- BPF_PATH = $(OUTPUT)
+ LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
+ LIBBPF_PATH = $(LIBBPF_OUTPUT)
else
- BPF_PATH = $(BPF_DIR)
+ LIBBPF_PATH = $(BPF_DIR)
endif
-LIBBPF = $(BPF_PATH)libbpf.a
+LIBBPF = $(LIBBPF_PATH)libbpf.a
-BPFTOOL_VERSION := $(shell make --no-print-directory -sC ../../.. kernelversion)
+BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
$(LIBBPF): FORCE
- $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(OUTPUT) $(OUTPUT)libbpf.a
+ $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
+ $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
$(LIBBPF)-clean:
$(call QUIET_CLEAN, libbpf)
- $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(OUTPUT) clean >/dev/null
+ $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
prefix ?= /usr/local
bash_compdir ?= /usr/share/bash-completion/completions
CFLAGS += -O2
-CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wshadow -Wno-missing-field-initializers
+CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers
+CFLAGS += $(filter-out -Wswitch-enum,$(EXTRA_WARNINGS))
CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \
-I$(srctree)/kernel/bpf/ \
-I$(srctree)/tools/include \
@@ -52,7 +55,7 @@
LDFLAGS += $(EXTRA_LDFLAGS)
endif
-LIBS = -lelf -lz $(LIBBPF)
+LIBS = $(LIBBPF) -lelf -lz
INSTALL ?= install
RM ?= rm -f
@@ -114,16 +117,18 @@
$(OUTPUT)feature.o: | zdep
$(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
- $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $^ $(LIBS)
+ $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(OBJS) $(LIBS)
$(OUTPUT)%.o: %.c
$(QUIET_CC)$(COMPILE.c) -MMD -o $@ $<
clean: $(LIBBPF)-clean
$(call QUIET_CLEAN, bpftool)
- $(Q)$(RM) $(OUTPUT)bpftool $(OUTPUT)*.o $(OUTPUT)*.d
+ $(Q)$(RM) -- $(OUTPUT)bpftool $(OUTPUT)*.o $(OUTPUT)*.d
+ $(Q)$(RM) -r -- $(OUTPUT)libbpf/
$(call QUIET_CLEAN, core-gen)
- $(Q)$(RM) $(OUTPUT)FEATURE-DUMP.bpftool
+ $(Q)$(RM) -- $(OUTPUT)FEATURE-DUMP.bpftool
+ $(Q)$(RM) -r -- $(OUTPUT)feature/
install: $(OUTPUT)bpftool
$(call QUIET_INSTALL, bpftool)
@@ -134,8 +139,8 @@
uninstall:
$(call QUIET_UNINST, bpftool)
- $(Q)$(RM) $(DESTDIR)$(prefix)/sbin/bpftool
- $(Q)$(RM) $(DESTDIR)$(bash_compdir)/bpftool
+ $(Q)$(RM) -- $(DESTDIR)$(prefix)/sbin/bpftool
+ $(Q)$(RM) -- $(DESTDIR)$(bash_compdir)/bpftool
doc:
$(call descend,Documentation)
diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index df16c54..70493a6 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -73,8 +73,8 @@
_bpftool_get_btf_ids()
{
- COMPREPLY+=( $( compgen -W "$( bpftool -jp prog 2>&1 | \
- command sed -n 's/.*"btf_id": \(.*\),\?$/\1/p' )" -- "$cur" ) )
+ COMPREPLY+=( $( compgen -W "$( bpftool -jp btf 2>&1 | \
+ command sed -n 's/.*"id": \(.*\),$/\1/p' )" -- "$cur" ) )
}
_bpftool_get_obj_map_names()
@@ -201,6 +201,10 @@
_bpftool_get_prog_tags
return 0
;;
+ dev)
+ _sysfs_get_netdevs
+ return 0
+ ;;
file|pinned)
_filedir
return 0
@@ -399,10 +403,6 @@
_filedir
return 0
;;
- dev)
- _sysfs_get_netdevs
- return 0
- ;;
*)
COMPREPLY=( $( compgen -W "map" -- "$cur" ) )
_bpftool_once_attr 'type'
@@ -449,7 +449,7 @@
map)
local MAP_TYPE='id pinned'
case $command in
- show|list|dump|peek|pop|dequeue)
+ show|list|dump|peek|pop|dequeue|freeze)
case $prev in
$command)
COMPREPLY=( $( compgen -W "$MAP_TYPE" -- "$cur" ) )
@@ -498,10 +498,6 @@
key|value|flags|name|entries)
return 0
;;
- dev)
- _sysfs_get_netdevs
- return 0
- ;;
*)
_bpftool_once_attr 'type'
_bpftool_once_attr 'key'
@@ -642,7 +638,7 @@
[[ $prev == $object ]] && \
COMPREPLY=( $( compgen -W 'delete dump getnext help \
lookup pin event_pipe show list update create \
- peek push enqueue pop dequeue' -- \
+ peek push enqueue pop dequeue freeze' -- \
"$cur" ) )
;;
esac
@@ -674,7 +670,7 @@
map)
_bpftool_get_map_ids
;;
- dump)
+ $command)
_bpftool_get_btf_ids
;;
esac
@@ -702,9 +698,21 @@
;;
esac
;;
+ show|list)
+ case $prev in
+ $command)
+ COMPREPLY+=( $( compgen -W "id" -- "$cur" ) )
+ ;;
+ id)
+ _bpftool_get_btf_ids
+ ;;
+ esac
+ return 0
+ ;;
*)
[[ $prev == $object ]] && \
- COMPREPLY=( $( compgen -W 'dump help' -- "$cur" ) )
+ COMPREPLY=( $( compgen -W 'dump help show list' \
+ -- "$cur" ) )
;;
esac
;;
@@ -778,18 +786,67 @@
esac
;;
net)
+ local PROG_TYPE='id pinned tag'
+ local ATTACH_TYPES='xdp xdpgeneric xdpdrv xdpoffload'
case $command in
+ show|list)
+ [[ $prev != "$command" ]] && return 0
+ COMPREPLY=( $( compgen -W 'dev' -- "$cur" ) )
+ return 0
+ ;;
+ attach)
+ case $cword in
+ 3)
+ COMPREPLY=( $( compgen -W "$ATTACH_TYPES" -- "$cur" ) )
+ return 0
+ ;;
+ 4)
+ COMPREPLY=( $( compgen -W "$PROG_TYPE" -- "$cur" ) )
+ return 0
+ ;;
+ 5)
+ case $prev in
+ id)
+ _bpftool_get_prog_ids
+ ;;
+ pinned)
+ _filedir
+ ;;
+ esac
+ return 0
+ ;;
+ 6)
+ COMPREPLY=( $( compgen -W 'dev' -- "$cur" ) )
+ return 0
+ ;;
+ 8)
+ _bpftool_once_attr 'overwrite'
+ return 0
+ ;;
+ esac
+ ;;
+ detach)
+ case $cword in
+ 3)
+ COMPREPLY=( $( compgen -W "$ATTACH_TYPES" -- "$cur" ) )
+ return 0
+ ;;
+ 4)
+ COMPREPLY=( $( compgen -W 'dev' -- "$cur" ) )
+ return 0
+ ;;
+ esac
+ ;;
*)
[[ $prev == $object ]] && \
COMPREPLY=( $( compgen -W 'help \
- show list' -- "$cur" ) )
+ show list attach detach' -- "$cur" ) )
;;
esac
;;
feature)
case $command in
probe)
- [[ $prev == "dev" ]] && _sysfs_get_netdevs && return 0
[[ $prev == "prefix" ]] && return 0
if _bpftool_search_list 'macros'; then
COMPREPLY+=( $( compgen -W 'prefix' -- "$cur" ) )
diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 1b8ec91..9a9376d 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -11,6 +11,7 @@
#include <bpf.h>
#include <libbpf.h>
#include <linux/btf.h>
+#include <linux/hashtable.h>
#include "btf.h"
#include "json_writer.h"
@@ -35,6 +36,16 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = {
[BTF_KIND_DATASEC] = "DATASEC",
};
+struct btf_attach_table {
+ DECLARE_HASHTABLE(table, 16);
+};
+
+struct btf_attach_point {
+ __u32 obj_id;
+ __u32 btf_id;
+ struct hlist_node hash;
+};
+
static const char *btf_int_enc_str(__u8 encoding)
{
switch (encoding) {
@@ -449,7 +460,7 @@ static int do_dump(int argc, char **argv)
btf_id = strtoul(*argv, &endptr, 0);
if (*endptr) {
- p_err("can't parse %s as ID", **argv);
+ p_err("can't parse %s as ID", *argv);
return -1;
}
NEXT_ARG();
@@ -522,6 +533,330 @@ static int do_dump(int argc, char **argv)
return err;
}
+static int btf_parse_fd(int *argc, char ***argv)
+{
+ unsigned int id;
+ char *endptr;
+ int fd;
+
+ if (!is_prefix(*argv[0], "id")) {
+ p_err("expected 'id', got: '%s'?", **argv);
+ return -1;
+ }
+ NEXT_ARGP();
+
+ id = strtoul(**argv, &endptr, 0);
+ if (*endptr) {
+ p_err("can't parse %s as ID", **argv);
+ return -1;
+ }
+ NEXT_ARGP();
+
+ fd = bpf_btf_get_fd_by_id(id);
+ if (fd < 0)
+ p_err("can't get BTF object by id (%u): %s",
+ id, strerror(errno));
+
+ return fd;
+}
+
+static void delete_btf_table(struct btf_attach_table *tab)
+{
+ struct btf_attach_point *obj;
+ struct hlist_node *tmp;
+
+ unsigned int bkt;
+
+ hash_for_each_safe(tab->table, bkt, tmp, obj, hash) {
+ hash_del(&obj->hash);
+ free(obj);
+ }
+}
+
+static int
+build_btf_type_table(struct btf_attach_table *tab, enum bpf_obj_type type,
+ void *info, __u32 *len)
+{
+ static const char * const names[] = {
+ [BPF_OBJ_UNKNOWN] = "unknown",
+ [BPF_OBJ_PROG] = "prog",
+ [BPF_OBJ_MAP] = "map",
+ };
+ struct btf_attach_point *obj_node;
+ __u32 btf_id, id = 0;
+ int err;
+ int fd;
+
+ while (true) {
+ switch (type) {
+ case BPF_OBJ_PROG:
+ err = bpf_prog_get_next_id(id, &id);
+ break;
+ case BPF_OBJ_MAP:
+ err = bpf_map_get_next_id(id, &id);
+ break;
+ default:
+ err = -1;
+ p_err("unexpected object type: %d", type);
+ goto err_free;
+ }
+ if (err) {
+ if (errno == ENOENT) {
+ err = 0;
+ break;
+ }
+ p_err("can't get next %s: %s%s", names[type],
+ strerror(errno),
+ errno == EINVAL ? " -- kernel too old?" : "");
+ goto err_free;
+ }
+
+ switch (type) {
+ case BPF_OBJ_PROG:
+ fd = bpf_prog_get_fd_by_id(id);
+ break;
+ case BPF_OBJ_MAP:
+ fd = bpf_map_get_fd_by_id(id);
+ break;
+ default:
+ err = -1;
+ p_err("unexpected object type: %d", type);
+ goto err_free;
+ }
+ if (fd < 0) {
+ if (errno == ENOENT)
+ continue;
+ p_err("can't get %s by id (%u): %s", names[type], id,
+ strerror(errno));
+ err = -1;
+ goto err_free;
+ }
+
+ memset(info, 0, *len);
+ err = bpf_obj_get_info_by_fd(fd, info, len);
+ close(fd);
+ if (err) {
+ p_err("can't get %s info: %s", names[type],
+ strerror(errno));
+ goto err_free;
+ }
+
+ switch (type) {
+ case BPF_OBJ_PROG:
+ btf_id = ((struct bpf_prog_info *)info)->btf_id;
+ break;
+ case BPF_OBJ_MAP:
+ btf_id = ((struct bpf_map_info *)info)->btf_id;
+ break;
+ default:
+ err = -1;
+ p_err("unexpected object type: %d", type);
+ goto err_free;
+ }
+ if (!btf_id)
+ continue;
+
+ obj_node = calloc(1, sizeof(*obj_node));
+ if (!obj_node) {
+ p_err("failed to allocate memory: %s", strerror(errno));
+ goto err_free;
+ }
+
+ obj_node->obj_id = id;
+ obj_node->btf_id = btf_id;
+ hash_add(tab->table, &obj_node->hash, obj_node->btf_id);
+ }
+
+ return 0;
+
+err_free:
+ delete_btf_table(tab);
+ return err;
+}
+
+static int
+build_btf_tables(struct btf_attach_table *btf_prog_table,
+ struct btf_attach_table *btf_map_table)
+{
+ struct bpf_prog_info prog_info;
+ __u32 prog_len = sizeof(prog_info);
+ struct bpf_map_info map_info;
+ __u32 map_len = sizeof(map_info);
+ int err = 0;
+
+ err = build_btf_type_table(btf_prog_table, BPF_OBJ_PROG, &prog_info,
+ &prog_len);
+ if (err)
+ return err;
+
+ err = build_btf_type_table(btf_map_table, BPF_OBJ_MAP, &map_info,
+ &map_len);
+ if (err) {
+ delete_btf_table(btf_prog_table);
+ return err;
+ }
+
+ return 0;
+}
+
+static void
+show_btf_plain(struct bpf_btf_info *info, int fd,
+ struct btf_attach_table *btf_prog_table,
+ struct btf_attach_table *btf_map_table)
+{
+ struct btf_attach_point *obj;
+ int n;
+
+ printf("%u: ", info->id);
+ printf("size %uB", info->btf_size);
+
+ n = 0;
+ hash_for_each_possible(btf_prog_table->table, obj, hash, info->id) {
+ if (obj->btf_id == info->id)
+ printf("%s%u", n++ == 0 ? " prog_ids " : ",",
+ obj->obj_id);
+ }
+
+ n = 0;
+ hash_for_each_possible(btf_map_table->table, obj, hash, info->id) {
+ if (obj->btf_id == info->id)
+ printf("%s%u", n++ == 0 ? " map_ids " : ",",
+ obj->obj_id);
+ }
+
+ printf("\n");
+}
+
+static void
+show_btf_json(struct bpf_btf_info *info, int fd,
+ struct btf_attach_table *btf_prog_table,
+ struct btf_attach_table *btf_map_table)
+{
+ struct btf_attach_point *obj;
+
+ jsonw_start_object(json_wtr); /* btf object */
+ jsonw_uint_field(json_wtr, "id", info->id);
+ jsonw_uint_field(json_wtr, "size", info->btf_size);
+
+ jsonw_name(json_wtr, "prog_ids");
+ jsonw_start_array(json_wtr); /* prog_ids */
+ hash_for_each_possible(btf_prog_table->table, obj, hash,
+ info->id) {
+ if (obj->btf_id == info->id)
+ jsonw_uint(json_wtr, obj->obj_id);
+ }
+ jsonw_end_array(json_wtr); /* prog_ids */
+
+ jsonw_name(json_wtr, "map_ids");
+ jsonw_start_array(json_wtr); /* map_ids */
+ hash_for_each_possible(btf_map_table->table, obj, hash,
+ info->id) {
+ if (obj->btf_id == info->id)
+ jsonw_uint(json_wtr, obj->obj_id);
+ }
+ jsonw_end_array(json_wtr); /* map_ids */
+ jsonw_end_object(json_wtr); /* btf object */
+}
+
+static int
+show_btf(int fd, struct btf_attach_table *btf_prog_table,
+ struct btf_attach_table *btf_map_table)
+{
+ struct bpf_btf_info info = {};
+ __u32 len = sizeof(info);
+ int err;
+
+ err = bpf_obj_get_info_by_fd(fd, &info, &len);
+ if (err) {
+ p_err("can't get BTF object info: %s", strerror(errno));
+ return -1;
+ }
+
+ if (json_output)
+ show_btf_json(&info, fd, btf_prog_table, btf_map_table);
+ else
+ show_btf_plain(&info, fd, btf_prog_table, btf_map_table);
+
+ return 0;
+}
+
+static int do_show(int argc, char **argv)
+{
+ struct btf_attach_table btf_prog_table;
+ struct btf_attach_table btf_map_table;
+ int err, fd = -1;
+ __u32 id = 0;
+
+ if (argc == 2) {
+ fd = btf_parse_fd(&argc, &argv);
+ if (fd < 0)
+ return -1;
+ }
+
+ if (argc) {
+ if (fd >= 0)
+ close(fd);
+ return BAD_ARG();
+ }
+
+ hash_init(btf_prog_table.table);
+ hash_init(btf_map_table.table);
+ err = build_btf_tables(&btf_prog_table, &btf_map_table);
+ if (err) {
+ if (fd >= 0)
+ close(fd);
+ return err;
+ }
+
+ if (fd >= 0) {
+ err = show_btf(fd, &btf_prog_table, &btf_map_table);
+ close(fd);
+ goto exit_free;
+ }
+
+ if (json_output)
+ jsonw_start_array(json_wtr); /* root array */
+
+ while (true) {
+ err = bpf_btf_get_next_id(id, &id);
+ if (err) {
+ if (errno == ENOENT) {
+ err = 0;
+ break;
+ }
+ p_err("can't get next BTF object: %s%s",
+ strerror(errno),
+ errno == EINVAL ? " -- kernel too old?" : "");
+ err = -1;
+ break;
+ }
+
+ fd = bpf_btf_get_fd_by_id(id);
+ if (fd < 0) {
+ if (errno == ENOENT)
+ continue;
+ p_err("can't get BTF object by id (%u): %s",
+ id, strerror(errno));
+ err = -1;
+ break;
+ }
+
+ err = show_btf(fd, &btf_prog_table, &btf_map_table);
+ close(fd);
+ if (err)
+ break;
+ }
+
+ if (json_output)
+ jsonw_end_array(json_wtr); /* root array */
+
+exit_free:
+ delete_btf_table(&btf_prog_table);
+ delete_btf_table(&btf_map_table);
+
+ return err;
+}
+
static int do_help(int argc, char **argv)
{
if (json_output) {
@@ -530,7 +865,8 @@ static int do_help(int argc, char **argv)
}
fprintf(stderr,
- "Usage: %s btf dump BTF_SRC [format FORMAT]\n"
+ "Usage: %s btf { show | list } [id BTF_ID]\n"
+ " %s btf dump BTF_SRC [format FORMAT]\n"
" %s btf help\n"
"\n"
" BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n"
@@ -539,12 +875,14 @@ static int do_help(int argc, char **argv)
" " HELP_SPEC_PROGRAM "\n"
" " HELP_SPEC_OPTIONS "\n"
"",
- bin_name, bin_name);
+ bin_name, bin_name, bin_name);
return 0;
}
static const struct cmd cmds[] = {
+ { "show", do_show },
+ { "list", do_show },
{ "help", do_help },
{ "dump", do_dump },
{ 0 }
diff --git a/tools/bpf/bpftool/btf_dumper.c b/tools/bpf/bpftool/btf_dumper.c
index 8cafb9b..d66131f 100644
--- a/tools/bpf/bpftool/btf_dumper.c
+++ b/tools/bpf/bpftool/btf_dumper.c
@@ -26,9 +26,9 @@ static void btf_dumper_ptr(const void *data, json_writer_t *jw,
bool is_plain_text)
{
if (is_plain_text)
- jsonw_printf(jw, "%p", *(unsigned long *)data);
+ jsonw_printf(jw, "%p", data);
else
- jsonw_printf(jw, "%u", *(unsigned long *)data);
+ jsonw_printf(jw, "%lu", *(unsigned long *)data);
}
static int btf_dumper_modifier(const struct btf_dumper *d, __u32 type_id,
@@ -216,7 +216,7 @@ static int btf_dumper_int(const struct btf_type *t, __u8 bit_offset,
switch (BTF_INT_ENCODING(*int_type)) {
case 0:
if (BTF_INT_BITS(*int_type) == 64)
- jsonw_printf(jw, "%lu", *(__u64 *)data);
+ jsonw_printf(jw, "%llu", *(__u64 *)data);
else if (BTF_INT_BITS(*int_type) == 32)
jsonw_printf(jw, "%u", *(__u32 *)data);
else if (BTF_INT_BITS(*int_type) == 16)
@@ -229,7 +229,7 @@ static int btf_dumper_int(const struct btf_type *t, __u8 bit_offset,
break;
case BTF_INT_SIGNED:
if (BTF_INT_BITS(*int_type) == 64)
- jsonw_printf(jw, "%ld", *(long long *)data);
+ jsonw_printf(jw, "%lld", *(long long *)data);
else if (BTF_INT_BITS(*int_type) == 32)
jsonw_printf(jw, "%d", *(int *)data);
else if (BTF_INT_BITS(*int_type) == 16)
diff --git a/tools/bpf/bpftool/cgroup.c b/tools/bpf/bpftool/cgroup.c
index 44352b5..1ef45e5 100644
--- a/tools/bpf/bpftool/cgroup.c
+++ b/tools/bpf/bpftool/cgroup.c
@@ -120,8 +120,8 @@ static int count_attached_bpf_progs(int cgroup_fd, enum bpf_attach_type type)
static int show_attached_bpf_progs(int cgroup_fd, enum bpf_attach_type type,
int level)
{
+ const char *attach_flags_str;
__u32 prog_ids[1024] = {0};
- char *attach_flags_str;
__u32 prog_cnt, iter;
__u32 attach_flags;
char buf[32];
diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c
index 6a71324..88264aba 100644
--- a/tools/bpf/bpftool/common.c
+++ b/tools/bpf/bpftool/common.c
@@ -29,7 +29,7 @@
#define BPF_FS_MAGIC 0xcafe4a11
#endif
-void __printf(1, 2) p_err(const char *fmt, ...)
+void p_err(const char *fmt, ...)
{
va_list ap;
@@ -47,7 +47,7 @@ void __printf(1, 2) p_err(const char *fmt, ...)
va_end(ap);
}
-void __printf(1, 2) p_info(const char *fmt, ...)
+void p_info(const char *fmt, ...)
{
va_list ap;
diff --git a/tools/bpf/bpftool/json_writer.c b/tools/bpf/bpftool/json_writer.c
index 6046dca..86501cd 100644
--- a/tools/bpf/bpftool/json_writer.c
+++ b/tools/bpf/bpftool/json_writer.c
@@ -15,7 +15,6 @@
#include <malloc.h>
#include <inttypes.h>
#include <stdint.h>
-#include <linux/compiler.h>
#include "json_writer.h"
@@ -153,8 +152,7 @@ void jsonw_name(json_writer_t *self, const char *name)
putc(' ', self->out);
}
-void __printf(2, 0)
-jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap)
+void jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap)
{
jsonw_eor(self);
putc('"', self->out);
@@ -162,7 +160,7 @@ jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap)
putc('"', self->out);
}
-void __printf(2, 3) jsonw_printf(json_writer_t *self, const char *fmt, ...)
+void jsonw_printf(json_writer_t *self, const char *fmt, ...)
{
va_list ap;
diff --git a/tools/bpf/bpftool/json_writer.h b/tools/bpf/bpftool/json_writer.h
index cb9a199..35cf1f0 100644
--- a/tools/bpf/bpftool/json_writer.h
+++ b/tools/bpf/bpftool/json_writer.h
@@ -14,6 +14,7 @@
#include <stdbool.h>
#include <stdint.h>
#include <stdarg.h>
+#include <linux/compiler.h>
/* Opaque class structure */
typedef struct json_writer json_writer_t;
@@ -30,8 +31,9 @@ void jsonw_pretty(json_writer_t *self, bool on);
void jsonw_name(json_writer_t *self, const char *name);
/* Add value */
-void jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap);
-void jsonw_printf(json_writer_t *self, const char *fmt, ...);
+void __printf(2, 0) jsonw_vprintf_enquote(json_writer_t *self, const char *fmt,
+ va_list ap);
+void __printf(2, 3) jsonw_printf(json_writer_t *self, const char *fmt, ...);
void jsonw_string(json_writer_t *self, const char *value);
void jsonw_bool(json_writer_t *self, bool value);
void jsonw_float(json_writer_t *self, double number);
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index e916ff2..93d0086 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -139,7 +139,7 @@ int detect_common_prefix(const char *arg, ...)
strncat(msg, "'", sizeof(msg) - strlen(msg) - 1);
if (count >= 2) {
- p_err(msg);
+ p_err("%s", msg);
return -1;
}
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 7031a4b..af9ad56 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -98,8 +98,8 @@ extern int bpf_flags;
extern struct pinned_obj_table prog_table;
extern struct pinned_obj_table map_table;
-void p_err(const char *fmt, ...);
-void p_info(const char *fmt, ...);
+void __printf(1, 2) p_err(const char *fmt, ...);
+void __printf(1, 2) p_info(const char *fmt, ...);
bool is_prefix(const char *pfx, const char *str);
int detect_common_prefix(const char *arg, ...);
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index bfbbc6b..de61d73 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -481,9 +481,11 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
static int show_map_close_json(int fd, struct bpf_map_info *info)
{
- char *memlock;
+ char *memlock, *frozen_str;
+ int frozen = 0;
memlock = get_fdinfo(fd, "memlock");
+ frozen_str = get_fdinfo(fd, "frozen");
jsonw_start_object(json_wtr);
@@ -533,6 +535,12 @@ static int show_map_close_json(int fd, struct bpf_map_info *info)
}
close(fd);
+ if (frozen_str) {
+ frozen = atoi(frozen_str);
+ free(frozen_str);
+ }
+ jsonw_int_field(json_wtr, "frozen", frozen);
+
if (info->btf_id)
jsonw_int_field(json_wtr, "btf_id", info->btf_id);
@@ -555,9 +563,11 @@ static int show_map_close_json(int fd, struct bpf_map_info *info)
static int show_map_close_plain(int fd, struct bpf_map_info *info)
{
- char *memlock;
+ char *memlock, *frozen_str;
+ int frozen = 0;
memlock = get_fdinfo(fd, "memlock");
+ frozen_str = get_fdinfo(fd, "frozen");
printf("%u: ", info->id);
if (info->type < ARRAY_SIZE(map_type_name))
@@ -610,9 +620,23 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info)
printf("\n\tpinned %s", obj->path);
}
}
+ printf("\n");
+
+ if (frozen_str) {
+ frozen = atoi(frozen_str);
+ free(frozen_str);
+ }
+
+ if (!info->btf_id && !frozen)
+ return 0;
+
+ printf("\t");
if (info->btf_id)
- printf("\n\tbtf_id %d", info->btf_id);
+ printf("btf_id %d", info->btf_id);
+
+ if (frozen)
+ printf("%sfrozen", info->btf_id ? " " : "");
printf("\n");
return 0;
@@ -1238,6 +1262,35 @@ static int do_pop_dequeue(int argc, char **argv)
return err;
}
+static int do_freeze(int argc, char **argv)
+{
+ int err, fd;
+
+ if (!REQ_ARGS(2))
+ return -1;
+
+ fd = map_parse_fd(&argc, &argv);
+ if (fd < 0)
+ return -1;
+
+ if (argc) {
+ close(fd);
+ return BAD_ARG();
+ }
+
+ err = bpf_map_freeze(fd);
+ close(fd);
+ if (err) {
+ p_err("failed to freeze map: %s", strerror(errno));
+ return err;
+ }
+
+ if (json_output)
+ jsonw_null(json_wtr);
+
+ return 0;
+}
+
static int do_help(int argc, char **argv)
{
if (json_output) {
@@ -1262,6 +1315,7 @@ static int do_help(int argc, char **argv)
" %s %s pop MAP\n"
" %s %s enqueue MAP value VALUE\n"
" %s %s dequeue MAP\n"
+ " %s %s freeze MAP\n"
" %s %s help\n"
"\n"
" " HELP_SPEC_MAP "\n"
@@ -1280,7 +1334,8 @@ static int do_help(int argc, char **argv)
bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
- bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2]);
+ bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
+ bin_name, argv[-2]);
return 0;
}
@@ -1302,6 +1357,7 @@ static const struct cmd cmds[] = {
{ "enqueue", do_update },
{ "pop", do_pop_dequeue },
{ "dequeue", do_pop_dequeue },
+ { "freeze", do_freeze },
{ 0 }
};
diff --git a/tools/bpf/bpftool/map_perf_ring.c b/tools/bpf/bpftool/map_perf_ring.c
index 3f108ab..4c5531d 100644
--- a/tools/bpf/bpftool/map_perf_ring.c
+++ b/tools/bpf/bpftool/map_perf_ring.c
@@ -157,7 +157,7 @@ int do_event_pipe(int argc, char **argv)
NEXT_ARG();
ctx.cpu = strtoul(*argv, &endptr, 0);
if (*endptr) {
- p_err("can't parse %s as CPU ID", **argv);
+ p_err("can't parse %s as CPU ID", *argv);
goto err_close_map;
}
@@ -168,7 +168,7 @@ int do_event_pipe(int argc, char **argv)
NEXT_ARG();
ctx.idx = strtoul(*argv, &endptr, 0);
if (*endptr) {
- p_err("can't parse %s as index", **argv);
+ p_err("can't parse %s as index", *argv);
goto err_close_map;
}
diff --git a/tools/bpf/bpftool/net.c b/tools/bpf/bpftool/net.c
index 67e99c5..4f52d31 100644
--- a/tools/bpf/bpftool/net.c
+++ b/tools/bpf/bpftool/net.c
@@ -55,6 +55,35 @@ struct bpf_attach_info {
__u32 flow_dissector_id;
};
+enum net_attach_type {
+ NET_ATTACH_TYPE_XDP,
+ NET_ATTACH_TYPE_XDP_GENERIC,
+ NET_ATTACH_TYPE_XDP_DRIVER,
+ NET_ATTACH_TYPE_XDP_OFFLOAD,
+};
+
+static const char * const attach_type_strings[] = {
+ [NET_ATTACH_TYPE_XDP] = "xdp",
+ [NET_ATTACH_TYPE_XDP_GENERIC] = "xdpgeneric",
+ [NET_ATTACH_TYPE_XDP_DRIVER] = "xdpdrv",
+ [NET_ATTACH_TYPE_XDP_OFFLOAD] = "xdpoffload",
+};
+
+const size_t net_attach_type_size = ARRAY_SIZE(attach_type_strings);
+
+static enum net_attach_type parse_attach_type(const char *str)
+{
+ enum net_attach_type type;
+
+ for (type = 0; type < net_attach_type_size; type++) {
+ if (attach_type_strings[type] &&
+ is_prefix(str, attach_type_strings[type]))
+ return type;
+ }
+
+ return net_attach_type_size;
+}
+
static int dump_link_nlmsg(void *cookie, void *msg, struct nlattr **tb)
{
struct bpf_netdev_t *netinfo = cookie;
@@ -197,7 +226,7 @@ static int query_flow_dissector(struct bpf_attach_info *attach_info)
fd = open("/proc/self/ns/net", O_RDONLY);
if (fd < 0) {
- p_err("can't open /proc/self/ns/net: %d",
+ p_err("can't open /proc/self/ns/net: %s",
strerror(errno));
return -1;
}
@@ -223,6 +252,134 @@ static int query_flow_dissector(struct bpf_attach_info *attach_info)
return 0;
}
+static int net_parse_dev(int *argc, char ***argv)
+{
+ int ifindex;
+
+ if (is_prefix(**argv, "dev")) {
+ NEXT_ARGP();
+
+ ifindex = if_nametoindex(**argv);
+ if (!ifindex)
+ p_err("invalid devname %s", **argv);
+
+ NEXT_ARGP();
+ } else {
+ p_err("expected 'dev', got: '%s'?", **argv);
+ return -1;
+ }
+
+ return ifindex;
+}
+
+static int do_attach_detach_xdp(int progfd, enum net_attach_type attach_type,
+ int ifindex, bool overwrite)
+{
+ __u32 flags = 0;
+
+ if (!overwrite)
+ flags = XDP_FLAGS_UPDATE_IF_NOEXIST;
+ if (attach_type == NET_ATTACH_TYPE_XDP_GENERIC)
+ flags |= XDP_FLAGS_SKB_MODE;
+ if (attach_type == NET_ATTACH_TYPE_XDP_DRIVER)
+ flags |= XDP_FLAGS_DRV_MODE;
+ if (attach_type == NET_ATTACH_TYPE_XDP_OFFLOAD)
+ flags |= XDP_FLAGS_HW_MODE;
+
+ return bpf_set_link_xdp_fd(ifindex, progfd, flags);
+}
+
+static int do_attach(int argc, char **argv)
+{
+ enum net_attach_type attach_type;
+ int progfd, ifindex, err = 0;
+ bool overwrite = false;
+
+ /* parse attach args */
+ if (!REQ_ARGS(5))
+ return -EINVAL;
+
+ attach_type = parse_attach_type(*argv);
+ if (attach_type == net_attach_type_size) {
+ p_err("invalid net attach/detach type: %s", *argv);
+ return -EINVAL;
+ }
+ NEXT_ARG();
+
+ progfd = prog_parse_fd(&argc, &argv);
+ if (progfd < 0)
+ return -EINVAL;
+
+ ifindex = net_parse_dev(&argc, &argv);
+ if (ifindex < 1) {
+ close(progfd);
+ return -EINVAL;
+ }
+
+ if (argc) {
+ if (is_prefix(*argv, "overwrite")) {
+ overwrite = true;
+ } else {
+ p_err("expected 'overwrite', got: '%s'?", *argv);
+ close(progfd);
+ return -EINVAL;
+ }
+ }
+
+ /* attach xdp prog */
+ if (is_prefix("xdp", attach_type_strings[attach_type]))
+ err = do_attach_detach_xdp(progfd, attach_type, ifindex,
+ overwrite);
+
+ if (err < 0) {
+ p_err("interface %s attach failed: %s",
+ attach_type_strings[attach_type], strerror(-err));
+ return err;
+ }
+
+ if (json_output)
+ jsonw_null(json_wtr);
+
+ return 0;
+}
+
+static int do_detach(int argc, char **argv)
+{
+ enum net_attach_type attach_type;
+ int progfd, ifindex, err = 0;
+
+ /* parse detach args */
+ if (!REQ_ARGS(3))
+ return -EINVAL;
+
+ attach_type = parse_attach_type(*argv);
+ if (attach_type == net_attach_type_size) {
+ p_err("invalid net attach/detach type: %s", *argv);
+ return -EINVAL;
+ }
+ NEXT_ARG();
+
+ ifindex = net_parse_dev(&argc, &argv);
+ if (ifindex < 1)
+ return -EINVAL;
+
+ /* detach xdp prog */
+ progfd = -1;
+ if (is_prefix("xdp", attach_type_strings[attach_type]))
+ err = do_attach_detach_xdp(progfd, attach_type, ifindex, NULL);
+
+ if (err < 0) {
+ p_err("interface %s detach failed: %s",
+ attach_type_strings[attach_type], strerror(-err));
+ return err;
+ }
+
+ if (json_output)
+ jsonw_null(json_wtr);
+
+ return 0;
+}
+
static int do_show(int argc, char **argv)
{
struct bpf_attach_info attach_info = {};
@@ -232,13 +389,9 @@ static int do_show(int argc, char **argv)
char err_buf[256];
if (argc == 2) {
- if (strcmp(argv[0], "dev") != 0)
- usage();
- filter_idx = if_nametoindex(argv[1]);
- if (filter_idx == 0) {
- fprintf(stderr, "invalid dev name %s\n", argv[1]);
+ filter_idx = net_parse_dev(&argc, &argv);
+ if (filter_idx < 1)
return -1;
- }
} else if (argc != 0) {
usage();
}
@@ -305,13 +458,20 @@ static int do_help(int argc, char **argv)
fprintf(stderr,
"Usage: %s %s { show | list } [dev <devname>]\n"
+ " %s %s attach ATTACH_TYPE PROG dev <devname> [ overwrite ]\n"
+ " %s %s detach ATTACH_TYPE dev <devname>\n"
" %s %s help\n"
+ "\n"
+ " " HELP_SPEC_PROGRAM "\n"
+ " ATTACH_TYPE := { xdp | xdpgeneric | xdpdrv | xdpoffload }\n"
+ "\n"
"Note: Only xdp and tc attachments are supported now.\n"
" For progs attached to cgroups, use \"bpftool cgroup\"\n"
" to dump program attachments. For program types\n"
" sk_{filter,skb,msg,reuseport} and lwt/seg6, please\n"
" consult iproute2.\n",
- bin_name, argv[-2], bin_name, argv[-2]);
+ bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
+ bin_name, argv[-2]);
return 0;
}
@@ -319,6 +479,8 @@ static int do_help(int argc, char **argv)
static const struct cmd cmds[] = {
{ "show", do_show },
{ "list", do_show },
+ { "attach", do_attach },
+ { "detach", do_detach },
{ "help", do_help },
{ 0 }
};
diff --git a/tools/bpf/bpftool/perf.c b/tools/bpf/bpftool/perf.c
index f2a545e..b2046f3 100644
--- a/tools/bpf/bpftool/perf.c
+++ b/tools/bpf/bpftool/perf.c
@@ -104,6 +104,8 @@ static void print_perf_json(int pid, int fd, __u32 prog_id, __u32 fd_type,
jsonw_string_field(json_wtr, "filename", buf);
jsonw_lluint_field(json_wtr, "offset", probe_offset);
break;
+ default:
+ break;
}
jsonw_end_object(json_wtr);
}
@@ -140,6 +142,8 @@ static void print_perf_plain(int pid, int fd, __u32 prog_id, __u32 fd_type,
printf("uretprobe filename %s offset %llu\n", buf,
probe_offset);
break;
+ default:
+ break;
}
}
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 66f04a4..43fdbbf 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -363,7 +363,9 @@ static int do_show(int argc, char **argv)
if (fd < 0)
return -1;
- return show_prog(fd);
+ err = show_prog(fd);
+ close(fd);
+ return err;
}
if (argc)
diff --git a/tools/hv/hv_kvp_daemon.c b/tools/hv/hv_kvp_daemon.c
index f559750..e9ef4ca 100644
--- a/tools/hv/hv_kvp_daemon.c
+++ b/tools/hv/hv_kvp_daemon.c
@@ -809,7 +809,7 @@ kvp_get_ip_info(int family, char *if_name, int op,
int sn_offset = 0;
int error = 0;
char *buffer;
- struct hv_kvp_ipaddr_value *ip_buffer;
+ struct hv_kvp_ipaddr_value *ip_buffer = NULL;
char cidr_mask[5]; /* /xyz */
int weight;
int i;
diff --git a/tools/include/linux/compiler-gcc.h b/tools/include/linux/compiler-gcc.h
index 0d35f18..95c072b 100644
--- a/tools/include/linux/compiler-gcc.h
+++ b/tools/include/linux/compiler-gcc.h
@@ -6,9 +6,11 @@
/*
* Common definitions for all gcc versions go here.
*/
+#ifndef GCC_VERSION
#define GCC_VERSION (__GNUC__ * 10000 \
+ __GNUC_MINOR__ * 100 \
+ __GNUC_PATCHLEVEL__)
+#endif
#if GCC_VERSION >= 70000 && !defined(__CHECKER__)
# define __fallthrough __attribute__ ((fallthrough))
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 0e66371..77c6be9 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -106,6 +106,7 @@ enum bpf_cmd {
BPF_TASK_FD_QUERY,
BPF_MAP_LOOKUP_AND_DELETE_ELEM,
BPF_MAP_FREEZE,
+ BPF_BTF_GET_NEXT_ID,
};
enum bpf_map_type {
@@ -284,6 +285,9 @@ enum bpf_attach_type {
*/
#define BPF_F_TEST_RND_HI32 (1U << 2)
+/* The verifier internal test flag. Behavior is undefined */
+#define BPF_F_TEST_STATE_FREQ (1U << 3)
+
/* When BPF ldimm64's insn[0].src_reg != 0 then this can have
* two extensions:
*
@@ -337,6 +341,9 @@ enum bpf_attach_type {
#define BPF_F_RDONLY_PROG (1U << 7)
#define BPF_F_WRONLY_PROG (1U << 8)
+/* Clone map from listener for newly accepted socket */
+#define BPF_F_CLONE (1U << 9)
+
/* flags for BPF_PROG_QUERY */
#define BPF_F_QUERY_EFFECTIVE (1U << 0)
@@ -576,6 +583,8 @@ union bpf_attr {
* limited to five).
*
* Each time the helper is called, it appends a line to the trace.
+ * Lines are discarded while *\/sys/kernel/debug/tracing/trace* is
+ * open, use *\/sys/kernel/debug/tracing/trace_pipe* to avoid this.
* The format of the trace is customizable, and the exact output
* one will get depends on the options set in
* *\/sys/kernel/debug/tracing/trace_options* (see also the
@@ -1014,7 +1023,7 @@ union bpf_attr {
* The realm of the route for the packet associated to *skb*, or 0
* if none was found.
*
- * int bpf_perf_event_output(struct pt_reg *ctx, struct bpf_map *map, u64 flags, void *data, u64 size)
+ * int bpf_perf_event_output(struct pt_regs *ctx, struct bpf_map *map, u64 flags, void *data, u64 size)
* Description
* Write raw *data* blob into a special BPF perf event held by
* *map* of type **BPF_MAP_TYPE_PERF_EVENT_ARRAY**. This perf
@@ -1076,7 +1085,7 @@ union bpf_attr {
* Return
* 0 on success, or a negative error in case of failure.
*
- * int bpf_get_stackid(struct pt_reg *ctx, struct bpf_map *map, u64 flags)
+ * int bpf_get_stackid(struct pt_regs *ctx, struct bpf_map *map, u64 flags)
* Description
* Walk a user or a kernel stack and return its id. To achieve
* this, the helper needs *ctx*, which is a pointer to the context
@@ -1725,7 +1734,7 @@ union bpf_attr {
* Return
* 0 on success, or a negative error in case of failure.
*
- * int bpf_override_return(struct pt_reg *regs, u64 rc)
+ * int bpf_override_return(struct pt_regs *regs, u64 rc)
* Description
* Used for error injection, this helper uses kprobes to override
* the return value of the probed function, and to set it to *rc*.
diff --git a/tools/include/uapi/linux/if_xdp.h b/tools/include/uapi/linux/if_xdp.h
index faaa5ca..be328c5 100644
--- a/tools/include/uapi/linux/if_xdp.h
+++ b/tools/include/uapi/linux/if_xdp.h
@@ -16,6 +16,18 @@
#define XDP_SHARED_UMEM (1 << 0)
#define XDP_COPY (1 << 1) /* Force copy-mode */
#define XDP_ZEROCOPY (1 << 2) /* Force zero-copy mode */
+/* If this option is set, the driver might go sleep and in that case
+ * the XDP_RING_NEED_WAKEUP flag in the fill and/or Tx rings will be
+ * set. If it is set, the application need to explicitly wake up the
+ * driver with a poll() (Rx and Tx) or sendto() (Tx only). If you are
+ * running the driver and the application on the same core, you should
+ * use this option so that the kernel will yield to the user space
+ * application.
+ */
+#define XDP_USE_NEED_WAKEUP (1 << 3)
+
+/* Flags for xsk_umem_config flags */
+#define XDP_UMEM_UNALIGNED_CHUNK_FLAG (1 << 0)
struct sockaddr_xdp {
__u16 sxdp_family;
@@ -25,10 +37,14 @@ struct sockaddr_xdp {
__u32 sxdp_shared_umem_fd;
};
+/* XDP_RING flags */
+#define XDP_RING_NEED_WAKEUP (1 << 0)
+
struct xdp_ring_offset {
__u64 producer;
__u64 consumer;
__u64 desc;
+ __u64 flags;
};
struct xdp_mmap_offsets {
@@ -53,6 +69,7 @@ struct xdp_umem_reg {
__u64 len; /* Length of packet data area */
__u32 chunk_size;
__u32 headroom;
+ __u32 flags;
};
struct xdp_statistics {
@@ -74,6 +91,11 @@ struct xdp_options {
#define XDP_UMEM_PGOFF_FILL_RING 0x100000000ULL
#define XDP_UMEM_PGOFF_COMPLETION_RING 0x180000000ULL
+/* Masks for unaligned chunks mode */
+#define XSK_UNALIGNED_BUF_OFFSET_SHIFT 48
+#define XSK_UNALIGNED_BUF_ADDR_MASK \
+ ((1ULL << XSK_UNALIGNED_BUF_OFFSET_SHIFT) - 1)
+
/* Rx/Tx descriptor */
struct xdp_desc {
__u64 addr;
diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index 9312066..c6f94cf 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -1,9 +1,10 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
# Most of this file is copied from tools/lib/traceevent/Makefile
-BPF_VERSION = 0
-BPF_PATCHLEVEL = 0
-BPF_EXTRAVERSION = 4
+LIBBPF_VERSION := $(shell \
+ grep -oE '^LIBBPF_([0-9.]+)' libbpf.map | \
+ sort -rV | head -n1 | cut -d'_' -f2)
+LIBBPF_MAJOR_VERSION := $(firstword $(subst ., ,$(LIBBPF_VERSION)))
MAKEFLAGS += --no-print-directory
@@ -79,15 +80,9 @@
libdir_SQ = $(subst ','\'',$(libdir))
libdir_relative_SQ = $(subst ','\'',$(libdir_relative))
-VERSION = $(BPF_VERSION)
-PATCHLEVEL = $(BPF_PATCHLEVEL)
-EXTRAVERSION = $(BPF_EXTRAVERSION)
-
OBJ = $@
N =
-LIBBPF_VERSION = $(BPF_VERSION).$(BPF_PATCHLEVEL).$(BPF_EXTRAVERSION)
-
LIB_TARGET = libbpf.a libbpf.so.$(LIBBPF_VERSION)
LIB_FILE = libbpf.a libbpf.so*
PC_FILE = libbpf.pc
@@ -113,6 +108,7 @@
override CFLAGS += -fPIC
override CFLAGS += $(INCLUDES)
override CFLAGS += -fvisibility=hidden
+override CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
ifeq ($(VERBOSE),1)
Q =
@@ -138,7 +134,9 @@
PC_FILE := $(addprefix $(OUTPUT),$(PC_FILE))
GLOBAL_SYM_COUNT = $(shell readelf -s --wide $(BPF_IN) | \
- awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {s++} END{print s}')
+ cut -d "@" -f1 | sed 's/_v[0-9]_[0-9]_[0-9].*//' | \
+ awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$8}' | \
+ sort -u | wc -l)
VERSIONED_SYM_COUNT = $(shell readelf -s --wide $(OUTPUT)libbpf.so | \
grep -Eo '[^ ]+@LIBBPF_' | cut -d@ -f1 | sort -u | wc -l)
@@ -178,10 +176,10 @@
$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)
$(OUTPUT)libbpf.so.$(LIBBPF_VERSION): $(BPF_IN)
- $(QUIET_LINK)$(CC) --shared -Wl,-soname,libbpf.so.$(VERSION) \
+ $(QUIET_LINK)$(CC) --shared -Wl,-soname,libbpf.so.$(LIBBPF_MAJOR_VERSION) \
-Wl,--version-script=$(VERSION_SCRIPT) $^ -lelf -o $@
@ln -sf $(@F) $(OUTPUT)libbpf.so
- @ln -sf $(@F) $(OUTPUT)libbpf.so.$(VERSION)
+ @ln -sf $(@F) $(OUTPUT)libbpf.so.$(LIBBPF_MAJOR_VERSION)
$(OUTPUT)libbpf.a: $(BPF_IN)
$(QUIET_LINK)$(RM) $@; $(AR) rcs $@ $^
@@ -205,6 +203,7 @@
"Please make sure all LIBBPF_API symbols are" \
"versioned in $(VERSION_SCRIPT)." >&2; \
readelf -s --wide $(OUTPUT)libbpf-in.o | \
+ cut -d "@" -f1 | sed 's/_v[0-9]_[0-9]_[0-9].*//' | \
awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$8}'| \
sort -u > $(OUTPUT)libbpf_global_syms.tmp; \
readelf -s --wide $(OUTPUT)libbpf.so | \
@@ -257,7 +256,8 @@
clean:
$(call QUIET_CLEAN, libbpf) $(RM) $(TARGETS) $(CXX_TEST_TARGET) \
- *.o *~ *.a *.so *.so.$(VERSION) .*.d .*.cmd *.pc LIBBPF-CFLAGS
+ *.o *~ *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) .*.d .*.cmd \
+ *.pc LIBBPF-CFLAGS
$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index c7d7993..cbb9335 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -568,7 +568,7 @@ int bpf_prog_test_run_xattr(struct bpf_prog_test_run_attr *test_attr)
return ret;
}
-int bpf_prog_get_next_id(__u32 start_id, __u32 *next_id)
+static int bpf_obj_get_next_id(__u32 start_id, __u32 *next_id, int cmd)
{
union bpf_attr attr;
int err;
@@ -576,26 +576,26 @@ int bpf_prog_get_next_id(__u32 start_id, __u32 *next_id)
memset(&attr, 0, sizeof(attr));
attr.start_id = start_id;
- err = sys_bpf(BPF_PROG_GET_NEXT_ID, &attr, sizeof(attr));
+ err = sys_bpf(cmd, &attr, sizeof(attr));
if (!err)
*next_id = attr.next_id;
return err;
}
+int bpf_prog_get_next_id(__u32 start_id, __u32 *next_id)
+{
+ return bpf_obj_get_next_id(start_id, next_id, BPF_PROG_GET_NEXT_ID);
+}
+
int bpf_map_get_next_id(__u32 start_id, __u32 *next_id)
{
- union bpf_attr attr;
- int err;
+ return bpf_obj_get_next_id(start_id, next_id, BPF_MAP_GET_NEXT_ID);
+}
- memset(&attr, 0, sizeof(attr));
- attr.start_id = start_id;
-
- err = sys_bpf(BPF_MAP_GET_NEXT_ID, &attr, sizeof(attr));
- if (!err)
- *next_id = attr.next_id;
-
- return err;
+int bpf_btf_get_next_id(__u32 start_id, __u32 *next_id)
+{
+ return bpf_obj_get_next_id(start_id, next_id, BPF_BTF_GET_NEXT_ID);
}
int bpf_prog_get_fd_by_id(__u32 id)
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index ff42ca0..0db0133 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -156,6 +156,7 @@ LIBBPF_API int bpf_prog_test_run(int prog_fd, int repeat, void *data,
__u32 *retval, __u32 *duration);
LIBBPF_API int bpf_prog_get_next_id(__u32 start_id, __u32 *next_id);
LIBBPF_API int bpf_map_get_next_id(__u32 start_id, __u32 *next_id);
+LIBBPF_API int bpf_btf_get_next_id(__u32 start_id, __u32 *next_id);
LIBBPF_API int bpf_prog_get_fd_by_id(__u32 id);
LIBBPF_API int bpf_map_get_fd_by_id(__u32 id);
LIBBPF_API int bpf_btf_get_fd_by_id(__u32 id);
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index f9d316e..d04c7cb 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -183,4 +183,10 @@
perf_buffer__new;
perf_buffer__new_raw;
perf_buffer__poll;
+ xsk_umem__create;
} LIBBPF_0.0.3;
+
+LIBBPF_0.0.5 {
+ global:
+ bpf_btf_get_next_id;
+} LIBBPF_0.0.4;
diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c
index 680e630..842c4fd 100644
--- a/tools/lib/bpf/xsk.c
+++ b/tools/lib/bpf/xsk.c
@@ -74,23 +74,6 @@ struct xsk_nl_info {
int fd;
};
-/* For 32-bit systems, we need to use mmap2 as the offsets are 64-bit.
- * Unfortunately, it is not part of glibc.
- */
-static inline void *xsk_mmap(void *addr, size_t length, int prot, int flags,
- int fd, __u64 offset)
-{
-#ifdef __NR_mmap2
- unsigned int page_shift = __builtin_ffs(getpagesize()) - 1;
- long ret = syscall(__NR_mmap2, addr, length, prot, flags, fd,
- (off_t)(offset >> page_shift));
-
- return (void *)ret;
-#else
- return mmap(addr, length, prot, flags, fd, offset);
-#endif
-}
-
int xsk_umem__fd(const struct xsk_umem *umem)
{
return umem ? umem->fd : -EINVAL;
@@ -116,6 +99,7 @@ static void xsk_set_umem_config(struct xsk_umem_config *cfg,
cfg->comp_size = XSK_RING_CONS__DEFAULT_NUM_DESCS;
cfg->frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
cfg->frame_headroom = XSK_UMEM__DEFAULT_FRAME_HEADROOM;
+ cfg->flags = XSK_UMEM__DEFAULT_FLAGS;
return;
}
@@ -123,6 +107,7 @@ static void xsk_set_umem_config(struct xsk_umem_config *cfg,
cfg->comp_size = usr_cfg->comp_size;
cfg->frame_size = usr_cfg->frame_size;
cfg->frame_headroom = usr_cfg->frame_headroom;
+ cfg->flags = usr_cfg->flags;
}
static int xsk_set_xdp_socket_config(struct xsk_socket_config *cfg,
@@ -149,9 +134,10 @@ static int xsk_set_xdp_socket_config(struct xsk_socket_config *cfg,
return 0;
}
-int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size,
- struct xsk_ring_prod *fill, struct xsk_ring_cons *comp,
- const struct xsk_umem_config *usr_config)
+int xsk_umem__create_v0_0_4(struct xsk_umem **umem_ptr, void *umem_area,
+ __u64 size, struct xsk_ring_prod *fill,
+ struct xsk_ring_cons *comp,
+ const struct xsk_umem_config *usr_config)
{
struct xdp_mmap_offsets off;
struct xdp_umem_reg mr;
@@ -182,6 +168,7 @@ int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size,
mr.len = size;
mr.chunk_size = umem->config.frame_size;
mr.headroom = umem->config.frame_headroom;
+ mr.flags = umem->config.flags;
err = setsockopt(umem->fd, SOL_XDP, XDP_UMEM_REG, &mr, sizeof(mr));
if (err) {
@@ -210,10 +197,9 @@ int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size,
goto out_socket;
}
- map = xsk_mmap(NULL, off.fr.desc +
- umem->config.fill_size * sizeof(__u64),
- PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE,
- umem->fd, XDP_UMEM_PGOFF_FILL_RING);
+ map = mmap(NULL, off.fr.desc + umem->config.fill_size * sizeof(__u64),
+ PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, umem->fd,
+ XDP_UMEM_PGOFF_FILL_RING);
if (map == MAP_FAILED) {
err = -errno;
goto out_socket;
@@ -224,13 +210,13 @@ int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size,
fill->size = umem->config.fill_size;
fill->producer = map + off.fr.producer;
fill->consumer = map + off.fr.consumer;
+ fill->flags = map + off.fr.flags;
fill->ring = map + off.fr.desc;
fill->cached_cons = umem->config.fill_size;
- map = xsk_mmap(NULL,
- off.cr.desc + umem->config.comp_size * sizeof(__u64),
- PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE,
- umem->fd, XDP_UMEM_PGOFF_COMPLETION_RING);
+ map = mmap(NULL, off.cr.desc + umem->config.comp_size * sizeof(__u64),
+ PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, umem->fd,
+ XDP_UMEM_PGOFF_COMPLETION_RING);
if (map == MAP_FAILED) {
err = -errno;
goto out_mmap;
@@ -241,6 +227,7 @@ int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size,
comp->size = umem->config.comp_size;
comp->producer = map + off.cr.producer;
comp->consumer = map + off.cr.consumer;
+ comp->flags = map + off.cr.flags;
comp->ring = map + off.cr.desc;
*umem_ptr = umem;
@@ -255,6 +242,29 @@ int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size,
return err;
}
+struct xsk_umem_config_v1 {
+ __u32 fill_size;
+ __u32 comp_size;
+ __u32 frame_size;
+ __u32 frame_headroom;
+};
+
+int xsk_umem__create_v0_0_2(struct xsk_umem **umem_ptr, void *umem_area,
+ __u64 size, struct xsk_ring_prod *fill,
+ struct xsk_ring_cons *comp,
+ const struct xsk_umem_config *usr_config)
+{
+ struct xsk_umem_config config;
+
+ memcpy(&config, usr_config, sizeof(struct xsk_umem_config_v1));
+ config.flags = 0;
+
+ return xsk_umem__create_v0_0_4(umem_ptr, umem_area, size, fill, comp,
+ &config);
+}
+asm(".symver xsk_umem__create_v0_0_2, xsk_umem__create@LIBBPF_0.0.2");
+asm(".symver xsk_umem__create_v0_0_4, xsk_umem__create@@LIBBPF_0.0.4");
+
static int xsk_load_xdp_prog(struct xsk_socket *xsk)
{
static const int log_buf_size = 16 * 1024;
@@ -550,11 +560,10 @@ int xsk_socket__create(struct xsk_socket **xsk_ptr, const char *ifname,
}
if (rx) {
- rx_map = xsk_mmap(NULL, off.rx.desc +
- xsk->config.rx_size * sizeof(struct xdp_desc),
- PROT_READ | PROT_WRITE,
- MAP_SHARED | MAP_POPULATE,
- xsk->fd, XDP_PGOFF_RX_RING);
+ rx_map = mmap(NULL, off.rx.desc +
+ xsk->config.rx_size * sizeof(struct xdp_desc),
+ PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE,
+ xsk->fd, XDP_PGOFF_RX_RING);
if (rx_map == MAP_FAILED) {
err = -errno;
goto out_socket;
@@ -564,16 +573,16 @@ int xsk_socket__create(struct xsk_socket **xsk_ptr, const char *ifname,
rx->size = xsk->config.rx_size;
rx->producer = rx_map + off.rx.producer;
rx->consumer = rx_map + off.rx.consumer;
+ rx->flags = rx_map + off.rx.flags;
rx->ring = rx_map + off.rx.desc;
}
xsk->rx = rx;
if (tx) {
- tx_map = xsk_mmap(NULL, off.tx.desc +
- xsk->config.tx_size * sizeof(struct xdp_desc),
- PROT_READ | PROT_WRITE,
- MAP_SHARED | MAP_POPULATE,
- xsk->fd, XDP_PGOFF_TX_RING);
+ tx_map = mmap(NULL, off.tx.desc +
+ xsk->config.tx_size * sizeof(struct xdp_desc),
+ PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE,
+ xsk->fd, XDP_PGOFF_TX_RING);
if (tx_map == MAP_FAILED) {
err = -errno;
goto out_mmap_rx;
@@ -583,6 +592,7 @@ int xsk_socket__create(struct xsk_socket **xsk_ptr, const char *ifname,
tx->size = xsk->config.tx_size;
tx->producer = tx_map + off.tx.producer;
tx->consumer = tx_map + off.tx.consumer;
+ tx->flags = tx_map + off.tx.flags;
tx->ring = tx_map + off.tx.desc;
tx->cached_cons = xsk->config.tx_size;
}
diff --git a/tools/lib/bpf/xsk.h b/tools/lib/bpf/xsk.h
index 833a6e6..584f682 100644
--- a/tools/lib/bpf/xsk.h
+++ b/tools/lib/bpf/xsk.h
@@ -32,6 +32,7 @@ struct name { \
__u32 *producer; \
__u32 *consumer; \
void *ring; \
+ __u32 *flags; \
}
DEFINE_XSK_RING(xsk_ring_prod);
@@ -76,6 +77,11 @@ xsk_ring_cons__rx_desc(const struct xsk_ring_cons *rx, __u32 idx)
return &descs[idx & rx->mask];
}
+static inline int xsk_ring_prod__needs_wakeup(const struct xsk_ring_prod *r)
+{
+ return *r->flags & XDP_RING_NEED_WAKEUP;
+}
+
static inline __u32 xsk_prod_nb_free(struct xsk_ring_prod *r, __u32 nb)
{
__u32 free_entries = r->cached_cons - r->cached_prod;
@@ -162,6 +168,21 @@ static inline void *xsk_umem__get_data(void *umem_area, __u64 addr)
return &((char *)umem_area)[addr];
}
+static inline __u64 xsk_umem__extract_addr(__u64 addr)
+{
+ return addr & XSK_UNALIGNED_BUF_ADDR_MASK;
+}
+
+static inline __u64 xsk_umem__extract_offset(__u64 addr)
+{
+ return addr >> XSK_UNALIGNED_BUF_OFFSET_SHIFT;
+}
+
+static inline __u64 xsk_umem__add_offset_to_addr(__u64 addr)
+{
+ return xsk_umem__extract_addr(addr) + xsk_umem__extract_offset(addr);
+}
+
LIBBPF_API int xsk_umem__fd(const struct xsk_umem *umem);
LIBBPF_API int xsk_socket__fd(const struct xsk_socket *xsk);
@@ -170,12 +191,14 @@ LIBBPF_API int xsk_socket__fd(const struct xsk_socket *xsk);
#define XSK_UMEM__DEFAULT_FRAME_SHIFT 12 /* 4096 bytes */
#define XSK_UMEM__DEFAULT_FRAME_SIZE (1 << XSK_UMEM__DEFAULT_FRAME_SHIFT)
#define XSK_UMEM__DEFAULT_FRAME_HEADROOM 0
+#define XSK_UMEM__DEFAULT_FLAGS 0
struct xsk_umem_config {
__u32 fill_size;
__u32 comp_size;
__u32 frame_size;
__u32 frame_headroom;
+ __u32 flags;
};
/* Flags for the libbpf_flags field. */
@@ -195,6 +218,16 @@ LIBBPF_API int xsk_umem__create(struct xsk_umem **umem,
struct xsk_ring_prod *fill,
struct xsk_ring_cons *comp,
const struct xsk_umem_config *config);
+LIBBPF_API int xsk_umem__create_v0_0_2(struct xsk_umem **umem,
+ void *umem_area, __u64 size,
+ struct xsk_ring_prod *fill,
+ struct xsk_ring_cons *comp,
+ const struct xsk_umem_config *config);
+LIBBPF_API int xsk_umem__create_v0_0_4(struct xsk_umem **umem,
+ void *umem_area, __u64 size,
+ struct xsk_ring_prod *fill,
+ struct xsk_ring_cons *comp,
+ const struct xsk_umem_config *config);
LIBBPF_API int xsk_socket__create(struct xsk_socket **xsk,
const char *ifname, __u32 queue_id,
struct xsk_umem *umem,
diff --git a/tools/power/x86/turbostat/Makefile b/tools/power/x86/turbostat/Makefile
index 045f5f7..13f1e8b 100644
--- a/tools/power/x86/turbostat/Makefile
+++ b/tools/power/x86/turbostat/Makefile
@@ -9,9 +9,10 @@
endif
turbostat : turbostat.c
-override CFLAGS += -Wall -I../../../include
+override CFLAGS += -O2 -Wall -I../../../include
override CFLAGS += -DMSRHEADER='"../../../../arch/x86/include/asm/msr-index.h"'
override CFLAGS += -DINTEL_FAMILY_HEADER='"../../../../arch/x86/include/asm/intel-family.h"'
+override CFLAGS += -D_FORTIFY_SOURCE=2
%: %.c
@mkdir -p $(BUILD_OUTPUT)
diff --git a/tools/power/x86/turbostat/turbostat.c b/tools/power/x86/turbostat/turbostat.c
index 75fc4fb..b2a8643 100644
--- a/tools/power/x86/turbostat/turbostat.c
+++ b/tools/power/x86/turbostat/turbostat.c
@@ -39,7 +39,6 @@ FILE *outf;
int *fd_percpu;
struct timeval interval_tv = {5, 0};
struct timespec interval_ts = {5, 0};
-struct timespec one_msec = {0, 1000000};
unsigned int num_iterations;
unsigned int debug;
unsigned int quiet;
@@ -60,6 +59,7 @@ unsigned int do_irtl_hsw;
unsigned int units = 1000000; /* MHz etc */
unsigned int genuine_intel;
unsigned int authentic_amd;
+unsigned int hygon_genuine;
unsigned int max_level, max_extended_level;
unsigned int has_invariant_tsc;
unsigned int do_nhm_platform_info;
@@ -100,6 +100,7 @@ unsigned int has_hwp_epp; /* IA32_HWP_REQUEST[bits 31:24] */
unsigned int has_hwp_pkg; /* IA32_HWP_REQUEST_PKG */
unsigned int has_misc_feature_control;
unsigned int first_counter_read = 1;
+int ignore_stdin;
#define RAPL_PKG (1 << 0)
/* 0x610 MSR_PKG_POWER_LIMIT */
@@ -166,6 +167,7 @@ size_t cpu_present_setsize, cpu_affinity_setsize, cpu_subset_size;
struct thread_data {
struct timeval tv_begin;
struct timeval tv_end;
+ struct timeval tv_delta;
unsigned long long tsc;
unsigned long long aperf;
unsigned long long mperf;
@@ -506,6 +508,7 @@ unsigned long long bic_enabled = (0xFFFFFFFFFFFFFFFFULL & ~BIC_DISABLED_BY_DEFAU
unsigned long long bic_present = BIC_USEC | BIC_TOD | BIC_sysfs | BIC_APIC | BIC_X2APIC;
#define DO_BIC(COUNTER_NAME) (bic_enabled & bic_present & COUNTER_NAME)
+#define DO_BIC_READ(COUNTER_NAME) (bic_present & COUNTER_NAME)
#define ENABLE_BIC(COUNTER_NAME) (bic_enabled |= COUNTER_NAME)
#define BIC_PRESENT(COUNTER_BIT) (bic_present |= COUNTER_BIT)
#define BIC_NOT_PRESENT(COUNTER_BIT) (bic_present &= ~COUNTER_BIT)
@@ -849,7 +852,6 @@ int dump_counters(struct thread_data *t, struct core_data *c,
outp += sprintf(outp, "pc8: %016llX\n", p->pc8);
outp += sprintf(outp, "pc9: %016llX\n", p->pc9);
outp += sprintf(outp, "pc10: %016llX\n", p->pc10);
- outp += sprintf(outp, "pc10: %016llX\n", p->pc10);
outp += sprintf(outp, "cpu_lpi: %016llX\n", p->cpu_lpi);
outp += sprintf(outp, "sys_lpi: %016llX\n", p->sys_lpi);
outp += sprintf(outp, "Joules PKG: %0X\n", p->energy_pkg);
@@ -911,7 +913,7 @@ int format_counters(struct thread_data *t, struct core_data *c,
if (DO_BIC(BIC_TOD))
outp += sprintf(outp, "%10ld.%06ld\t", t->tv_end.tv_sec, t->tv_end.tv_usec);
- interval_float = tv_delta.tv_sec + tv_delta.tv_usec/1000000.0;
+ interval_float = t->tv_delta.tv_sec + t->tv_delta.tv_usec/1000000.0;
tsc = t->tsc * tsc_tweak;
@@ -1287,6 +1289,14 @@ delta_core(struct core_data *new, struct core_data *old)
}
}
+int soft_c1_residency_display(int bic)
+{
+ if (!DO_BIC(BIC_CPU_c1) || use_c1_residency_msr)
+ return 0;
+
+ return DO_BIC_READ(bic);
+}
+
/*
* old = new - old
*/
@@ -1309,6 +1319,7 @@ delta_thread(struct thread_data *new, struct thread_data *old,
* over-write old w/ new so we can print end of interval values
*/
+ timersub(&new->tv_begin, &old->tv_begin, &old->tv_delta);
old->tv_begin = new->tv_begin;
old->tv_end = new->tv_end;
@@ -1322,7 +1333,8 @@ delta_thread(struct thread_data *new, struct thread_data *old,
old->c1 = new->c1 - old->c1;
- if (DO_BIC(BIC_Avg_MHz) || DO_BIC(BIC_Busy) || DO_BIC(BIC_Bzy_MHz)) {
+ if (DO_BIC(BIC_Avg_MHz) || DO_BIC(BIC_Busy) || DO_BIC(BIC_Bzy_MHz) ||
+ soft_c1_residency_display(BIC_Avg_MHz)) {
if ((new->aperf > old->aperf) && (new->mperf > old->mperf)) {
old->aperf = new->aperf - old->aperf;
old->mperf = new->mperf - old->mperf;
@@ -1404,6 +1416,8 @@ void clear_counters(struct thread_data *t, struct core_data *c, struct pkg_data
t->tv_begin.tv_usec = 0;
t->tv_end.tv_sec = 0;
t->tv_end.tv_usec = 0;
+ t->tv_delta.tv_sec = 0;
+ t->tv_delta.tv_usec = 0;
t->tsc = 0;
t->aperf = 0;
@@ -1573,6 +1587,9 @@ void compute_average(struct thread_data *t, struct core_data *c,
for_all_cpus(sum_counters, t, c, p);
+ /* Use the global time delta for the average. */
+ average.threads.tv_delta = tv_delta;
+
average.threads.tsc /= topo.num_cpus;
average.threads.aperf /= topo.num_cpus;
average.threads.mperf /= topo.num_cpus;
@@ -1714,7 +1731,7 @@ void get_apic_id(struct thread_data *t)
if (!DO_BIC(BIC_X2APIC))
return;
- if (authentic_amd) {
+ if (authentic_amd || hygon_genuine) {
unsigned int topology_extensions;
if (max_extended_level < 0x8000001e)
@@ -1762,19 +1779,20 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p)
struct msr_counter *mp;
int i;
- gettimeofday(&t->tv_begin, (struct timezone *)NULL);
-
if (cpu_migrate(cpu)) {
fprintf(outf, "Could not migrate to CPU %d\n", cpu);
return -1;
}
+ gettimeofday(&t->tv_begin, (struct timezone *)NULL);
+
if (first_counter_read)
get_apic_id(t);
retry:
t->tsc = rdtsc(); /* we are running on local CPU of interest */
- if (DO_BIC(BIC_Avg_MHz) || DO_BIC(BIC_Busy) || DO_BIC(BIC_Bzy_MHz)) {
+ if (DO_BIC(BIC_Avg_MHz) || DO_BIC(BIC_Busy) || DO_BIC(BIC_Bzy_MHz) ||
+ soft_c1_residency_display(BIC_Avg_MHz)) {
unsigned long long tsc_before, tsc_between, tsc_after, aperf_time, mperf_time;
/*
@@ -1851,20 +1869,20 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p)
if (!(t->flags & CPU_IS_FIRST_THREAD_IN_CORE))
goto done;
- if (DO_BIC(BIC_CPU_c3)) {
+ if (DO_BIC(BIC_CPU_c3) || soft_c1_residency_display(BIC_CPU_c3)) {
if (get_msr(cpu, MSR_CORE_C3_RESIDENCY, &c->c3))
return -6;
}
- if (DO_BIC(BIC_CPU_c6) && !do_knl_cstates) {
+ if ((DO_BIC(BIC_CPU_c6) || soft_c1_residency_display(BIC_CPU_c6)) && !do_knl_cstates) {
if (get_msr(cpu, MSR_CORE_C6_RESIDENCY, &c->c6))
return -7;
- } else if (do_knl_cstates) {
+ } else if (do_knl_cstates || soft_c1_residency_display(BIC_CPU_c6)) {
if (get_msr(cpu, MSR_KNL_CORE_C6_RESIDENCY, &c->c6))
return -7;
}
- if (DO_BIC(BIC_CPU_c7))
+ if (DO_BIC(BIC_CPU_c7) || soft_c1_residency_display(BIC_CPU_c7))
if (get_msr(cpu, MSR_CORE_C7_RESIDENCY, &c->c7))
return -8;
@@ -2912,6 +2930,7 @@ int snapshot_cpu_lpi_us(void)
if (retval != 1) {
fprintf(stderr, "Disabling Low Power Idle CPU output\n");
BIC_NOT_PRESENT(BIC_CPU_LPI);
+ fclose(fp);
return -1;
}
@@ -2938,6 +2957,7 @@ int snapshot_sys_lpi_us(void)
if (retval != 1) {
fprintf(stderr, "Disabling Low Power Idle System output\n");
BIC_NOT_PRESENT(BIC_SYS_LPI);
+ fclose(fp);
return -1;
}
fclose(fp);
@@ -2985,8 +3005,6 @@ static void signal_handler (int signal)
fprintf(stderr, "SIGUSR1\n");
break;
}
- /* make sure this manually-invoked interval is at least 1ms long */
- nanosleep(&one_msec, NULL);
}
void setup_signal_handler(void)
@@ -3005,29 +3023,38 @@ void setup_signal_handler(void)
void do_sleep(void)
{
- struct timeval select_timeout;
+ struct timeval tout;
+ struct timespec rest;
fd_set readfds;
int retval;
FD_ZERO(&readfds);
FD_SET(0, &readfds);
- if (!isatty(fileno(stdin))) {
+ if (ignore_stdin) {
nanosleep(&interval_ts, NULL);
return;
}
- select_timeout = interval_tv;
- retval = select(1, &readfds, NULL, NULL, &select_timeout);
+ tout = interval_tv;
+ retval = select(1, &readfds, NULL, NULL, &tout);
if (retval == 1) {
switch (getc(stdin)) {
case 'q':
exit_requested = 1;
break;
+ case EOF:
+ /*
+ * 'stdin' is a pipe closed on the other end. There
+ * won't be any further input.
+ */
+ ignore_stdin = 1;
+ /* Sleep the rest of the time */
+ rest.tv_sec = (tout.tv_sec + tout.tv_usec / 1000000);
+ rest.tv_nsec = (tout.tv_usec % 1000000) * 1000;
+ nanosleep(&rest, NULL);
}
- /* make sure this manually-invoked interval is at least 1ms long */
- nanosleep(&one_msec, NULL);
}
}
@@ -3209,6 +3236,7 @@ int probe_nhm_msrs(unsigned int family, unsigned int model)
break;
case INTEL_FAM6_HASWELL_CORE: /* HSW */
case INTEL_FAM6_HASWELL_X: /* HSX */
+ case INTEL_FAM6_HASWELL_ULT: /* HSW */
case INTEL_FAM6_HASWELL_GT3E: /* HSW */
case INTEL_FAM6_BROADWELL_CORE: /* BDW */
case INTEL_FAM6_BROADWELL_GT3E: /* BDW */
@@ -3405,6 +3433,7 @@ int has_config_tdp(unsigned int family, unsigned int model)
case INTEL_FAM6_IVYBRIDGE: /* IVB */
case INTEL_FAM6_HASWELL_CORE: /* HSW */
case INTEL_FAM6_HASWELL_X: /* HSX */
+ case INTEL_FAM6_HASWELL_ULT: /* HSW */
case INTEL_FAM6_HASWELL_GT3E: /* HSW */
case INTEL_FAM6_BROADWELL_CORE: /* BDW */
case INTEL_FAM6_BROADWELL_GT3E: /* BDW */
@@ -3803,6 +3832,7 @@ double get_tdp_amd(unsigned int family)
{
switch (family) {
case 0x17:
+ case 0x18:
default:
/* This is the max stock TDP of HEDT/Server Fam17h chips */
return 250.0;
@@ -3841,6 +3871,7 @@ void rapl_probe_intel(unsigned int family, unsigned int model)
case INTEL_FAM6_SANDYBRIDGE:
case INTEL_FAM6_IVYBRIDGE:
case INTEL_FAM6_HASWELL_CORE: /* HSW */
+ case INTEL_FAM6_HASWELL_ULT: /* HSW */
case INTEL_FAM6_HASWELL_GT3E: /* HSW */
case INTEL_FAM6_BROADWELL_CORE: /* BDW */
case INTEL_FAM6_BROADWELL_GT3E: /* BDW */
@@ -3982,6 +4013,7 @@ void rapl_probe_amd(unsigned int family, unsigned int model)
switch (family) {
case 0x17: /* Zen, Zen+ */
+ case 0x18: /* Hygon Dhyana */
do_rapl = RAPL_AMD_F17H | RAPL_PER_CORE_ENERGY;
if (rapl_joules) {
BIC_PRESENT(BIC_Pkg_J);
@@ -4002,7 +4034,7 @@ void rapl_probe_amd(unsigned int family, unsigned int model)
rapl_energy_units = ldexp(1.0, -(msr >> 8 & 0x1f));
rapl_power_units = ldexp(1.0, -(msr & 0xf));
- tdp = get_tdp_amd(model);
+ tdp = get_tdp_amd(family);
rapl_joule_counter_range = 0xFFFFFFFF * rapl_energy_units / tdp;
if (!quiet)
@@ -4018,7 +4050,7 @@ void rapl_probe(unsigned int family, unsigned int model)
{
if (genuine_intel)
rapl_probe_intel(family, model);
- if (authentic_amd)
+ if (authentic_amd || hygon_genuine)
rapl_probe_amd(family, model);
}
@@ -4032,6 +4064,7 @@ void perf_limit_reasons_probe(unsigned int family, unsigned int model)
switch (model) {
case INTEL_FAM6_HASWELL_CORE: /* HSW */
+ case INTEL_FAM6_HASWELL_ULT: /* HSW */
case INTEL_FAM6_HASWELL_GT3E: /* HSW */
do_gfx_perf_limit_reasons = 1;
case INTEL_FAM6_HASWELL_X: /* HSX */
@@ -4251,6 +4284,7 @@ int has_snb_msrs(unsigned int family, unsigned int model)
case INTEL_FAM6_IVYBRIDGE_X: /* IVB Xeon */
case INTEL_FAM6_HASWELL_CORE: /* HSW */
case INTEL_FAM6_HASWELL_X: /* HSW */
+ case INTEL_FAM6_HASWELL_ULT: /* HSW */
case INTEL_FAM6_HASWELL_GT3E: /* HSW */
case INTEL_FAM6_BROADWELL_CORE: /* BDW */
case INTEL_FAM6_BROADWELL_GT3E: /* BDW */
@@ -4267,7 +4301,7 @@ int has_snb_msrs(unsigned int family, unsigned int model)
}
/*
- * HSW adds support for additional MSRs:
+ * HSW ULT added support for C8/C9/C10 MSRs:
*
* MSR_PKG_C8_RESIDENCY 0x00000630
* MSR_PKG_C9_RESIDENCY 0x00000631
@@ -4278,13 +4312,13 @@ int has_snb_msrs(unsigned int family, unsigned int model)
* MSR_PKGC10_IRTL 0x00000635
*
*/
-int has_hsw_msrs(unsigned int family, unsigned int model)
+int has_c8910_msrs(unsigned int family, unsigned int model)
{
if (!genuine_intel)
return 0;
switch (model) {
- case INTEL_FAM6_HASWELL_CORE:
+ case INTEL_FAM6_HASWELL_ULT: /* HSW */
case INTEL_FAM6_BROADWELL_CORE: /* BDW */
case INTEL_FAM6_SKYLAKE_MOBILE: /* SKL */
case INTEL_FAM6_CANNONLAKE_MOBILE: /* CNL */
@@ -4568,9 +4602,6 @@ unsigned int intel_model_duplicates(unsigned int model)
case INTEL_FAM6_XEON_PHI_KNM:
return INTEL_FAM6_XEON_PHI_KNL;
- case INTEL_FAM6_HASWELL_ULT:
- return INTEL_FAM6_HASWELL_CORE;
-
case INTEL_FAM6_BROADWELL_X:
case INTEL_FAM6_BROADWELL_XEON_D: /* BDX-DE */
return INTEL_FAM6_BROADWELL_X;
@@ -4582,7 +4613,11 @@ unsigned int intel_model_duplicates(unsigned int model)
return INTEL_FAM6_SKYLAKE_MOBILE;
case INTEL_FAM6_ICELAKE_MOBILE:
+ case INTEL_FAM6_ICELAKE_NNPI:
return INTEL_FAM6_CANNONLAKE_MOBILE;
+
+ case INTEL_FAM6_ATOM_TREMONT_X:
+ return INTEL_FAM6_ATOM_GOLDMONT_X;
}
return model;
}
@@ -4600,6 +4635,8 @@ void process_cpuid()
genuine_intel = 1;
else if (ebx == 0x68747541 && ecx == 0x444d4163 && edx == 0x69746e65)
authentic_amd = 1;
+ else if (ebx == 0x6f677948 && ecx == 0x656e6975 && edx == 0x6e65476e)
+ hygon_genuine = 1;
if (!quiet)
fprintf(outf, "CPUID(0): %.4s%.4s%.4s ",
@@ -4820,12 +4857,12 @@ void process_cpuid()
BIC_NOT_PRESENT(BIC_CPU_c7);
BIC_NOT_PRESENT(BIC_Pkgpc7);
}
- if (has_hsw_msrs(family, model)) {
+ if (has_c8910_msrs(family, model)) {
BIC_PRESENT(BIC_Pkgpc8);
BIC_PRESENT(BIC_Pkgpc9);
BIC_PRESENT(BIC_Pkgpc10);
}
- do_irtl_hsw = has_hsw_msrs(family, model);
+ do_irtl_hsw = has_c8910_msrs(family, model);
if (has_skl_msrs(family, model)) {
BIC_PRESENT(BIC_Totl_c0);
BIC_PRESENT(BIC_Any_c0);
@@ -5123,7 +5160,7 @@ int initialize_counters(int cpu_id)
void allocate_output_buffer()
{
- output_buffer = calloc(1, (1 + topo.num_cpus) * 1024);
+ output_buffer = calloc(1, (1 + topo.num_cpus) * 2048);
outp = output_buffer;
if (outp == NULL)
err(-1, "calloc output buffer");
@@ -5269,7 +5306,7 @@ int get_and_dump_counters(void)
}
void print_version() {
- fprintf(outf, "turbostat version 19.03.20"
+ fprintf(outf, "turbostat version 19.08.31"
" - Len Brown <lenb@kernel.org>\n");
}
diff --git a/tools/power/x86/x86_energy_perf_policy/Makefile b/tools/power/x86/x86_energy_perf_policy/Makefile
index 1fdeef8..666b325 100644
--- a/tools/power/x86/x86_energy_perf_policy/Makefile
+++ b/tools/power/x86/x86_energy_perf_policy/Makefile
@@ -9,8 +9,9 @@
endif
x86_energy_perf_policy : x86_energy_perf_policy.c
-override CFLAGS += -Wall -I../../../include
+override CFLAGS += -O2 -Wall -I../../../include
override CFLAGS += -DMSRHEADER='"../../../../arch/x86/include/asm/msr-index.h"'
+override CFLAGS += -D_FORTIFY_SOURCE=2
%: %.c
@mkdir -p $(BUILD_OUTPUT)
diff --git a/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.8 b/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.8
index 17db1c3..78c6361 100644
--- a/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.8
+++ b/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.8
@@ -40,7 +40,7 @@
Hardware P-States (HWP) are effectively an expansion of hardware
P-state control from the opportunistic turbo-mode P-state range
to include the entire range of available P-states.
-On Broadwell Xeon, the initial HWP implementation, EBP influenced HWP.
+On Broadwell Xeon, the initial HWP implementation, EPB influenced HWP.
That influence was removed in subsequent generations,
where it was moved to the
Energy_Performance_Preference (EPP) field in
diff --git a/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c b/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c
index 34a796b..3fe1eed 100644
--- a/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c
+++ b/tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c
@@ -545,7 +545,7 @@ void cmdline(int argc, char **argv)
progname = argv[0];
- while ((opt = getopt_long_only(argc, argv, "+a:c:dD:E:e:f:m:M:rt:u:vw",
+ while ((opt = getopt_long_only(argc, argv, "+a:c:dD:E:e:f:m:M:rt:u:vw:",
long_options, &option_index)) != -1) {
switch (opt) {
case 'a':
@@ -1259,6 +1259,15 @@ void probe_dev_msr(void)
if (system("/sbin/modprobe msr > /dev/null 2>&1"))
err(-5, "no /dev/cpu/0/msr, Try \"# modprobe msr\" ");
}
+
+static void get_cpuid_or_exit(unsigned int leaf,
+ unsigned int *eax, unsigned int *ebx,
+ unsigned int *ecx, unsigned int *edx)
+{
+ if (!__get_cpuid(leaf, eax, ebx, ecx, edx))
+ errx(1, "Processor not supported\n");
+}
+
/*
* early_cpuid()
* initialize turbo_is_enabled, has_hwp, has_epb
@@ -1266,15 +1275,10 @@ void probe_dev_msr(void)
*/
void early_cpuid(void)
{
- unsigned int eax, ebx, ecx, edx, max_level;
+ unsigned int eax, ebx, ecx, edx;
unsigned int fms, family, model;
- __get_cpuid(0, &max_level, &ebx, &ecx, &edx);
-
- if (max_level < 6)
- errx(1, "Processor not supported\n");
-
- __get_cpuid(1, &fms, &ebx, &ecx, &edx);
+ get_cpuid_or_exit(1, &fms, &ebx, &ecx, &edx);
family = (fms >> 8) & 0xf;
model = (fms >> 4) & 0xf;
if (family == 6 || family == 0xf)
@@ -1288,7 +1292,7 @@ void early_cpuid(void)
bdx_highest_ratio = msr & 0xFF;
}
- __get_cpuid(0x6, &eax, &ebx, &ecx, &edx);
+ get_cpuid_or_exit(0x6, &eax, &ebx, &ecx, &edx);
turbo_is_enabled = (eax >> 1) & 1;
has_hwp = (eax >> 7) & 1;
has_epb = (ecx >> 3) & 1;
@@ -1306,7 +1310,7 @@ void parse_cpuid(void)
eax = ebx = ecx = edx = 0;
- __get_cpuid(0, &max_level, &ebx, &ecx, &edx);
+ get_cpuid_or_exit(0, &max_level, &ebx, &ecx, &edx);
if (ebx == 0x756e6547 && edx == 0x49656e69 && ecx == 0x6c65746e)
genuine_intel = 1;
@@ -1315,7 +1319,7 @@ void parse_cpuid(void)
fprintf(stderr, "CPUID(0): %.4s%.4s%.4s ",
(char *)&ebx, (char *)&edx, (char *)&ecx);
- __get_cpuid(1, &fms, &ebx, &ecx, &edx);
+ get_cpuid_or_exit(1, &fms, &ebx, &ecx, &edx);
family = (fms >> 8) & 0xf;
model = (fms >> 4) & 0xf;
stepping = fms & 0xf;
@@ -1340,7 +1344,7 @@ void parse_cpuid(void)
errx(1, "CPUID: no MSR");
- __get_cpuid(0x6, &eax, &ebx, &ecx, &edx);
+ get_cpuid_or_exit(0x6, &eax, &ebx, &ecx, &edx);
/* turbo_is_enabled already set */
/* has_hwp already set */
has_hwp_notify = eax & (1 << 8);
diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 90f70d2..60c9338 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -42,4 +42,5 @@
test_sockopt
test_sockopt_sk
test_sockopt_multi
+test_sockopt_inherit
test_tcp_rtt
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 29001f9..9eef5ed 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -29,11 +29,14 @@
test_cgroup_storage test_select_reuseport test_section_names \
test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \
test_btf_dump test_cgroup_attach xdping test_sockopt test_sockopt_sk \
- test_sockopt_multi test_tcp_rtt
+ test_sockopt_multi test_sockopt_inherit test_tcp_rtt
BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c)))
TEST_GEN_FILES = $(BPF_OBJ_FILES)
+BTF_C_FILES = $(wildcard progs/btf_dump_test_case_*.c)
+TEST_FILES = $(BTF_C_FILES)
+
# Also test sub-register code-gen if LLVM has eBPF v3 processor support which
# contains both ALU32 and JMP32 instructions.
SUBREG_CODEGEN := $(shell echo "int cal(int a) { return a > 0; }" | \
@@ -63,12 +66,14 @@
test_tcp_check_syncookie.sh \
test_tc_tunnel.sh \
test_tc_edt.sh \
- test_xdping.sh
+ test_xdping.sh \
+ test_bpftool_build.sh
TEST_PROGS_EXTENDED := with_addr.sh \
with_tunnels.sh \
tcp_client.py \
- tcp_server.py
+ tcp_server.py \
+ test_xdp_vlan.sh
# Compile but not part of 'make run_tests'
TEST_GEN_PROGS_EXTENDED = test_libbpf_open test_sock_addr test_skb_cgroup_id_user \
@@ -111,6 +116,7 @@
$(OUTPUT)/test_sockopt: cgroup_helpers.c
$(OUTPUT)/test_sockopt_sk: cgroup_helpers.c
$(OUTPUT)/test_sockopt_multi: cgroup_helpers.c
+$(OUTPUT)/test_sockopt_inherit: cgroup_helpers.c
$(OUTPUT)/test_tcp_rtt: cgroup_helpers.c
.PHONY: force
diff --git a/tools/testing/selftests/bpf/bpf_endian.h b/tools/testing/selftests/bpf/bpf_endian.h
index 05f036d..fbe28008 100644
--- a/tools/testing/selftests/bpf/bpf_endian.h
+++ b/tools/testing/selftests/bpf/bpf_endian.h
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: GPL-2.0 */
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __BPF_ENDIAN__
#define __BPF_ENDIAN__
@@ -29,6 +29,10 @@
# define __bpf_htonl(x) __builtin_bswap32(x)
# define __bpf_constant_ntohl(x) ___constant_swab32(x)
# define __bpf_constant_htonl(x) ___constant_swab32(x)
+# define __bpf_be64_to_cpu(x) __builtin_bswap64(x)
+# define __bpf_cpu_to_be64(x) __builtin_bswap64(x)
+# define __bpf_constant_be64_to_cpu(x) ___constant_swab64(x)
+# define __bpf_constant_cpu_to_be64(x) ___constant_swab64(x)
#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
# define __bpf_ntohs(x) (x)
# define __bpf_htons(x) (x)
@@ -38,6 +42,10 @@
# define __bpf_htonl(x) (x)
# define __bpf_constant_ntohl(x) (x)
# define __bpf_constant_htonl(x) (x)
+# define __bpf_be64_to_cpu(x) (x)
+# define __bpf_cpu_to_be64(x) (x)
+# define __bpf_constant_be64_to_cpu(x) (x)
+# define __bpf_constant_cpu_to_be64(x) (x)
#else
# error "Fix your compiler's __BYTE_ORDER__?!"
#endif
@@ -54,5 +62,11 @@
#define bpf_ntohl(x) \
(__builtin_constant_p(x) ? \
__bpf_constant_ntohl(x) : __bpf_ntohl(x))
+#define bpf_cpu_to_be64(x) \
+ (__builtin_constant_p(x) ? \
+ __bpf_constant_cpu_to_be64(x) : __bpf_cpu_to_be64(x))
+#define bpf_be64_to_cpu(x) \
+ (__builtin_constant_p(x) ? \
+ __bpf_constant_be64_to_cpu(x) : __bpf_be64_to_cpu(x))
#endif /* __BPF_ENDIAN__ */
diff --git a/tools/testing/selftests/bpf/bpf_helpers.h b/tools/testing/selftests/bpf/bpf_helpers.h
index 8b503ea..6c4930bc 100644
--- a/tools/testing/selftests/bpf/bpf_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_helpers.h
@@ -1,4 +1,4 @@
-/* SPDX-License-Identifier: GPL-2.0 */
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __BPF_HELPERS_H
#define __BPF_HELPERS_H
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index f7a0744..5dc109f 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -34,3 +34,4 @@
CONFIG_MPLS_ROUTING=m
CONFIG_MPLS_IPTUNNEL=m
CONFIG_IPV6_SIT=m
+CONFIG_BPF_JIT=y
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c b/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c
index fb5840a..f100298 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_obj_id.c
@@ -48,16 +48,17 @@ void test_bpf_obj_id(void)
/* test_obj_id.o is a dumb prog. It should never fail
* to load.
*/
- if (err)
- error_cnt++;
- assert(!err);
+ if (CHECK_FAIL(err))
+ continue;
/* Insert a magic value to the map */
map_fds[i] = bpf_find_map(__func__, objs[i], "test_map_id");
- assert(map_fds[i] >= 0);
+ if (CHECK_FAIL(map_fds[i] < 0))
+ goto done;
err = bpf_map_update_elem(map_fds[i], &array_key,
&array_magic_value, 0);
- assert(!err);
+ if (CHECK_FAIL(err))
+ goto done;
/* Check getting map info */
info_len = sizeof(struct bpf_map_info) * 2;
@@ -96,9 +97,11 @@ void test_bpf_obj_id(void)
prog_infos[i].map_ids = ptr_to_u64(map_ids + i);
prog_infos[i].nr_map_ids = 2;
err = clock_gettime(CLOCK_REALTIME, &real_time_ts);
- assert(!err);
+ if (CHECK_FAIL(err))
+ goto done;
err = clock_gettime(CLOCK_BOOTTIME, &boot_time_ts);
- assert(!err);
+ if (CHECK_FAIL(err))
+ goto done;
err = bpf_obj_get_info_by_fd(prog_fds[i], &prog_infos[i],
&info_len);
load_time = (real_time_ts.tv_sec - boot_time_ts.tv_sec)
@@ -224,7 +227,8 @@ void test_bpf_obj_id(void)
nr_id_found++;
err = bpf_map_lookup_elem(map_fd, &array_key, &array_value);
- assert(!err);
+ if (CHECK_FAIL(err))
+ goto done;
err = bpf_obj_get_info_by_fd(map_fd, &map_info, &info_len);
CHECK(err || info_len != sizeof(struct bpf_map_info) ||
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c b/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c
index 1a1eae3..1c01ee2 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c
@@ -28,8 +28,6 @@ static int check_load(const char *file, enum bpf_prog_type type)
attr.prog_flags = BPF_F_TEST_RND_HI32;
err = bpf_prog_load_xattr(&attr, &obj, &prog_fd);
bpf_object__close(obj);
- if (err)
- error_cnt++;
return err;
}
@@ -105,12 +103,7 @@ void test_bpf_verif_scale(void)
continue;
err = check_load(test->file, test->attach_type);
- if (test->fails) { /* expected to fail */
- if (err)
- error_cnt--;
- else
- error_cnt++;
- }
+ CHECK_FAIL(err && !test->fails);
}
if (env.verifier_stats)
diff --git a/tools/testing/selftests/bpf/prog_tests/flow_dissector.c b/tools/testing/selftests/bpf/prog_tests/flow_dissector.c
index 6892b88..9256389 100644
--- a/tools/testing/selftests/bpf/prog_tests/flow_dissector.c
+++ b/tools/testing/selftests/bpf/prog_tests/flow_dissector.c
@@ -344,7 +344,6 @@ struct test tests[] = {
.tcp.dest = 8080,
},
.keys = {
- .nhoff = 0,
.nhoff = ETH_HLEN,
.thoff = ETH_HLEN + sizeof(struct iphdr) +
sizeof(struct iphdr),
@@ -452,10 +451,8 @@ void test_flow_dissector(void)
err = bpf_flow_load(&obj, "./bpf_flow.o", "flow_dissector",
"jmp_table", "last_dissection", &prog_fd, &keys_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
for (i = 0; i < ARRAY_SIZE(tests); i++) {
struct bpf_flow_keys flow_keys;
diff --git a/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c b/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c
index 3d59b3c..eba9a97 100644
--- a/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c
+++ b/tools/testing/selftests/bpf/prog_tests/get_stack_raw_tp.c
@@ -135,10 +135,7 @@ void test_get_stack_raw_tp(void)
exp_cnt -= err;
}
- goto close_prog_noerr;
close_prog:
- error_cnt++;
-close_prog_noerr:
if (!IS_ERR_OR_NULL(link))
bpf_link__destroy(link);
if (!IS_ERR_OR_NULL(pb))
diff --git a/tools/testing/selftests/bpf/prog_tests/global_data.c b/tools/testing/selftests/bpf/prog_tests/global_data.c
index d011079..c680926 100644
--- a/tools/testing/selftests/bpf/prog_tests/global_data.c
+++ b/tools/testing/selftests/bpf/prog_tests/global_data.c
@@ -7,10 +7,8 @@ static void test_global_data_number(struct bpf_object *obj, __u32 duration)
uint64_t num;
map_fd = bpf_find_map(__func__, obj, "result_number");
- if (map_fd < 0) {
- error_cnt++;
+ if (CHECK_FAIL(map_fd < 0))
return;
- }
struct {
char *name;
@@ -44,10 +42,8 @@ static void test_global_data_string(struct bpf_object *obj, __u32 duration)
char str[32];
map_fd = bpf_find_map(__func__, obj, "result_string");
- if (map_fd < 0) {
- error_cnt++;
+ if (CHECK_FAIL(map_fd < 0))
return;
- }
struct {
char *name;
@@ -81,10 +77,8 @@ static void test_global_data_struct(struct bpf_object *obj, __u32 duration)
struct foo val;
map_fd = bpf_find_map(__func__, obj, "result_struct");
- if (map_fd < 0) {
- error_cnt++;
+ if (CHECK_FAIL(map_fd < 0))
return;
- }
struct {
char *name;
@@ -112,16 +106,12 @@ static void test_global_data_rdonly(struct bpf_object *obj, __u32 duration)
__u8 *buff;
map = bpf_object__find_map_by_name(obj, "test_glo.rodata");
- if (!map || !bpf_map__is_internal(map)) {
- error_cnt++;
+ if (CHECK_FAIL(!map || !bpf_map__is_internal(map)))
return;
- }
map_fd = bpf_map__fd(map);
- if (map_fd < 0) {
- error_cnt++;
+ if (CHECK_FAIL(map_fd < 0))
return;
- }
buff = malloc(bpf_map__def(map)->value_size);
if (buff)
diff --git a/tools/testing/selftests/bpf/prog_tests/l4lb_all.c b/tools/testing/selftests/bpf/prog_tests/l4lb_all.c
index 20ddca8..eaf6459 100644
--- a/tools/testing/selftests/bpf/prog_tests/l4lb_all.c
+++ b/tools/testing/selftests/bpf/prog_tests/l4lb_all.c
@@ -30,10 +30,8 @@ static void test_l4lb(const char *file)
u32 *magic = (u32 *)buf;
err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
map_fd = bpf_find_map(__func__, obj, "vip_map");
if (map_fd < 0)
@@ -72,10 +70,9 @@ static void test_l4lb(const char *file)
bytes += stats[i].bytes;
pkts += stats[i].pkts;
}
- if (bytes != MAGIC_BYTES * NUM_ITER * 2 || pkts != NUM_ITER * 2) {
- error_cnt++;
+ if (CHECK_FAIL(bytes != MAGIC_BYTES * NUM_ITER * 2 ||
+ pkts != NUM_ITER * 2))
printf("test_l4lb:FAIL:stats %lld %lld\n", bytes, pkts);
- }
out:
bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/map_lock.c b/tools/testing/selftests/bpf/prog_tests/map_lock.c
index ee99368..8f91f18 100644
--- a/tools/testing/selftests/bpf/prog_tests/map_lock.c
+++ b/tools/testing/selftests/bpf/prog_tests/map_lock.c
@@ -8,14 +8,12 @@ static void *parallel_map_access(void *arg)
for (i = 0; i < 10000; i++) {
err = bpf_map_lookup_elem_flags(map_fd, &key, vars, BPF_F_LOCK);
- if (err) {
+ if (CHECK_FAIL(err)) {
printf("lookup failed\n");
- error_cnt++;
goto out;
}
- if (vars[0] != 0) {
+ if (CHECK_FAIL(vars[0] != 0)) {
printf("lookup #%d var[0]=%d\n", i, vars[0]);
- error_cnt++;
goto out;
}
rnd = vars[1];
@@ -24,7 +22,7 @@ static void *parallel_map_access(void *arg)
continue;
printf("lookup #%d var[1]=%d var[%d]=%d\n",
i, rnd, j, vars[j]);
- error_cnt++;
+ CHECK_FAIL(vars[j] != rnd);
goto out;
}
}
@@ -42,34 +40,36 @@ void test_map_lock(void)
void *ret;
err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
- if (err) {
+ if (CHECK_FAIL(err)) {
printf("test_map_lock:bpf_prog_load errno %d\n", errno);
goto close_prog;
}
map_fd[0] = bpf_find_map(__func__, obj, "hash_map");
- if (map_fd[0] < 0)
+ if (CHECK_FAIL(map_fd[0] < 0))
goto close_prog;
map_fd[1] = bpf_find_map(__func__, obj, "array_map");
- if (map_fd[1] < 0)
+ if (CHECK_FAIL(map_fd[1] < 0))
goto close_prog;
bpf_map_update_elem(map_fd[0], &key, vars, BPF_F_LOCK);
for (i = 0; i < 4; i++)
- assert(pthread_create(&thread_id[i], NULL,
- &spin_lock_thread, &prog_fd) == 0);
+ if (CHECK_FAIL(pthread_create(&thread_id[i], NULL,
+ &spin_lock_thread, &prog_fd)))
+ goto close_prog;
for (i = 4; i < 6; i++)
- assert(pthread_create(&thread_id[i], NULL,
- ¶llel_map_access, &map_fd[i - 4]) == 0);
+ if (CHECK_FAIL(pthread_create(&thread_id[i], NULL,
+ ¶llel_map_access,
+ &map_fd[i - 4])))
+ goto close_prog;
for (i = 0; i < 4; i++)
- assert(pthread_join(thread_id[i], &ret) == 0 &&
- ret == (void *)&prog_fd);
+ if (CHECK_FAIL(pthread_join(thread_id[i], &ret) ||
+ ret != (void *)&prog_fd))
+ goto close_prog;
for (i = 4; i < 6; i++)
- assert(pthread_join(thread_id[i], &ret) == 0 &&
- ret == (void *)&map_fd[i - 4]);
- goto close_prog_noerr;
+ if (CHECK_FAIL(pthread_join(thread_id[i], &ret) ||
+ ret != (void *)&map_fd[i - 4]))
+ goto close_prog;
close_prog:
- error_cnt++;
-close_prog_noerr:
bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/pkt_access.c b/tools/testing/selftests/bpf/prog_tests/pkt_access.c
index 4ecfd72..a2537df 100644
--- a/tools/testing/selftests/bpf/prog_tests/pkt_access.c
+++ b/tools/testing/selftests/bpf/prog_tests/pkt_access.c
@@ -9,10 +9,8 @@ void test_pkt_access(void)
int err, prog_fd;
err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
err = bpf_prog_test_run(prog_fd, 100000, &pkt_v4, sizeof(pkt_v4),
NULL, NULL, &retval, &duration);
diff --git a/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c b/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c
index ac0d434..5f7aea6 100644
--- a/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c
+++ b/tools/testing/selftests/bpf/prog_tests/pkt_md_access.c
@@ -9,10 +9,8 @@ void test_pkt_md_access(void)
int err, prog_fd;
err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
err = bpf_prog_test_run(prog_fd, 10, &pkt_v4, sizeof(pkt_v4),
NULL, NULL, &retval, &duration);
diff --git a/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c b/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c
index e60cd5f..faccc66 100644
--- a/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c
+++ b/tools/testing/selftests/bpf/prog_tests/queue_stack_map.c
@@ -27,10 +27,8 @@ static void test_queue_stack_map_by_type(int type)
return;
err = bpf_prog_load(file, BPF_PROG_TYPE_SCHED_CLS, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
map_in_fd = bpf_find_map(__func__, obj, "map_in");
if (map_in_fd < 0)
@@ -43,10 +41,8 @@ static void test_queue_stack_map_by_type(int type)
/* Push 32 elements to the input map */
for (i = 0; i < MAP_SIZE; i++) {
err = bpf_map_update_elem(map_in_fd, NULL, &vals[i], 0);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
goto out;
- }
}
/* The eBPF program pushes iph.saddr in the output map,
diff --git a/tools/testing/selftests/bpf/prog_tests/reference_tracking.c b/tools/testing/selftests/bpf/prog_tests/reference_tracking.c
index 4a4f428..5c78e2b 100644
--- a/tools/testing/selftests/bpf/prog_tests/reference_tracking.c
+++ b/tools/testing/selftests/bpf/prog_tests/reference_tracking.c
@@ -10,10 +10,8 @@ void test_reference_tracking(void)
int err = 0;
obj = bpf_object__open(file);
- if (IS_ERR(obj)) {
- error_cnt++;
+ if (CHECK_FAIL(IS_ERR(obj)))
return;
- }
bpf_object__for_each_program(prog, obj) {
const char *title;
diff --git a/tools/testing/selftests/bpf/prog_tests/send_signal.c b/tools/testing/selftests/bpf/prog_tests/send_signal.c
index 1575f0a..b607112 100644
--- a/tools/testing/selftests/bpf/prog_tests/send_signal.c
+++ b/tools/testing/selftests/bpf/prog_tests/send_signal.c
@@ -8,7 +8,7 @@ static void sigusr1_handler(int signum)
sigusr1_received++;
}
-static int test_send_signal_common(struct perf_event_attr *attr,
+static void test_send_signal_common(struct perf_event_attr *attr,
int prog_type,
const char *test_name)
{
@@ -23,13 +23,13 @@ static int test_send_signal_common(struct perf_event_attr *attr,
if (CHECK(pipe(pipe_c2p), test_name,
"pipe pipe_c2p error: %s\n", strerror(errno)))
- goto no_fork_done;
+ return;
if (CHECK(pipe(pipe_p2c), test_name,
"pipe pipe_p2c error: %s\n", strerror(errno))) {
close(pipe_c2p[0]);
close(pipe_c2p[1]);
- goto no_fork_done;
+ return;
}
pid = fork();
@@ -38,7 +38,7 @@ static int test_send_signal_common(struct perf_event_attr *attr,
close(pipe_c2p[1]);
close(pipe_p2c[0]);
close(pipe_p2c[1]);
- goto no_fork_done;
+ return;
}
if (pid == 0) {
@@ -125,7 +125,7 @@ static int test_send_signal_common(struct perf_event_attr *attr,
goto disable_pmu;
}
- err = CHECK(buf[0] != '2', test_name, "incorrect result\n");
+ CHECK(buf[0] != '2', test_name, "incorrect result\n");
/* notify child safe to exit */
write(pipe_p2c[1], buf, 1);
@@ -138,11 +138,9 @@ static int test_send_signal_common(struct perf_event_attr *attr,
close(pipe_c2p[0]);
close(pipe_p2c[1]);
wait(NULL);
-no_fork_done:
- return err;
}
-static int test_send_signal_tracepoint(void)
+static void test_send_signal_tracepoint(void)
{
const char *id_path = "/sys/kernel/debug/tracing/events/syscalls/sys_enter_nanosleep/id";
struct perf_event_attr attr = {
@@ -159,21 +157,21 @@ static int test_send_signal_tracepoint(void)
if (CHECK(efd < 0, "tracepoint",
"open syscalls/sys_enter_nanosleep/id failure: %s\n",
strerror(errno)))
- return -1;
+ return;
bytes = read(efd, buf, sizeof(buf));
close(efd);
if (CHECK(bytes <= 0 || bytes >= sizeof(buf), "tracepoint",
"read syscalls/sys_enter_nanosleep/id failure: %s\n",
strerror(errno)))
- return -1;
+ return;
attr.config = strtol(buf, NULL, 0);
- return test_send_signal_common(&attr, BPF_PROG_TYPE_TRACEPOINT, "tracepoint");
+ test_send_signal_common(&attr, BPF_PROG_TYPE_TRACEPOINT, "tracepoint");
}
-static int test_send_signal_perf(void)
+static void test_send_signal_perf(void)
{
struct perf_event_attr attr = {
.sample_period = 1,
@@ -181,11 +179,11 @@ static int test_send_signal_perf(void)
.config = PERF_COUNT_SW_CPU_CLOCK,
};
- return test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT,
- "perf_sw_event");
+ test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT,
+ "perf_sw_event");
}
-static int test_send_signal_nmi(void)
+static void test_send_signal_nmi(void)
{
struct perf_event_attr attr = {
.sample_freq = 50,
@@ -204,25 +202,24 @@ static int test_send_signal_nmi(void)
if (errno == ENOENT) {
printf("%s:SKIP:no PERF_COUNT_HW_CPU_CYCLES\n",
__func__);
- return 0;
+ test__skip();
+ return;
}
/* Let the test fail with a more informative message */
} else {
close(pmu_fd);
}
- return test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT,
- "perf_hw_event");
+ test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT,
+ "perf_hw_event");
}
void test_send_signal(void)
{
- int ret = 0;
-
if (test__start_subtest("send_signal_tracepoint"))
- ret |= test_send_signal_tracepoint();
+ test_send_signal_tracepoint();
if (test__start_subtest("send_signal_perf"))
- ret |= test_send_signal_perf();
+ test_send_signal_perf();
if (test__start_subtest("send_signal_nmi"))
- ret |= test_send_signal_nmi();
+ test_send_signal_nmi();
}
diff --git a/tools/testing/selftests/bpf/prog_tests/spinlock.c b/tools/testing/selftests/bpf/prog_tests/spinlock.c
index 114ebe6..1ae00cd 100644
--- a/tools/testing/selftests/bpf/prog_tests/spinlock.c
+++ b/tools/testing/selftests/bpf/prog_tests/spinlock.c
@@ -11,19 +11,19 @@ void test_spinlock(void)
void *ret;
err = bpf_prog_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd);
- if (err) {
+ if (CHECK_FAIL(err)) {
printf("test_spin_lock:bpf_prog_load errno %d\n", errno);
goto close_prog;
}
for (i = 0; i < 4; i++)
- assert(pthread_create(&thread_id[i], NULL,
- &spin_lock_thread, &prog_fd) == 0);
+ if (CHECK_FAIL(pthread_create(&thread_id[i], NULL,
+ &spin_lock_thread, &prog_fd)))
+ goto close_prog;
+
for (i = 0; i < 4; i++)
- assert(pthread_join(thread_id[i], &ret) == 0 &&
- ret == (void *)&prog_fd);
- goto close_prog_noerr;
+ if (CHECK_FAIL(pthread_join(thread_id[i], &ret) ||
+ ret != (void *)&prog_fd))
+ goto close_prog;
close_prog:
- error_cnt++;
-close_prog_noerr:
bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id.c
index ac44fda..d841dce 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id.c
@@ -51,9 +51,10 @@ void test_stacktrace_build_id(void)
"err %d errno %d\n", err, errno))
goto disable_pmu;
- assert(system("dd if=/dev/urandom of=/dev/zero count=4 2> /dev/null")
- == 0);
- assert(system("./urandom_read") == 0);
+ if (CHECK_FAIL(system("dd if=/dev/urandom of=/dev/zero count=4 2> /dev/null")))
+ goto disable_pmu;
+ if (CHECK_FAIL(system("./urandom_read")))
+ goto disable_pmu;
/* disable stack trace collection */
key = 0;
val = 1;
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id_nmi.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id_nmi.c
index 9557b7df..f62aa0e 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id_nmi.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_build_id_nmi.c
@@ -82,9 +82,10 @@ void test_stacktrace_build_id_nmi(void)
"err %d errno %d\n", err, errno))
goto disable_pmu;
- assert(system("dd if=/dev/urandom of=/dev/zero count=4 2> /dev/null")
- == 0);
- assert(system("taskset 0x1 ./urandom_read 100000") == 0);
+ if (CHECK_FAIL(system("dd if=/dev/urandom of=/dev/zero count=4 2> /dev/null")))
+ goto disable_pmu;
+ if (CHECK_FAIL(system("taskset 0x1 ./urandom_read 100000")))
+ goto disable_pmu;
/* disable stack trace collection */
key = 0;
val = 1;
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c
index fc53933..37269d2 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_map.c
@@ -26,19 +26,19 @@ void test_stacktrace_map(void)
/* find map fds */
control_map_fd = bpf_find_map(__func__, obj, "control_map");
- if (control_map_fd < 0)
+ if (CHECK_FAIL(control_map_fd < 0))
goto disable_pmu;
stackid_hmap_fd = bpf_find_map(__func__, obj, "stackid_hmap");
- if (stackid_hmap_fd < 0)
+ if (CHECK_FAIL(stackid_hmap_fd < 0))
goto disable_pmu;
stackmap_fd = bpf_find_map(__func__, obj, "stackmap");
- if (stackmap_fd < 0)
+ if (CHECK_FAIL(stackmap_fd < 0))
goto disable_pmu;
stack_amap_fd = bpf_find_map(__func__, obj, "stack_amap");
- if (stack_amap_fd < 0)
+ if (CHECK_FAIL(stack_amap_fd < 0))
goto disable_pmu;
/* give some time for bpf program run */
@@ -55,23 +55,20 @@ void test_stacktrace_map(void)
err = compare_map_keys(stackid_hmap_fd, stackmap_fd);
if (CHECK(err, "compare_map_keys stackid_hmap vs. stackmap",
"err %d errno %d\n", err, errno))
- goto disable_pmu_noerr;
+ goto disable_pmu;
err = compare_map_keys(stackmap_fd, stackid_hmap_fd);
if (CHECK(err, "compare_map_keys stackmap vs. stackid_hmap",
"err %d errno %d\n", err, errno))
- goto disable_pmu_noerr;
+ goto disable_pmu;
stack_trace_len = PERF_MAX_STACK_DEPTH * sizeof(__u64);
err = compare_stack_ips(stackmap_fd, stack_amap_fd, stack_trace_len);
if (CHECK(err, "compare_stack_ips stackmap vs. stack_amap",
"err %d errno %d\n", err, errno))
- goto disable_pmu_noerr;
+ goto disable_pmu;
- goto disable_pmu_noerr;
disable_pmu:
- error_cnt++;
-disable_pmu_noerr:
bpf_link__destroy(link);
close_prog:
bpf_object__close(obj);
diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c
index fbfa8e7..404a549 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_map_raw_tp.c
@@ -26,15 +26,15 @@ void test_stacktrace_map_raw_tp(void)
/* find map fds */
control_map_fd = bpf_find_map(__func__, obj, "control_map");
- if (control_map_fd < 0)
+ if (CHECK_FAIL(control_map_fd < 0))
goto close_prog;
stackid_hmap_fd = bpf_find_map(__func__, obj, "stackid_hmap");
- if (stackid_hmap_fd < 0)
+ if (CHECK_FAIL(stackid_hmap_fd < 0))
goto close_prog;
stackmap_fd = bpf_find_map(__func__, obj, "stackmap");
- if (stackmap_fd < 0)
+ if (CHECK_FAIL(stackmap_fd < 0))
goto close_prog;
/* give some time for bpf program run */
@@ -58,10 +58,7 @@ void test_stacktrace_map_raw_tp(void)
"err %d errno %d\n", err, errno))
goto close_prog;
- goto close_prog_noerr;
close_prog:
- error_cnt++;
-close_prog_noerr:
if (!IS_ERR_OR_NULL(link))
bpf_link__destroy(link);
bpf_object__close(obj);
diff --git a/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c b/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c
index 958a3d8..1bdc1d8 100644
--- a/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c
+++ b/tools/testing/selftests/bpf/prog_tests/task_fd_query_rawtp.c
@@ -70,9 +70,6 @@ void test_task_fd_query_rawtp(void)
if (CHECK(!err, "check_results", "fd_type %d len %u\n", fd_type, len))
goto close_prog;
- goto close_prog_noerr;
close_prog:
- error_cnt++;
-close_prog_noerr:
bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c b/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c
index f9b70e8..3f131b8 100644
--- a/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c
+++ b/tools/testing/selftests/bpf/prog_tests/task_fd_query_tp.c
@@ -62,14 +62,9 @@ static void test_task_fd_query_tp_core(const char *probe_name,
fd_type, buf))
goto close_pmu;
- close(pmu_fd);
- goto close_prog_noerr;
-
close_pmu:
close(pmu_fd);
close_prog:
- error_cnt++;
-close_prog_noerr:
bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/tcp_estats.c b/tools/testing/selftests/bpf/prog_tests/tcp_estats.c
index bb8759d..594307d 100644
--- a/tools/testing/selftests/bpf/prog_tests/tcp_estats.c
+++ b/tools/testing/selftests/bpf/prog_tests/tcp_estats.c
@@ -10,10 +10,8 @@ void test_tcp_estats(void)
err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
CHECK(err, "", "err %d errno %d\n", err, errno);
- if (err) {
- error_cnt++;
+ if (err)
return;
- }
bpf_object__close(obj);
}
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp.c b/tools/testing/selftests/bpf/prog_tests/xdp.c
index a741672..dcb5eca 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp.c
@@ -16,10 +16,8 @@ void test_xdp(void)
int err, prog_fd, map_fd;
err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
map_fd = bpf_find_map(__func__, obj, "vip2tnl");
if (map_fd < 0)
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c b/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c
index 922aa0a..3744196 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_adjust_tail.c
@@ -10,10 +10,8 @@ void test_xdp_adjust_tail(void)
int err, prog_fd;
err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
err = bpf_prog_test_run(prog_fd, 1, &pkt_v4, sizeof(pkt_v4),
buf, &size, &retval, &duration);
diff --git a/tools/testing/selftests/bpf/prog_tests/xdp_noinline.c b/tools/testing/selftests/bpf/prog_tests/xdp_noinline.c
index 15f7c27..c9404e6 100644
--- a/tools/testing/selftests/bpf/prog_tests/xdp_noinline.c
+++ b/tools/testing/selftests/bpf/prog_tests/xdp_noinline.c
@@ -31,10 +31,8 @@ void test_xdp_noinline(void)
u32 *magic = (u32 *)buf;
err = bpf_prog_load(file, BPF_PROG_TYPE_XDP, &obj, &prog_fd);
- if (err) {
- error_cnt++;
+ if (CHECK_FAIL(err))
return;
- }
map_fd = bpf_find_map(__func__, obj, "vip_map");
if (map_fd < 0)
@@ -73,8 +71,8 @@ void test_xdp_noinline(void)
bytes += stats[i].bytes;
pkts += stats[i].pkts;
}
- if (bytes != MAGIC_BYTES * NUM_ITER * 2 || pkts != NUM_ITER * 2) {
- error_cnt++;
+ if (CHECK_FAIL(bytes != MAGIC_BYTES * NUM_ITER * 2 ||
+ pkts != NUM_ITER * 2)) {
printf("test_xdp_noinline:FAIL:stats %lld %lld\n",
bytes, pkts);
}
diff --git a/tools/testing/selftests/bpf/progs/sockopt_inherit.c b/tools/testing/selftests/bpf/progs/sockopt_inherit.c
new file mode 100644
index 0000000..dede0fc
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/sockopt_inherit.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+char _license[] SEC("license") = "GPL";
+__u32 _version SEC("version") = 1;
+
+#define SOL_CUSTOM 0xdeadbeef
+#define CUSTOM_INHERIT1 0
+#define CUSTOM_INHERIT2 1
+#define CUSTOM_LISTENER 2
+
+struct sockopt_inherit {
+ __u8 val;
+};
+
+struct {
+ __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC | BPF_F_CLONE);
+ __type(key, int);
+ __type(value, struct sockopt_inherit);
+} cloned1_map SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC | BPF_F_CLONE);
+ __type(key, int);
+ __type(value, struct sockopt_inherit);
+} cloned2_map SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __type(key, int);
+ __type(value, struct sockopt_inherit);
+} listener_only_map SEC(".maps");
+
+static __inline struct sockopt_inherit *get_storage(struct bpf_sockopt *ctx)
+{
+ if (ctx->optname == CUSTOM_INHERIT1)
+ return bpf_sk_storage_get(&cloned1_map, ctx->sk, 0,
+ BPF_SK_STORAGE_GET_F_CREATE);
+ else if (ctx->optname == CUSTOM_INHERIT2)
+ return bpf_sk_storage_get(&cloned2_map, ctx->sk, 0,
+ BPF_SK_STORAGE_GET_F_CREATE);
+ else
+ return bpf_sk_storage_get(&listener_only_map, ctx->sk, 0,
+ BPF_SK_STORAGE_GET_F_CREATE);
+}
+
+SEC("cgroup/getsockopt")
+int _getsockopt(struct bpf_sockopt *ctx)
+{
+ __u8 *optval_end = ctx->optval_end;
+ struct sockopt_inherit *storage;
+ __u8 *optval = ctx->optval;
+
+ if (ctx->level != SOL_CUSTOM)
+ return 1; /* only interested in SOL_CUSTOM */
+
+ if (optval + 1 > optval_end)
+ return 0; /* EPERM, bounds check */
+
+ storage = get_storage(ctx);
+ if (!storage)
+ return 0; /* EPERM, couldn't get sk storage */
+
+ ctx->retval = 0; /* Reset system call return value to zero */
+
+ optval[0] = storage->val;
+ ctx->optlen = 1;
+
+ return 1;
+}
+
+SEC("cgroup/setsockopt")
+int _setsockopt(struct bpf_sockopt *ctx)
+{
+ __u8 *optval_end = ctx->optval_end;
+ struct sockopt_inherit *storage;
+ __u8 *optval = ctx->optval;
+
+ if (ctx->level != SOL_CUSTOM)
+ return 1; /* only interested in SOL_CUSTOM */
+
+ if (optval + 1 > optval_end)
+ return 0; /* EPERM, bounds check */
+
+ storage = get_storage(ctx);
+ if (!storage)
+ return 0; /* EPERM, couldn't get sk storage */
+
+ storage->val = optval[0];
+ ctx->optlen = -1;
+
+ return 1;
+}
diff --git a/tools/testing/selftests/bpf/progs/test_lwt_seg6local.c b/tools/testing/selftests/bpf/progs/test_lwt_seg6local.c
index a334a0e..41a3ebc 100644
--- a/tools/testing/selftests/bpf/progs/test_lwt_seg6local.c
+++ b/tools/testing/selftests/bpf/progs/test_lwt_seg6local.c
@@ -12,10 +12,6 @@
#define SR6_FLAG_ALERT (1 << 4)
-#define htonll(x) ((bpf_htonl(1)) == 1 ? (x) : ((uint64_t)bpf_htonl((x) & \
- 0xFFFFFFFF) << 32) | bpf_htonl((x) >> 32))
-#define ntohll(x) ((bpf_ntohl(1)) == 1 ? (x) : ((uint64_t)bpf_ntohl((x) & \
- 0xFFFFFFFF) << 32) | bpf_ntohl((x) >> 32))
#define BPF_PACKET_HEADER __attribute__((packed))
struct ip6_t {
@@ -276,8 +272,8 @@ int has_egr_tlv(struct __sk_buff *skb, struct ip6_srh_t *srh)
return 0;
// check if egress TLV value is correct
- if (ntohll(egr_addr.hi) == 0xfd00000000000000 &&
- ntohll(egr_addr.lo) == 0x4)
+ if (bpf_be64_to_cpu(egr_addr.hi) == 0xfd00000000000000 &&
+ bpf_be64_to_cpu(egr_addr.lo) == 0x4)
return 1;
}
@@ -308,8 +304,8 @@ int __encap_srh(struct __sk_buff *skb)
#pragma clang loop unroll(full)
for (unsigned long long lo = 0; lo < 4; lo++) {
- seg->lo = htonll(4 - lo);
- seg->hi = htonll(hi);
+ seg->lo = bpf_cpu_to_be64(4 - lo);
+ seg->hi = bpf_cpu_to_be64(hi);
seg = (struct ip6_addr_t *)((char *)seg + sizeof(*seg));
}
@@ -349,8 +345,8 @@ int __add_egr_x(struct __sk_buff *skb)
if (err)
return BPF_DROP;
- addr.lo = htonll(lo);
- addr.hi = htonll(hi);
+ addr.lo = bpf_cpu_to_be64(lo);
+ addr.hi = bpf_cpu_to_be64(hi);
err = bpf_lwt_seg6_action(skb, SEG6_LOCAL_ACTION_END_X,
(void *)&addr, sizeof(addr));
if (err)
diff --git a/tools/testing/selftests/bpf/progs/test_seg6_loop.c b/tools/testing/selftests/bpf/progs/test_seg6_loop.c
index 1dbe1d4..c4d1044 100644
--- a/tools/testing/selftests/bpf/progs/test_seg6_loop.c
+++ b/tools/testing/selftests/bpf/progs/test_seg6_loop.c
@@ -12,10 +12,6 @@
#define SR6_FLAG_ALERT (1 << 4)
-#define htonll(x) ((bpf_htonl(1)) == 1 ? (x) : ((uint64_t)bpf_htonl((x) & \
- 0xFFFFFFFF) << 32) | bpf_htonl((x) >> 32))
-#define ntohll(x) ((bpf_ntohl(1)) == 1 ? (x) : ((uint64_t)bpf_ntohl((x) & \
- 0xFFFFFFFF) << 32) | bpf_ntohl((x) >> 32))
#define BPF_PACKET_HEADER __attribute__((packed))
struct ip6_t {
@@ -251,8 +247,8 @@ int __add_egr_x(struct __sk_buff *skb)
if (err)
return BPF_DROP;
- addr.lo = htonll(lo);
- addr.hi = htonll(hi);
+ addr.lo = bpf_cpu_to_be64(lo);
+ addr.hi = bpf_cpu_to_be64(hi);
err = bpf_lwt_seg6_action(skb, SEG6_LOCAL_ACTION_END_X,
(void *)&addr, sizeof(addr));
if (err)
diff --git a/tools/testing/selftests/bpf/test_bpftool_build.sh b/tools/testing/selftests/bpf/test_bpftool_build.sh
new file mode 100755
index 0000000..4ba5a34
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_bpftool_build.sh
@@ -0,0 +1,143 @@
+#!/bin/bash
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+ERROR=0
+TMPDIR=
+
+# If one build fails, continue but return non-0 on exit.
+return_value() {
+ if [ -d "$TMPDIR" ] ; then
+ rm -rf -- $TMPDIR
+ fi
+ exit $ERROR
+}
+trap return_value EXIT
+
+case $1 in
+ -h|--help)
+ echo -e "$0 [-j <n>]"
+ echo -e "\tTest the different ways of building bpftool."
+ echo -e ""
+ echo -e "\tOptions:"
+ echo -e "\t\t-j <n>:\tPass -j flag to 'make'."
+ exit
+ ;;
+esac
+
+J=$*
+
+# Assume script is located under tools/testing/selftests/bpf/. We want to start
+# build attempts from the top of kernel repository.
+SCRIPT_REL_PATH=$(realpath --relative-to=$PWD $0)
+SCRIPT_REL_DIR=$(dirname $SCRIPT_REL_PATH)
+KDIR_ROOT_DIR=$(realpath $PWD/$SCRIPT_REL_DIR/../../../../)
+cd $KDIR_ROOT_DIR
+
+check() {
+ local dir=$(realpath $1)
+
+ echo -n "binary: "
+ # Returns non-null if file is found (and "false" is run)
+ find $dir -type f -executable -name bpftool -print -exec false {} + && \
+ ERROR=1 && printf "FAILURE: Did not find bpftool\n"
+}
+
+make_and_clean() {
+ echo -e "\$PWD: $PWD"
+ echo -e "command: make -s $* >/dev/null"
+ make $J -s $* >/dev/null
+ if [ $? -ne 0 ] ; then
+ ERROR=1
+ fi
+ if [ $# -ge 1 ] ; then
+ check ${@: -1}
+ else
+ check .
+ fi
+ (
+ if [ $# -ge 1 ] ; then
+ cd ${@: -1}
+ fi
+ make -s clean
+ )
+ echo
+}
+
+make_with_tmpdir() {
+ local ARGS
+
+ TMPDIR=$(mktemp -d)
+ if [ $# -ge 2 ] ; then
+ ARGS=${@:1:(($# - 1))}
+ fi
+ echo -e "\$PWD: $PWD"
+ echo -e "command: make -s $ARGS ${@: -1}=$TMPDIR/ >/dev/null"
+ make $J -s $ARGS ${@: -1}=$TMPDIR/ >/dev/null
+ if [ $? -ne 0 ] ; then
+ ERROR=1
+ fi
+ check $TMPDIR
+ rm -rf -- $TMPDIR
+ echo
+}
+
+echo "Trying to build bpftool"
+echo -e "... through kbuild\n"
+
+if [ -f ".config" ] ; then
+ make_and_clean tools/bpf
+
+ ## $OUTPUT is overwritten in kbuild Makefile, and thus cannot be passed
+ ## down from toplevel Makefile to bpftool's Makefile.
+
+ # make_with_tmpdir tools/bpf OUTPUT
+ echo -e "skip: make tools/bpf OUTPUT=<dir> (not supported)\n"
+
+ make_with_tmpdir tools/bpf O
+else
+ echo -e "skip: make tools/bpf (no .config found)\n"
+ echo -e "skip: make tools/bpf OUTPUT=<dir> (not supported)\n"
+ echo -e "skip: make tools/bpf O=<dir> (no .config found)\n"
+fi
+
+echo -e "... from kernel source tree\n"
+
+make_and_clean -C tools/bpf/bpftool
+
+make_with_tmpdir -C tools/bpf/bpftool OUTPUT
+
+make_with_tmpdir -C tools/bpf/bpftool O
+
+echo -e "... from tools/\n"
+cd tools/
+
+make_and_clean bpf
+
+## In tools/bpf/Makefile, function "descend" is called and passes $(O) and
+## $(OUTPUT). We would like $(OUTPUT) to have "bpf/bpftool/" appended before
+## calling bpftool's Makefile, but this is not the case as the "descend"
+## function focuses on $(O)/$(subdir). However, in the present case, updating
+## $(O) to have $(OUTPUT) recomputed from it in bpftool's Makefile does not
+## work, because $(O) is not defined from command line and $(OUTPUT) is not
+## updated in tools/scripts/Makefile.include.
+##
+## Workarounds would require to a) edit "descend" or use an alternative way to
+## call bpftool's Makefile, b) modify the conditions to update $(OUTPUT) and
+## other variables in tools/scripts/Makefile.include (at the risk of breaking
+## the build of other tools), or c) append manually the "bpf/bpftool" suffix to
+## $(OUTPUT) in bpf's Makefile, which may break if targets for other directories
+## use "descend" in the future.
+
+# make_with_tmpdir bpf OUTPUT
+echo -e "skip: make bpf OUTPUT=<dir> (not supported)\n"
+
+make_with_tmpdir bpf O
+
+echo -e "... from bpftool's dir\n"
+cd bpf/bpftool
+
+make_and_clean
+
+make_with_tmpdir OUTPUT
+
+make_with_tmpdir O
diff --git a/tools/testing/selftests/bpf/test_btf_dump.c b/tools/testing/selftests/bpf/test_btf_dump.c
index 8f85082..6e75dd3 100644
--- a/tools/testing/selftests/bpf/test_btf_dump.c
+++ b/tools/testing/selftests/bpf/test_btf_dump.c
@@ -97,6 +97,13 @@ int test_btf_dump_case(int n, struct btf_dump_test_case *test_case)
}
snprintf(test_file, sizeof(test_file), "progs/%s.c", test_case->name);
+ if (access(test_file, R_OK) == -1)
+ /*
+ * When the test is run with O=, kselftest copies TEST_FILES
+ * without preserving the directory structure.
+ */
+ snprintf(test_file, sizeof(test_file), "%s.c",
+ test_case->name);
/*
* Diff test output and expected test output, contained between
* START-EXPECTED-OUTPUT and END-EXPECTED-OUTPUT lines in test case.
diff --git a/tools/testing/selftests/bpf/test_cgroup_storage.c b/tools/testing/selftests/bpf/test_cgroup_storage.c
index 2fc4625..6557290 100644
--- a/tools/testing/selftests/bpf/test_cgroup_storage.c
+++ b/tools/testing/selftests/bpf/test_cgroup_storage.c
@@ -20,9 +20,9 @@ int main(int argc, char **argv)
BPF_MOV64_IMM(BPF_REG_2, 0), /* flags, not used */
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0,
BPF_FUNC_get_local_storage),
- BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_0, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_0, 0),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_3, 0x1),
- BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_3, 0),
+ BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_3, 0),
BPF_LD_MAP_FD(BPF_REG_1, 0), /* map fd */
BPF_MOV64_IMM(BPF_REG_2, 0), /* flags, not used */
@@ -30,7 +30,7 @@ int main(int argc, char **argv)
BPF_FUNC_get_local_storage),
BPF_MOV64_IMM(BPF_REG_1, 1),
BPF_STX_XADD(BPF_DW, BPF_REG_0, BPF_REG_1, 0),
- BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0),
BPF_ALU64_IMM(BPF_AND, BPF_REG_1, 0x1),
BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
BPF_EXIT_INSN(),
diff --git a/tools/testing/selftests/bpf/test_offload.py b/tools/testing/selftests/bpf/test_offload.py
index 425f9ed..15a6663 100755
--- a/tools/testing/selftests/bpf/test_offload.py
+++ b/tools/testing/selftests/bpf/test_offload.py
@@ -1353,7 +1353,7 @@
bpftool_prog_list_wait(expected=1)
ifnameB = bpftool("prog show %s" % (progB))[1]["dev"]["ifname"]
- fail(ifnameB != simB1['ifname'], "program not bound to originial device")
+ fail(ifnameB != simB1['ifname'], "program not bound to original device")
simB1.remove()
bpftool_prog_list_wait(expected=1)
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index 12895d0..e8616e7 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -8,22 +8,20 @@
/* defined in test_progs.h */
struct test_env env;
-int error_cnt, pass_cnt;
struct prog_test_def {
const char *test_name;
int test_num;
void (*run_test)(void);
bool force_log;
- int pass_cnt;
int error_cnt;
+ int skip_cnt;
bool tested;
const char *subtest_name;
int subtest_num;
/* store counts before subtest started */
- int old_pass_cnt;
int old_error_cnt;
};
@@ -47,6 +45,7 @@ static void dump_test_log(const struct prog_test_def *test, bool failed)
if (env.verbose || test->force_log || failed) {
if (env.log_cnt) {
+ env.log_buf[env.log_cnt] = '\0';
fprintf(env.stdout, "%s", env.log_buf);
if (env.log_buf[env.log_cnt - 1] != '\n')
fprintf(env.stdout, "\n");
@@ -56,15 +55,24 @@ static void dump_test_log(const struct prog_test_def *test, bool failed)
fseeko(stdout, 0, SEEK_SET); /* rewind */
}
+static void skip_account(void)
+{
+ if (env.test->skip_cnt) {
+ env.skip_cnt++;
+ env.test->skip_cnt = 0;
+ }
+}
+
void test__end_subtest()
{
struct prog_test_def *test = env.test;
- int sub_error_cnt = error_cnt - test->old_error_cnt;
+ int sub_error_cnt = test->error_cnt - test->old_error_cnt;
if (sub_error_cnt)
env.fail_cnt++;
else
env.sub_succ_cnt++;
+ skip_account();
dump_test_log(test, sub_error_cnt);
@@ -95,8 +103,7 @@ bool test__start_subtest(const char *name)
return false;
test->subtest_name = name;
- env.test->old_pass_cnt = pass_cnt;
- env.test->old_error_cnt = error_cnt;
+ env.test->old_error_cnt = env.test->error_cnt;
return true;
}
@@ -105,6 +112,16 @@ void test__force_log() {
env.test->force_log = true;
}
+void test__skip(void)
+{
+ env.test->skip_cnt++;
+}
+
+void test__fail(void)
+{
+ env.test->error_cnt++;
+}
+
struct ipv4_packet pkt_v4 = {
.eth.h_proto = __bpf_constant_htons(ETH_P_IP),
.iph.ihl = 5,
@@ -129,7 +146,7 @@ int bpf_find_map(const char *test, struct bpf_object *obj, const char *name)
map = bpf_object__find_map_by_name(obj, name);
if (!map) {
printf("%s:FAIL:map '%s' not found\n", test, name);
- error_cnt++;
+ test__fail();
return -1;
}
return bpf_map__fd(map);
@@ -488,8 +505,6 @@ int main(int argc, char **argv)
stdio_hijack();
for (i = 0; i < prog_test_cnt; i++) {
struct prog_test_def *test = &prog_test_defs[i];
- int old_pass_cnt = pass_cnt;
- int old_error_cnt = error_cnt;
env.test = test;
test->test_num = i + 1;
@@ -504,12 +519,11 @@ int main(int argc, char **argv)
test__end_subtest();
test->tested = true;
- test->pass_cnt = pass_cnt - old_pass_cnt;
- test->error_cnt = error_cnt - old_error_cnt;
if (test->error_cnt)
env.fail_cnt++;
else
env.succ_cnt++;
+ skip_account();
dump_test_log(test, test->error_cnt);
@@ -518,11 +532,11 @@ int main(int argc, char **argv)
test->error_cnt ? "FAIL" : "OK");
}
stdio_restore();
- printf("Summary: %d/%d PASSED, %d FAILED\n",
- env.succ_cnt, env.sub_succ_cnt, env.fail_cnt);
+ printf("Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n",
+ env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt);
free(env.test_selector.num_set);
free(env.subtest_selector.num_set);
- return error_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
+ return env.fail_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
}
diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h
index 37d427f..c8edb94 100644
--- a/tools/testing/selftests/bpf/test_progs.h
+++ b/tools/testing/selftests/bpf/test_progs.h
@@ -38,8 +38,6 @@ typedef __u16 __sum16;
#include "trace_helpers.h"
#include "flow_dissector_load.h"
-struct prog_test_def;
-
struct test_selector {
const char *name;
bool *num_set;
@@ -64,14 +62,15 @@ struct test_env {
int succ_cnt; /* successful tests */
int sub_succ_cnt; /* successful sub-tests */
int fail_cnt; /* total failed tests + sub-tests */
+ int skip_cnt; /* skipped tests */
};
-extern int error_cnt;
-extern int pass_cnt;
extern struct test_env env;
extern void test__force_log();
extern bool test__start_subtest(const char *name);
+extern void test__skip(void);
+extern void test__fail(void);
#define MAGIC_BYTES 123
@@ -94,17 +93,25 @@ extern struct ipv6_packet pkt_v6;
#define _CHECK(condition, tag, duration, format...) ({ \
int __ret = !!(condition); \
if (__ret) { \
- error_cnt++; \
+ test__fail(); \
printf("%s:FAIL:%s ", __func__, tag); \
printf(format); \
} else { \
- pass_cnt++; \
printf("%s:PASS:%s %d nsec\n", \
__func__, tag, duration); \
} \
__ret; \
})
+#define CHECK_FAIL(condition) ({ \
+ int __ret = !!(condition); \
+ if (__ret) { \
+ test__fail(); \
+ printf("%s:FAIL:%d\n", __func__, __LINE__); \
+ } \
+ __ret; \
+})
+
#define CHECK(condition, tag, format...) \
_CHECK(condition, tag, duration, format)
#define CHECK_ATTR(condition, tag, format...) \
diff --git a/tools/testing/selftests/bpf/test_sock.c b/tools/testing/selftests/bpf/test_sock.c
index fb679ac..0e66527 100644
--- a/tools/testing/selftests/bpf/test_sock.c
+++ b/tools/testing/selftests/bpf/test_sock.c
@@ -13,6 +13,7 @@
#include <bpf/bpf.h>
#include "cgroup_helpers.h"
+#include "bpf_endian.h"
#include "bpf_rlimit.h"
#include "bpf_util.h"
@@ -232,7 +233,8 @@ static struct sock_test tests[] = {
/* if (ip == expected && port == expected) */
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
offsetof(struct bpf_sock, src_ip6[3])),
- BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x01000000, 4),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7,
+ __bpf_constant_ntohl(0x00000001), 4),
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
offsetof(struct bpf_sock, src_port)),
BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x2001, 2),
@@ -261,7 +263,8 @@ static struct sock_test tests[] = {
/* if (ip == expected && port == expected) */
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
offsetof(struct bpf_sock, src_ip4)),
- BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x0100007F, 4),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_7,
+ __bpf_constant_ntohl(0x7F000001), 4),
BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_6,
offsetof(struct bpf_sock, src_port)),
BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 0x1002, 2),
diff --git a/tools/testing/selftests/bpf/test_sockopt_inherit.c b/tools/testing/selftests/bpf/test_sockopt_inherit.c
new file mode 100644
index 0000000..1bf6998
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_sockopt_inherit.c
@@ -0,0 +1,253 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <error.h>
+#include <errno.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <pthread.h>
+
+#include <linux/filter.h>
+#include <bpf/bpf.h>
+#include <bpf/libbpf.h>
+
+#include "bpf_rlimit.h"
+#include "bpf_util.h"
+#include "cgroup_helpers.h"
+
+#define CG_PATH "/sockopt_inherit"
+#define SOL_CUSTOM 0xdeadbeef
+#define CUSTOM_INHERIT1 0
+#define CUSTOM_INHERIT2 1
+#define CUSTOM_LISTENER 2
+
+static int connect_to_server(int server_fd)
+{
+ struct sockaddr_storage addr;
+ socklen_t len = sizeof(addr);
+ int fd;
+
+ fd = socket(AF_INET, SOCK_STREAM, 0);
+ if (fd < 0) {
+ log_err("Failed to create client socket");
+ return -1;
+ }
+
+ if (getsockname(server_fd, (struct sockaddr *)&addr, &len)) {
+ log_err("Failed to get server addr");
+ goto out;
+ }
+
+ if (connect(fd, (const struct sockaddr *)&addr, len) < 0) {
+ log_err("Fail to connect to server");
+ goto out;
+ }
+
+ return fd;
+
+out:
+ close(fd);
+ return -1;
+}
+
+static int verify_sockopt(int fd, int optname, const char *msg, char expected)
+{
+ socklen_t optlen = 1;
+ char buf = 0;
+ int err;
+
+ err = getsockopt(fd, SOL_CUSTOM, optname, &buf, &optlen);
+ if (err) {
+ log_err("%s: failed to call getsockopt", msg);
+ return 1;
+ }
+
+ printf("%s %d: got=0x%x ? expected=0x%x\n", msg, optname, buf, expected);
+
+ if (buf != expected) {
+ log_err("%s: unexpected getsockopt value %d != %d", msg,
+ buf, expected);
+ return 1;
+ }
+
+ return 0;
+}
+
+static void *server_thread(void *arg)
+{
+ struct sockaddr_storage addr;
+ socklen_t len = sizeof(addr);
+ int fd = *(int *)arg;
+ int client_fd;
+ int err = 0;
+
+ if (listen(fd, 1) < 0)
+ error(1, errno, "Failed to listed on socket");
+
+ err += verify_sockopt(fd, CUSTOM_INHERIT1, "listen", 1);
+ err += verify_sockopt(fd, CUSTOM_INHERIT2, "listen", 1);
+ err += verify_sockopt(fd, CUSTOM_LISTENER, "listen", 1);
+
+ client_fd = accept(fd, (struct sockaddr *)&addr, &len);
+ if (client_fd < 0)
+ error(1, errno, "Failed to accept client");
+
+ err += verify_sockopt(client_fd, CUSTOM_INHERIT1, "accept", 1);
+ err += verify_sockopt(client_fd, CUSTOM_INHERIT2, "accept", 1);
+ err += verify_sockopt(client_fd, CUSTOM_LISTENER, "accept", 0);
+
+ close(client_fd);
+
+ return (void *)(long)err;
+}
+
+static int start_server(void)
+{
+ struct sockaddr_in addr = {
+ .sin_family = AF_INET,
+ .sin_addr.s_addr = htonl(INADDR_LOOPBACK),
+ };
+ char buf;
+ int err;
+ int fd;
+ int i;
+
+ fd = socket(AF_INET, SOCK_STREAM, 0);
+ if (fd < 0) {
+ log_err("Failed to create server socket");
+ return -1;
+ }
+
+ for (i = CUSTOM_INHERIT1; i <= CUSTOM_LISTENER; i++) {
+ buf = 0x01;
+ err = setsockopt(fd, SOL_CUSTOM, i, &buf, 1);
+ if (err) {
+ log_err("Failed to call setsockopt(%d)", i);
+ close(fd);
+ return -1;
+ }
+ }
+
+ if (bind(fd, (const struct sockaddr *)&addr, sizeof(addr)) < 0) {
+ log_err("Failed to bind socket");
+ close(fd);
+ return -1;
+ }
+
+ return fd;
+}
+
+static int prog_attach(struct bpf_object *obj, int cgroup_fd, const char *title)
+{
+ enum bpf_attach_type attach_type;
+ enum bpf_prog_type prog_type;
+ struct bpf_program *prog;
+ int err;
+
+ err = libbpf_prog_type_by_name(title, &prog_type, &attach_type);
+ if (err) {
+ log_err("Failed to deduct types for %s BPF program", title);
+ return -1;
+ }
+
+ prog = bpf_object__find_program_by_title(obj, title);
+ if (!prog) {
+ log_err("Failed to find %s BPF program", title);
+ return -1;
+ }
+
+ err = bpf_prog_attach(bpf_program__fd(prog), cgroup_fd,
+ attach_type, 0);
+ if (err) {
+ log_err("Failed to attach %s BPF program", title);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int run_test(int cgroup_fd)
+{
+ struct bpf_prog_load_attr attr = {
+ .file = "./sockopt_inherit.o",
+ };
+ int server_fd = -1, client_fd;
+ struct bpf_object *obj;
+ void *server_err;
+ pthread_t tid;
+ int ignored;
+ int err;
+
+ err = bpf_prog_load_xattr(&attr, &obj, &ignored);
+ if (err) {
+ log_err("Failed to load BPF object");
+ return -1;
+ }
+
+ err = prog_attach(obj, cgroup_fd, "cgroup/getsockopt");
+ if (err)
+ goto close_bpf_object;
+
+ err = prog_attach(obj, cgroup_fd, "cgroup/setsockopt");
+ if (err)
+ goto close_bpf_object;
+
+ server_fd = start_server();
+ if (server_fd < 0) {
+ err = -1;
+ goto close_bpf_object;
+ }
+
+ pthread_create(&tid, NULL, server_thread, (void *)&server_fd);
+
+ client_fd = connect_to_server(server_fd);
+ if (client_fd < 0) {
+ err = -1;
+ goto close_server_fd;
+ }
+
+ err += verify_sockopt(client_fd, CUSTOM_INHERIT1, "connect", 0);
+ err += verify_sockopt(client_fd, CUSTOM_INHERIT2, "connect", 0);
+ err += verify_sockopt(client_fd, CUSTOM_LISTENER, "connect", 0);
+
+ pthread_join(tid, &server_err);
+
+ err += (int)(long)server_err;
+
+ close(client_fd);
+
+close_server_fd:
+ close(server_fd);
+close_bpf_object:
+ bpf_object__close(obj);
+ return err;
+}
+
+int main(int args, char **argv)
+{
+ int cgroup_fd;
+ int err = EXIT_SUCCESS;
+
+ if (setup_cgroup_environment())
+ return err;
+
+ cgroup_fd = create_and_get_cgroup(CG_PATH);
+ if (cgroup_fd < 0)
+ goto cleanup_cgroup_env;
+
+ if (join_cgroup(CG_PATH))
+ goto cleanup_cgroup;
+
+ if (run_test(cgroup_fd))
+ err = EXIT_FAILURE;
+
+ printf("test_sockopt_inherit: %s\n",
+ err == EXIT_SUCCESS ? "PASSED" : "FAILED");
+
+cleanup_cgroup:
+ close(cgroup_fd);
+cleanup_cgroup_env:
+ cleanup_cgroup_environment();
+ return err;
+}
diff --git a/tools/testing/selftests/bpf/test_sysctl.c b/tools/testing/selftests/bpf/test_sysctl.c
index a3bebd7..fc33ae3 100644
--- a/tools/testing/selftests/bpf/test_sysctl.c
+++ b/tools/testing/selftests/bpf/test_sysctl.c
@@ -13,6 +13,7 @@
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
+#include "bpf_endian.h"
#include "bpf_rlimit.h"
#include "bpf_util.h"
#include "cgroup_helpers.h"
@@ -100,7 +101,7 @@ static struct sysctl_test tests[] = {
.descr = "ctx:write sysctl:write read ok",
.insns = {
/* If (write) */
- BPF_LDX_MEM(BPF_B, BPF_REG_7, BPF_REG_1,
+ BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_1,
offsetof(struct bpf_sysctl, write)),
BPF_JMP_IMM(BPF_JNE, BPF_REG_7, 1, 2),
@@ -214,7 +215,8 @@ static struct sysctl_test tests[] = {
/* if (ret == expected && */
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, sizeof("tcp_mem") - 1, 6),
/* buf == "tcp_mem\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x006d656d5f706374ULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x7463705f6d656d00ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -255,7 +257,8 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, -E2BIG, 6),
/* buf[0:7] == "tcp_me\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x00656d5f706374ULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x7463705f6d650000ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -298,12 +301,14 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 16, 14),
/* buf[0:8] == "net/ipv4" && */
- BPF_LD_IMM64(BPF_REG_8, 0x347670692f74656eULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x6e65742f69707634ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 10),
/* buf[8:16] == "/tcp_mem" && */
- BPF_LD_IMM64(BPF_REG_8, 0x6d656d5f7063742fULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x2f7463705f6d656dULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 8),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 6),
@@ -350,12 +355,14 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, -E2BIG, 10),
/* buf[0:8] == "net/ipv4" && */
- BPF_LD_IMM64(BPF_REG_8, 0x347670692f74656eULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x6e65742f69707634ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 6),
/* buf[8:16] == "/tcp_me\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x00656d5f7063742fULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x2f7463705f6d6500ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 8),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -396,7 +403,8 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, -E2BIG, 6),
/* buf[0:8] == "net/ip\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x000070692f74656eULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x6e65742f69700000ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -431,7 +439,8 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 6, 6),
/* buf[0:6] == "Linux\n\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x000a78756e694cULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x4c696e75780a0000ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -469,7 +478,8 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 6, 6),
/* buf[0:6] == "Linux\n\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x000a78756e694cULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x4c696e75780a0000ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -507,7 +517,8 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, -E2BIG, 6),
/* buf[0:6] == "Linux\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x000078756e694cULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x4c696e7578000000ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -650,7 +661,8 @@ static struct sysctl_test tests[] = {
/* buf[0:4] == "606\0") */
BPF_LDX_MEM(BPF_W, BPF_REG_9, BPF_REG_7, 0),
- BPF_JMP_IMM(BPF_JNE, BPF_REG_9, 0x00363036, 2),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_9,
+ bpf_ntohl(0x36303600), 2),
/* return DENY; */
BPF_MOV64_IMM(BPF_REG_0, 0),
@@ -685,17 +697,20 @@ static struct sysctl_test tests[] = {
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 23, 14),
/* buf[0:8] == "3000000 " && */
- BPF_LD_IMM64(BPF_REG_8, 0x2030303030303033ULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x3330303030303020ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 0),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 10),
/* buf[8:16] == "4000000 " && */
- BPF_LD_IMM64(BPF_REG_8, 0x2030303030303034ULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x3430303030303020ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 8),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 6),
/* buf[16:24] == "6000000\0") */
- BPF_LD_IMM64(BPF_REG_8, 0x0030303030303036ULL),
+ BPF_LD_IMM64(BPF_REG_8,
+ bpf_be64_to_cpu(0x3630303030303000ULL)),
BPF_LDX_MEM(BPF_DW, BPF_REG_9, BPF_REG_7, 16),
BPF_JMP_REG(BPF_JNE, BPF_REG_8, BPF_REG_9, 2),
@@ -735,7 +750,8 @@ static struct sysctl_test tests[] = {
/* buf[0:3] == "60\0") */
BPF_LDX_MEM(BPF_W, BPF_REG_9, BPF_REG_7, 0),
- BPF_JMP_IMM(BPF_JNE, BPF_REG_9, 0x003036, 2),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_9,
+ bpf_ntohl(0x36300000), 2),
/* return DENY; */
BPF_MOV64_IMM(BPF_REG_0, 0),
@@ -757,7 +773,8 @@ static struct sysctl_test tests[] = {
/* sysctl_set_new_value arg2 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00303036),
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x36303000)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_2, BPF_REG_7),
@@ -791,7 +808,7 @@ static struct sysctl_test tests[] = {
/* sysctl_set_new_value arg2 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, FIXUP_SYSCTL_VALUE),
+ BPF_LD_IMM64(BPF_REG_0, FIXUP_SYSCTL_VALUE),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_2, BPF_REG_7),
@@ -825,8 +842,9 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00303036),
- BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x36303000)),
+ BPF_STX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -869,7 +887,8 @@ static struct sysctl_test tests[] = {
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
/* "600 602\0" */
- BPF_LD_IMM64(BPF_REG_0, 0x0032303620303036ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3630302036303200ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -937,7 +956,8 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00303036),
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x36303000)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -969,8 +989,9 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00373730),
- BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x30373700)),
+ BPF_STX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1012,7 +1033,8 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00303036),
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x36303000)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1052,7 +1074,8 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x090a0c0d),
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x0d0c0a09)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1092,7 +1115,9 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00362d0a), /* " -6\0" */
+ /* " -6\0" */
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x0a2d3600)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1132,8 +1157,10 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x00362d0a), /* " -6\0" */
- BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
+ /* " -6\0" */
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x0a2d3600)),
+ BPF_STX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1175,8 +1202,10 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -8),
- BPF_MOV64_IMM(BPF_REG_0, 0x65667830), /* "0xfe" */
- BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
+ /* "0xfe" */
+ BPF_MOV64_IMM(BPF_REG_0,
+ bpf_ntohl(0x30786665)),
+ BPF_STX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, 0),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1218,11 +1247,14 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) 9223372036854775807 */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -24),
- BPF_LD_IMM64(BPF_REG_0, 0x3032373333323239ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3932323333373230ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
- BPF_LD_IMM64(BPF_REG_0, 0x3537373435383633ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3336383534373735ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 8),
- BPF_LD_IMM64(BPF_REG_0, 0x0000000000373038ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3830370000000000ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 16),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1266,11 +1298,14 @@ static struct sysctl_test tests[] = {
/* arg1 (buf) 9223372036854775808 */
BPF_MOV64_REG(BPF_REG_7, BPF_REG_10),
BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, -24),
- BPF_LD_IMM64(BPF_REG_0, 0x3032373333323239ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3932323333373230ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 0),
- BPF_LD_IMM64(BPF_REG_0, 0x3537373435383633ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3336383534373735ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 8),
- BPF_LD_IMM64(BPF_REG_0, 0x0000000000383038ULL),
+ BPF_LD_IMM64(BPF_REG_0,
+ bpf_be64_to_cpu(0x3830380000000000ULL)),
BPF_STX_MEM(BPF_DW, BPF_REG_7, BPF_REG_0, 16),
BPF_MOV64_REG(BPF_REG_1, BPF_REG_7),
@@ -1344,20 +1379,24 @@ static size_t probe_prog_length(const struct bpf_insn *fp)
static int fixup_sysctl_value(const char *buf, size_t buf_len,
struct bpf_insn *prog, size_t insn_num)
{
- uint32_t value_num = 0;
+ union {
+ uint8_t raw[sizeof(uint64_t)];
+ uint64_t num;
+ } value = {};
uint8_t c, i;
- if (buf_len > sizeof(value_num)) {
+ if (buf_len > sizeof(value)) {
log_err("Value is too big (%zd) to use in fixup", buf_len);
return -1;
}
-
- for (i = 0; i < buf_len; ++i) {
- c = buf[i];
- value_num |= (c << i * 8);
+ if (prog[insn_num].code != (BPF_LD | BPF_DW | BPF_IMM)) {
+ log_err("Can fixup only BPF_LD_IMM64 insns");
+ return -1;
}
- prog[insn_num].imm = value_num;
+ memcpy(value.raw, buf, buf_len);
+ prog[insn_num].imm = (uint32_t)value.num;
+ prog[insn_num + 1].imm = (uint32_t)(value.num >> 32);
return 0;
}
@@ -1499,6 +1538,7 @@ static int run_test_case(int cgfd, struct sysctl_test *test)
goto err;
}
+ errno = 0;
if (access_sysctl(sysctl_path, test) == -1) {
if (test->result == OP_EPERM && errno == EPERM)
goto out;
@@ -1507,7 +1547,7 @@ static int run_test_case(int cgfd, struct sysctl_test *test)
}
if (test->result != SUCCESS) {
- log_err("Unexpected failure");
+ log_err("Unexpected success");
goto err;
}
diff --git a/tools/testing/selftests/bpf/test_tcp_rtt.c b/tools/testing/selftests/bpf/test_tcp_rtt.c
index 90c3862..93916a6 100644
--- a/tools/testing/selftests/bpf/test_tcp_rtt.c
+++ b/tools/testing/selftests/bpf/test_tcp_rtt.c
@@ -6,6 +6,7 @@
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
+#include <netinet/tcp.h>
#include <pthread.h>
#include <linux/filter.h>
@@ -34,6 +35,30 @@ static void send_byte(int fd)
error(1, errno, "Failed to send single byte");
}
+static int wait_for_ack(int fd, int retries)
+{
+ struct tcp_info info;
+ socklen_t optlen;
+ int i, err;
+
+ for (i = 0; i < retries; i++) {
+ optlen = sizeof(info);
+ err = getsockopt(fd, SOL_TCP, TCP_INFO, &info, &optlen);
+ if (err < 0) {
+ log_err("Failed to lookup TCP stats");
+ return err;
+ }
+
+ if (info.tcpi_unacked == 0)
+ return 0;
+
+ usleep(10);
+ }
+
+ log_err("Did not receive ACK");
+ return -1;
+}
+
static int verify_sk(int map_fd, int client_fd, const char *msg, __u32 invoked,
__u32 dsack_dups, __u32 delivered, __u32 delivered_ce,
__u32 icsk_retransmits)
@@ -149,6 +174,11 @@ static int run_test(int cgroup_fd, int server_fd)
/*icsk_retransmits=*/0);
send_byte(client_fd);
+ if (wait_for_ack(client_fd, 100) < 0) {
+ err = -1;
+ goto close_client_fd;
+ }
+
err += verify_sk(map_fd, client_fd, "first payload byte",
/*invoked=*/2,
@@ -157,6 +187,7 @@ static int run_test(int cgroup_fd, int server_fd)
/*delivered_ce=*/0,
/*icsk_retransmits=*/0);
+close_client_fd:
close(client_fd);
close_bpf_object:
diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c
index 44e2d64..d27fd92 100644
--- a/tools/testing/selftests/bpf/test_verifier.c
+++ b/tools/testing/selftests/bpf/test_verifier.c
@@ -61,6 +61,7 @@
#define UNPRIV_SYSCTL "kernel/unprivileged_bpf_disabled"
static bool unpriv_disabled = false;
static int skips;
+static bool verbose = false;
struct bpf_test {
const char *descr;
@@ -92,7 +93,8 @@ struct bpf_test {
enum {
UNDEF,
ACCEPT,
- REJECT
+ REJECT,
+ VERBOSE_ACCEPT,
} result, result_unpriv;
enum bpf_prog_type prog_type;
uint8_t flags;
@@ -859,6 +861,36 @@ static int do_prog_test_run(int fd_prog, bool unpriv, uint32_t expected_val,
return 0;
}
+static bool cmp_str_seq(const char *log, const char *exp)
+{
+ char needle[80];
+ const char *p, *q;
+ int len;
+
+ do {
+ p = strchr(exp, '\t');
+ if (!p)
+ p = exp + strlen(exp);
+
+ len = p - exp;
+ if (len >= sizeof(needle) || !len) {
+ printf("FAIL\nTestcase bug\n");
+ return false;
+ }
+ strncpy(needle, exp, len);
+ needle[len] = 0;
+ q = strstr(log, needle);
+ if (!q) {
+ printf("FAIL\nUnexpected verifier log in successful load!\n"
+ "EXP: %s\nRES:\n", needle);
+ return false;
+ }
+ log = q + len;
+ exp = p + 1;
+ } while (*p);
+ return true;
+}
+
static void do_test_single(struct bpf_test *test, bool unpriv,
int *passes, int *errors)
{
@@ -897,14 +929,20 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
pflags |= BPF_F_STRICT_ALIGNMENT;
if (test->flags & F_NEEDS_EFFICIENT_UNALIGNED_ACCESS)
pflags |= BPF_F_ANY_ALIGNMENT;
+ if (test->flags & ~3)
+ pflags |= test->flags;
+ expected_ret = unpriv && test->result_unpriv != UNDEF ?
+ test->result_unpriv : test->result;
+ expected_err = unpriv && test->errstr_unpriv ?
+ test->errstr_unpriv : test->errstr;
memset(&attr, 0, sizeof(attr));
attr.prog_type = prog_type;
attr.expected_attach_type = test->expected_attach_type;
attr.insns = prog;
attr.insns_cnt = prog_len;
attr.license = "GPL";
- attr.log_level = 4;
+ attr.log_level = verbose || expected_ret == VERBOSE_ACCEPT ? 1 : 4;
attr.prog_flags = pflags;
fd_prog = bpf_load_program_xattr(&attr, bpf_vlog, sizeof(bpf_vlog));
@@ -914,14 +952,9 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
goto close_fds;
}
- expected_ret = unpriv && test->result_unpriv != UNDEF ?
- test->result_unpriv : test->result;
- expected_err = unpriv && test->errstr_unpriv ?
- test->errstr_unpriv : test->errstr;
-
alignment_prevented_execution = 0;
- if (expected_ret == ACCEPT) {
+ if (expected_ret == ACCEPT || expected_ret == VERBOSE_ACCEPT) {
if (fd_prog < 0) {
printf("FAIL\nFailed to load prog '%s'!\n",
strerror(errno));
@@ -932,6 +965,9 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
(test->flags & F_NEEDS_EFFICIENT_UNALIGNED_ACCESS))
alignment_prevented_execution = 1;
#endif
+ if (expected_ret == VERBOSE_ACCEPT && !cmp_str_seq(bpf_vlog, expected_err)) {
+ goto fail_log;
+ }
} else {
if (fd_prog >= 0) {
printf("FAIL\nUnexpected success to load!\n");
@@ -957,6 +993,9 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
}
}
+ if (verbose)
+ printf(", verifier log:\n%s", bpf_vlog);
+
run_errs = 0;
run_successes = 0;
if (!alignment_prevented_execution && fd_prog >= 0) {
@@ -1097,17 +1136,24 @@ int main(int argc, char **argv)
{
unsigned int from = 0, to = ARRAY_SIZE(tests);
bool unpriv = !is_admin();
+ int arg = 1;
+
+ if (argc > 1 && strcmp(argv[1], "-v") == 0) {
+ arg++;
+ verbose = true;
+ argc--;
+ }
if (argc == 3) {
- unsigned int l = atoi(argv[argc - 2]);
- unsigned int u = atoi(argv[argc - 1]);
+ unsigned int l = atoi(argv[arg]);
+ unsigned int u = atoi(argv[arg + 1]);
if (l < to && u < to) {
from = l;
to = u + 1;
}
} else if (argc == 2) {
- unsigned int t = atoi(argv[argc - 1]);
+ unsigned int t = atoi(argv[arg]);
if (t < to) {
from = t;
diff --git a/tools/testing/selftests/bpf/verifier/precise.c b/tools/testing/selftests/bpf/verifier/precise.c
new file mode 100644
index 0000000..02151f8
--- /dev/null
+++ b/tools/testing/selftests/bpf/verifier/precise.c
@@ -0,0 +1,194 @@
+{
+ "precise: test 1",
+ .insns = {
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_LD_MAP_FD(BPF_REG_6, 0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_ST_MEM(BPF_DW, BPF_REG_FP, -8, 0),
+ BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+
+ BPF_MOV64_REG(BPF_REG_9, BPF_REG_0),
+
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+
+ BPF_MOV64_REG(BPF_REG_8, BPF_REG_0),
+
+ BPF_ALU64_REG(BPF_SUB, BPF_REG_9, BPF_REG_8), /* map_value_ptr -= map_value_ptr */
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_9),
+ BPF_JMP_IMM(BPF_JLT, BPF_REG_2, 8, 1),
+ BPF_EXIT_INSN(),
+
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1), /* R2=inv(umin=1, umax=8) */
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_FP),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8),
+ BPF_MOV64_IMM(BPF_REG_3, 0),
+ BPF_EMIT_CALL(BPF_FUNC_probe_read),
+ BPF_EXIT_INSN(),
+ },
+ .prog_type = BPF_PROG_TYPE_TRACEPOINT,
+ .fixup_map_array_48b = { 1 },
+ .result = VERBOSE_ACCEPT,
+ .errstr =
+ "26: (85) call bpf_probe_read#4\
+ last_idx 26 first_idx 20\
+ regs=4 stack=0 before 25\
+ regs=4 stack=0 before 24\
+ regs=4 stack=0 before 23\
+ regs=4 stack=0 before 22\
+ regs=4 stack=0 before 20\
+ parent didn't have regs=4 stack=0 marks\
+ last_idx 19 first_idx 10\
+ regs=4 stack=0 before 19\
+ regs=200 stack=0 before 18\
+ regs=300 stack=0 before 17\
+ regs=201 stack=0 before 15\
+ regs=201 stack=0 before 14\
+ regs=200 stack=0 before 13\
+ regs=200 stack=0 before 12\
+ regs=200 stack=0 before 11\
+ regs=200 stack=0 before 10\
+ parent already had regs=0 stack=0 marks",
+},
+{
+ "precise: test 2",
+ .insns = {
+ BPF_MOV64_IMM(BPF_REG_0, 1),
+ BPF_LD_MAP_FD(BPF_REG_6, 0),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_ST_MEM(BPF_DW, BPF_REG_FP, -8, 0),
+ BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+
+ BPF_MOV64_REG(BPF_REG_9, BPF_REG_0),
+
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_6),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_FP),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+
+ BPF_MOV64_REG(BPF_REG_8, BPF_REG_0),
+
+ BPF_ALU64_REG(BPF_SUB, BPF_REG_9, BPF_REG_8), /* map_value_ptr -= map_value_ptr */
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_9),
+ BPF_JMP_IMM(BPF_JLT, BPF_REG_2, 8, 1),
+ BPF_EXIT_INSN(),
+
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1), /* R2=inv(umin=1, umax=8) */
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_FP),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8),
+ BPF_MOV64_IMM(BPF_REG_3, 0),
+ BPF_EMIT_CALL(BPF_FUNC_probe_read),
+ BPF_EXIT_INSN(),
+ },
+ .prog_type = BPF_PROG_TYPE_TRACEPOINT,
+ .fixup_map_array_48b = { 1 },
+ .result = VERBOSE_ACCEPT,
+ .flags = BPF_F_TEST_STATE_FREQ,
+ .errstr =
+ "26: (85) call bpf_probe_read#4\
+ last_idx 26 first_idx 22\
+ regs=4 stack=0 before 25\
+ regs=4 stack=0 before 24\
+ regs=4 stack=0 before 23\
+ regs=4 stack=0 before 22\
+ parent didn't have regs=4 stack=0 marks\
+ last_idx 20 first_idx 20\
+ regs=4 stack=0 before 20\
+ parent didn't have regs=4 stack=0 marks\
+ last_idx 19 first_idx 17\
+ regs=4 stack=0 before 19\
+ regs=200 stack=0 before 18\
+ regs=300 stack=0 before 17\
+ parent already had regs=0 stack=0 marks",
+},
+{
+ "precise: cross frame pruning",
+ .insns = {
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_get_prandom_u32),
+ BPF_MOV64_IMM(BPF_REG_8, 0),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_MOV64_IMM(BPF_REG_8, 1),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_get_prandom_u32),
+ BPF_MOV64_IMM(BPF_REG_9, 0),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
+ BPF_MOV64_IMM(BPF_REG_9, 1),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 4),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_8, 1, 1),
+ BPF_LDX_MEM(BPF_B, BPF_REG_1, BPF_REG_2, 0),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .prog_type = BPF_PROG_TYPE_XDP,
+ .flags = BPF_F_TEST_STATE_FREQ,
+ .errstr = "!read_ok",
+ .result = REJECT,
+},
+{
+ "precise: ST insn causing spi > allocated_stack",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_3, BPF_REG_10),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_3, 123, 0),
+ BPF_ST_MEM(BPF_DW, BPF_REG_3, -8, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_4, BPF_REG_10, -8),
+ BPF_MOV64_IMM(BPF_REG_0, -1),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .prog_type = BPF_PROG_TYPE_XDP,
+ .flags = BPF_F_TEST_STATE_FREQ,
+ .errstr = "5: (2d) if r4 > r0 goto pc+0\
+ last_idx 5 first_idx 5\
+ parent didn't have regs=10 stack=0 marks\
+ last_idx 4 first_idx 2\
+ regs=10 stack=0 before 4\
+ regs=10 stack=0 before 3\
+ regs=0 stack=1 before 2\
+ last_idx 5 first_idx 5\
+ parent didn't have regs=1 stack=0 marks",
+ .result = VERBOSE_ACCEPT,
+ .retval = -1,
+},
+{
+ "precise: STX insn causing spi > allocated_stack",
+ .insns = {
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_get_prandom_u32),
+ BPF_MOV64_REG(BPF_REG_3, BPF_REG_10),
+ BPF_JMP_IMM(BPF_JNE, BPF_REG_3, 123, 0),
+ BPF_STX_MEM(BPF_DW, BPF_REG_3, BPF_REG_0, -8),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_4, BPF_REG_10, -8),
+ BPF_MOV64_IMM(BPF_REG_0, -1),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .prog_type = BPF_PROG_TYPE_XDP,
+ .flags = BPF_F_TEST_STATE_FREQ,
+ .errstr = "last_idx 6 first_idx 6\
+ parent didn't have regs=10 stack=0 marks\
+ last_idx 5 first_idx 3\
+ regs=10 stack=0 before 5\
+ regs=10 stack=0 before 4\
+ regs=0 stack=1 before 3\
+ last_idx 6 first_idx 6\
+ parent didn't have regs=1 stack=0 marks\
+ last_idx 5 first_idx 3\
+ regs=1 stack=0 before 5",
+ .result = VERBOSE_ACCEPT,
+ .retval = -1,
+},
diff --git a/tools/testing/selftests/kvm/include/evmcs.h b/tools/testing/selftests/kvm/include/evmcs.h
index 4059014..4912d23 100644
--- a/tools/testing/selftests/kvm/include/evmcs.h
+++ b/tools/testing/selftests/kvm/include/evmcs.h
@@ -220,6 +220,8 @@ struct hv_enlightened_vmcs {
struct hv_enlightened_vmcs *current_evmcs;
struct hv_vp_assist_page *current_vp_assist;
+int vcpu_enable_evmcs(struct kvm_vm *vm, int vcpu_id);
+
static inline int enable_vp_assist(uint64_t vp_assist_pa, void *vp_assist)
{
u64 val = (vp_assist_pa & HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK) |
diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c
index 6cb34a0..0a5e487 100644
--- a/tools/testing/selftests/kvm/lib/x86_64/processor.c
+++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c
@@ -1060,9 +1060,11 @@ struct kvm_x86_state *vcpu_save_state(struct kvm_vm *vm, uint32_t vcpuid)
TEST_ASSERT(r == 0, "Unexpected result from KVM_GET_XSAVE, r: %i",
r);
- r = ioctl(vcpu->fd, KVM_GET_XCRS, &state->xcrs);
- TEST_ASSERT(r == 0, "Unexpected result from KVM_GET_XCRS, r: %i",
- r);
+ if (kvm_check_cap(KVM_CAP_XCRS)) {
+ r = ioctl(vcpu->fd, KVM_GET_XCRS, &state->xcrs);
+ TEST_ASSERT(r == 0, "Unexpected result from KVM_GET_XCRS, r: %i",
+ r);
+ }
r = ioctl(vcpu->fd, KVM_GET_SREGS, &state->sregs);
TEST_ASSERT(r == 0, "Unexpected result from KVM_GET_SREGS, r: %i",
@@ -1103,9 +1105,11 @@ void vcpu_load_state(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_x86_state *s
TEST_ASSERT(r == 0, "Unexpected result from KVM_SET_XSAVE, r: %i",
r);
- r = ioctl(vcpu->fd, KVM_SET_XCRS, &state->xcrs);
- TEST_ASSERT(r == 0, "Unexpected result from KVM_SET_XCRS, r: %i",
- r);
+ if (kvm_check_cap(KVM_CAP_XCRS)) {
+ r = ioctl(vcpu->fd, KVM_SET_XCRS, &state->xcrs);
+ TEST_ASSERT(r == 0, "Unexpected result from KVM_SET_XCRS, r: %i",
+ r);
+ }
r = ioctl(vcpu->fd, KVM_SET_SREGS, &state->sregs);
TEST_ASSERT(r == 0, "Unexpected result from KVM_SET_SREGS, r: %i",
diff --git a/tools/testing/selftests/kvm/lib/x86_64/vmx.c b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
index 204f847..9cef045 100644
--- a/tools/testing/selftests/kvm/lib/x86_64/vmx.c
+++ b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
@@ -12,6 +12,26 @@
bool enable_evmcs;
+int vcpu_enable_evmcs(struct kvm_vm *vm, int vcpu_id)
+{
+ uint16_t evmcs_ver;
+
+ struct kvm_enable_cap enable_evmcs_cap = {
+ .cap = KVM_CAP_HYPERV_ENLIGHTENED_VMCS,
+ .args[0] = (unsigned long)&evmcs_ver
+ };
+
+ vcpu_ioctl(vm, vcpu_id, KVM_ENABLE_CAP, &enable_evmcs_cap);
+
+ /* KVM should return supported EVMCS version range */
+ TEST_ASSERT(((evmcs_ver >> 8) >= (evmcs_ver & 0xff)) &&
+ (evmcs_ver & 0xff) > 0,
+ "Incorrect EVMCS version range: %x:%x\n",
+ evmcs_ver & 0xff, evmcs_ver >> 8);
+
+ return evmcs_ver;
+}
+
/* Allocate memory regions for nested VMX tests.
*
* Input Args:
diff --git a/tools/testing/selftests/kvm/x86_64/evmcs_test.c b/tools/testing/selftests/kvm/x86_64/evmcs_test.c
index f95c083..92915e6 100644
--- a/tools/testing/selftests/kvm/x86_64/evmcs_test.c
+++ b/tools/testing/selftests/kvm/x86_64/evmcs_test.c
@@ -79,11 +79,6 @@ int main(int argc, char *argv[])
struct kvm_x86_state *state;
struct ucall uc;
int stage;
- uint16_t evmcs_ver;
- struct kvm_enable_cap enable_evmcs_cap = {
- .cap = KVM_CAP_HYPERV_ENLIGHTENED_VMCS,
- .args[0] = (unsigned long)&evmcs_ver
- };
/* Create VM */
vm = vm_create_default(VCPU_ID, 0, guest_code);
@@ -96,13 +91,7 @@ int main(int argc, char *argv[])
exit(KSFT_SKIP);
}
- vcpu_ioctl(vm, VCPU_ID, KVM_ENABLE_CAP, &enable_evmcs_cap);
-
- /* KVM should return supported EVMCS version range */
- TEST_ASSERT(((evmcs_ver >> 8) >= (evmcs_ver & 0xff)) &&
- (evmcs_ver & 0xff) > 0,
- "Incorrect EVMCS version range: %x:%x\n",
- evmcs_ver & 0xff, evmcs_ver >> 8);
+ vcpu_enable_evmcs(vm, VCPU_ID);
run = vcpu_state(vm, VCPU_ID);
@@ -146,7 +135,7 @@ int main(int argc, char *argv[])
kvm_vm_restart(vm, O_RDWR);
vm_vcpu_add(vm, VCPU_ID);
vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
- vcpu_ioctl(vm, VCPU_ID, KVM_ENABLE_CAP, &enable_evmcs_cap);
+ vcpu_enable_evmcs(vm, VCPU_ID);
vcpu_load_state(vm, VCPU_ID, state);
run = vcpu_state(vm, VCPU_ID);
free(state);
diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_cpuid.c b/tools/testing/selftests/kvm/x86_64/hyperv_cpuid.c
index f72b3043..ee59831 100644
--- a/tools/testing/selftests/kvm/x86_64/hyperv_cpuid.c
+++ b/tools/testing/selftests/kvm/x86_64/hyperv_cpuid.c
@@ -18,6 +18,7 @@
#include "test_util.h"
#include "kvm_util.h"
#include "processor.h"
+#include "vmx.h"
#define VCPU_ID 0
@@ -106,12 +107,7 @@ int main(int argc, char *argv[])
{
struct kvm_vm *vm;
int rv;
- uint16_t evmcs_ver;
struct kvm_cpuid2 *hv_cpuid_entries;
- struct kvm_enable_cap enable_evmcs_cap = {
- .cap = KVM_CAP_HYPERV_ENLIGHTENED_VMCS,
- .args[0] = (unsigned long)&evmcs_ver
- };
/* Tell stdout not to buffer its content */
setbuf(stdout, NULL);
@@ -136,14 +132,14 @@ int main(int argc, char *argv[])
free(hv_cpuid_entries);
- rv = _vcpu_ioctl(vm, VCPU_ID, KVM_ENABLE_CAP, &enable_evmcs_cap);
-
- if (rv) {
+ if (!kvm_check_cap(KVM_CAP_HYPERV_ENLIGHTENED_VMCS)) {
fprintf(stderr,
"Enlightened VMCS is unsupported, skip related test\n");
goto vm_free;
}
+ vcpu_enable_evmcs(vm, VCPU_ID);
+
hv_cpuid_entries = kvm_get_supported_hv_cpuid(vm);
if (!hv_cpuid_entries)
return 1;
diff --git a/tools/testing/selftests/kvm/x86_64/platform_info_test.c b/tools/testing/selftests/kvm/x86_64/platform_info_test.c
index 40050e4..f9334bd 100644
--- a/tools/testing/selftests/kvm/x86_64/platform_info_test.c
+++ b/tools/testing/selftests/kvm/x86_64/platform_info_test.c
@@ -99,8 +99,8 @@ int main(int argc, char *argv[])
msr_platform_info = vcpu_get_msr(vm, VCPU_ID, MSR_PLATFORM_INFO);
vcpu_set_msr(vm, VCPU_ID, MSR_PLATFORM_INFO,
msr_platform_info | MSR_PLATFORM_INFO_MAX_TURBO_RATIO);
- test_msr_platform_info_disabled(vm);
test_msr_platform_info_enabled(vm);
+ test_msr_platform_info_disabled(vm);
vcpu_set_msr(vm, VCPU_ID, MSR_PLATFORM_INFO, msr_platform_info);
kvm_vm_free(vm);
diff --git a/tools/testing/selftests/kvm/x86_64/vmx_set_nested_state_test.c b/tools/testing/selftests/kvm/x86_64/vmx_set_nested_state_test.c
index ed7218d..853e370 100644
--- a/tools/testing/selftests/kvm/x86_64/vmx_set_nested_state_test.c
+++ b/tools/testing/selftests/kvm/x86_64/vmx_set_nested_state_test.c
@@ -25,24 +25,17 @@
#define VMCS12_REVISION 0x11e57ed0
#define VCPU_ID 5
+bool have_evmcs;
+
void test_nested_state(struct kvm_vm *vm, struct kvm_nested_state *state)
{
- volatile struct kvm_run *run;
-
vcpu_nested_state_set(vm, VCPU_ID, state, false);
- run = vcpu_state(vm, VCPU_ID);
- vcpu_run(vm, VCPU_ID);
- TEST_ASSERT(run->exit_reason == KVM_EXIT_SHUTDOWN,
- "Got exit_reason other than KVM_EXIT_SHUTDOWN: %u (%s),\n",
- run->exit_reason,
- exit_reason_str(run->exit_reason));
}
void test_nested_state_expect_errno(struct kvm_vm *vm,
struct kvm_nested_state *state,
int expected_errno)
{
- volatile struct kvm_run *run;
int rv;
rv = vcpu_nested_state_set(vm, VCPU_ID, state, true);
@@ -50,12 +43,6 @@ void test_nested_state_expect_errno(struct kvm_vm *vm,
"Expected %s (%d) from vcpu_nested_state_set but got rv: %i errno: %s (%d)",
strerror(expected_errno), expected_errno, rv, strerror(errno),
errno);
- run = vcpu_state(vm, VCPU_ID);
- vcpu_run(vm, VCPU_ID);
- TEST_ASSERT(run->exit_reason == KVM_EXIT_SHUTDOWN,
- "Got exit_reason other than KVM_EXIT_SHUTDOWN: %u (%s),\n",
- run->exit_reason,
- exit_reason_str(run->exit_reason));
}
void test_nested_state_expect_einval(struct kvm_vm *vm,
@@ -90,8 +77,9 @@ void set_default_vmx_state(struct kvm_nested_state *state, int size)
{
memset(state, 0, size);
state->flags = KVM_STATE_NESTED_GUEST_MODE |
- KVM_STATE_NESTED_RUN_PENDING |
- KVM_STATE_NESTED_EVMCS;
+ KVM_STATE_NESTED_RUN_PENDING;
+ if (have_evmcs)
+ state->flags |= KVM_STATE_NESTED_EVMCS;
state->format = 0;
state->size = size;
state->hdr.vmx.vmxon_pa = 0x1000;
@@ -141,13 +129,19 @@ void test_vmx_nested_state(struct kvm_vm *vm)
/*
* Setting vmxon_pa == -1ull and vmcs_pa == -1ull exits early without
* setting the nested state but flags other than eVMCS must be clear.
+ * The eVMCS flag can be set if the enlightened VMCS capability has
+ * been enabled.
*/
set_default_vmx_state(state, state_sz);
state->hdr.vmx.vmxon_pa = -1ull;
state->hdr.vmx.vmcs12_pa = -1ull;
test_nested_state_expect_einval(vm, state);
- state->flags = KVM_STATE_NESTED_EVMCS;
+ state->flags &= KVM_STATE_NESTED_EVMCS;
+ if (have_evmcs) {
+ test_nested_state_expect_einval(vm, state);
+ vcpu_enable_evmcs(vm, VCPU_ID);
+ }
test_nested_state(vm, state);
/* It is invalid to have vmxon_pa == -1ull and SMM flags non-zero. */
@@ -232,6 +226,8 @@ int main(int argc, char *argv[])
struct kvm_nested_state state;
struct kvm_cpuid_entry2 *entry = kvm_get_supported_cpuid_entry(1);
+ have_evmcs = kvm_check_cap(KVM_CAP_HYPERV_ENLIGHTENED_VMCS);
+
if (!kvm_check_cap(KVM_CAP_NESTED_STATE)) {
printf("KVM_CAP_NESTED_STATE not available, skipping test\n");
exit(KSFT_SKIP);
diff --git a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py
index affa7f2..9539cff 100644
--- a/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py
+++ b/tools/testing/selftests/tc-testing/plugin-lib/nsPlugin.py
@@ -64,7 +64,7 @@
cmdlist.insert(0, self.args.NAMES['NS'])
cmdlist.insert(0, 'exec')
cmdlist.insert(0, 'netns')
- cmdlist.insert(0, 'ip')
+ cmdlist.insert(0, self.args.NAMES['IP'])
else:
pass
@@ -78,16 +78,16 @@
return command
def _ports_create(self):
- cmd = 'ip link add $DEV0 type veth peer name $DEV1'
+ cmd = '$IP link add $DEV0 type veth peer name $DEV1'
self._exec_cmd('pre', cmd)
- cmd = 'ip link set $DEV0 up'
+ cmd = '$IP link set $DEV0 up'
self._exec_cmd('pre', cmd)
if not self.args.namespace:
- cmd = 'ip link set $DEV1 up'
+ cmd = '$IP link set $DEV1 up'
self._exec_cmd('pre', cmd)
def _ports_destroy(self):
- cmd = 'ip link del $DEV0'
+ cmd = '$IP link del $DEV0'
self._exec_cmd('post', cmd)
def _ns_create(self):
@@ -97,16 +97,16 @@
'''
self._ports_create()
if self.args.namespace:
- cmd = 'ip netns add {}'.format(self.args.NAMES['NS'])
+ cmd = '$IP netns add {}'.format(self.args.NAMES['NS'])
self._exec_cmd('pre', cmd)
- cmd = 'ip link set $DEV1 netns {}'.format(self.args.NAMES['NS'])
+ cmd = '$IP link set $DEV1 netns {}'.format(self.args.NAMES['NS'])
self._exec_cmd('pre', cmd)
- cmd = 'ip -n {} link set $DEV1 up'.format(self.args.NAMES['NS'])
+ cmd = '$IP -n {} link set $DEV1 up'.format(self.args.NAMES['NS'])
self._exec_cmd('pre', cmd)
if self.args.device:
- cmd = 'ip link set $DEV2 netns {}'.format(self.args.NAMES['NS'])
+ cmd = '$IP link set $DEV2 netns {}'.format(self.args.NAMES['NS'])
self._exec_cmd('pre', cmd)
- cmd = 'ip -n {} link set $DEV2 up'.format(self.args.NAMES['NS'])
+ cmd = '$IP -n {} link set $DEV2 up'.format(self.args.NAMES['NS'])
self._exec_cmd('pre', cmd)
def _ns_destroy(self):
@@ -115,7 +115,7 @@
devices as well)
'''
if self.args.namespace:
- cmd = 'ip netns delete {}'.format(self.args.NAMES['NS'])
+ cmd = '$IP netns delete {}'.format(self.args.NAMES['NS'])
self._exec_cmd('post', cmd)
def _exec_cmd(self, stage, command):
diff --git a/virt/kvm/arm/mmio.c b/virt/kvm/arm/mmio.c
index a8a6a0c..6af5c91 100644
--- a/virt/kvm/arm/mmio.c
+++ b/virt/kvm/arm/mmio.c
@@ -86,6 +86,12 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run)
unsigned int len;
int mask;
+ /* Detect an already handled MMIO return */
+ if (unlikely(!vcpu->mmio_needed))
+ return 0;
+
+ vcpu->mmio_needed = 0;
+
if (!run->mmio.is_write) {
len = run->mmio.len;
if (len > sizeof(unsigned long))
@@ -188,6 +194,7 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
run->mmio.is_write = is_write;
run->mmio.phys_addr = fault_ipa;
run->mmio.len = len;
+ vcpu->mmio_needed = 1;
if (!ret) {
/* We handled the access successfully in the kernel. */
diff --git a/virt/kvm/arm/vgic/vgic-init.c b/virt/kvm/arm/vgic/vgic-init.c
index bdbc297..e621b5d 100644
--- a/virt/kvm/arm/vgic/vgic-init.c
+++ b/virt/kvm/arm/vgic/vgic-init.c
@@ -8,6 +8,7 @@
#include <linux/cpu.h>
#include <linux/kvm_host.h>
#include <kvm/arm_vgic.h>
+#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
#include "vgic.h"
@@ -164,12 +165,18 @@ static int kvm_vgic_dist_init(struct kvm *kvm, unsigned int nr_spis)
irq->vcpu = NULL;
irq->target_vcpu = vcpu0;
kref_init(&irq->refcount);
- if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) {
+ switch (dist->vgic_model) {
+ case KVM_DEV_TYPE_ARM_VGIC_V2:
irq->targets = 0;
irq->group = 0;
- } else {
+ break;
+ case KVM_DEV_TYPE_ARM_VGIC_V3:
irq->mpidr = 0;
irq->group = 1;
+ break;
+ default:
+ kfree(dist->spis);
+ return -EINVAL;
}
}
return 0;
@@ -209,7 +216,6 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
irq->intid = i;
irq->vcpu = NULL;
irq->target_vcpu = vcpu;
- irq->targets = 1U << vcpu->vcpu_id;
kref_init(&irq->refcount);
if (vgic_irq_is_sgi(i)) {
/* SGIs */
@@ -219,11 +225,6 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
/* PPIs */
irq->config = VGIC_CONFIG_LEVEL;
}
-
- if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
- irq->group = 1;
- else
- irq->group = 0;
}
if (!irqchip_in_kernel(vcpu->kvm))
@@ -286,10 +287,19 @@ int vgic_init(struct kvm *kvm)
for (i = 0; i < VGIC_NR_PRIVATE_IRQS; i++) {
struct vgic_irq *irq = &vgic_cpu->private_irqs[i];
- if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3)
+ switch (dist->vgic_model) {
+ case KVM_DEV_TYPE_ARM_VGIC_V3:
irq->group = 1;
- else
+ irq->mpidr = kvm_vcpu_get_mpidr_aff(vcpu);
+ break;
+ case KVM_DEV_TYPE_ARM_VGIC_V2:
irq->group = 0;
+ irq->targets = 1U << idx;
+ break;
+ default:
+ ret = -EINVAL;
+ goto out;
+ }
}
}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 44efc2f..0d09048 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -211,6 +211,12 @@ static void vgic_hw_irq_spending(struct kvm_vcpu *vcpu, struct vgic_irq *irq,
vgic_irq_set_phys_active(irq, true);
}
+static bool is_vgic_v2_sgi(struct kvm_vcpu *vcpu, struct vgic_irq *irq)
+{
+ return (vgic_irq_is_sgi(irq->intid) &&
+ vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2);
+}
+
void vgic_mmio_write_spending(struct kvm_vcpu *vcpu,
gpa_t addr, unsigned int len,
unsigned long val)
@@ -223,6 +229,12 @@ void vgic_mmio_write_spending(struct kvm_vcpu *vcpu,
for_each_set_bit(i, &val, len * 8) {
struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+ /* GICD_ISPENDR0 SGI bits are WI */
+ if (is_vgic_v2_sgi(vcpu, irq)) {
+ vgic_put_irq(vcpu->kvm, irq);
+ continue;
+ }
+
raw_spin_lock_irqsave(&irq->irq_lock, flags);
if (irq->hw)
vgic_hw_irq_spending(vcpu, irq, is_uaccess);
@@ -270,6 +282,12 @@ void vgic_mmio_write_cpending(struct kvm_vcpu *vcpu,
for_each_set_bit(i, &val, len * 8) {
struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+ /* GICD_ICPENDR0 SGI bits are WI */
+ if (is_vgic_v2_sgi(vcpu, irq)) {
+ vgic_put_irq(vcpu->kvm, irq);
+ continue;
+ }
+
raw_spin_lock_irqsave(&irq->irq_lock, flags);
if (irq->hw)
diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c
index 96aab77..b00aa30 100644
--- a/virt/kvm/arm/vgic/vgic-v2.c
+++ b/virt/kvm/arm/vgic/vgic-v2.c
@@ -184,7 +184,10 @@ void vgic_v2_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr)
if (vgic_irq_is_sgi(irq->intid)) {
u32 src = ffs(irq->source);
- BUG_ON(!src);
+ if (WARN_RATELIMIT(!src, "No SGI source for INTID %d\n",
+ irq->intid))
+ return;
+
val |= (src - 1) << GICH_LR_PHYSID_CPUID_SHIFT;
irq->source &= ~(1 << (src - 1));
if (irq->source) {
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 0c653a1..a4ad431 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -167,7 +167,10 @@ void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr)
model == KVM_DEV_TYPE_ARM_VGIC_V2) {
u32 src = ffs(irq->source);
- BUG_ON(!src);
+ if (WARN_RATELIMIT(!src, "No SGI source for INTID %d\n",
+ irq->intid))
+ return;
+
val |= (src - 1) << GICH_LR_PHYSID_CPUID_SHIFT;
irq->source &= ~(1 << (src - 1));
if (irq->source) {
diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c
index 13d4b38..e7bde65 100644
--- a/virt/kvm/arm/vgic/vgic.c
+++ b/virt/kvm/arm/vgic/vgic.c
@@ -254,6 +254,13 @@ static int vgic_irq_cmp(void *priv, struct list_head *a, struct list_head *b)
bool penda, pendb;
int ret;
+ /*
+ * list_sort may call this function with the same element when
+ * the list is fairly long.
+ */
+ if (unlikely(irqa == irqb))
+ return 0;
+
raw_spin_lock(&irqa->irq_lock);
raw_spin_lock_nested(&irqb->irq_lock, SINGLE_DEPTH_NESTING);